I felt that way too the first few times I went out with friends without my A. But it got a little bit easier every time. You deserve to have some fun without worrying about him. Do it whenever you have a chance, in a little while it will get to feel natural, as it should.