Listen to you to instincts - what are they telling you? Your feelings are not right or wrong. They are yours and you are entitled to them.
Your bf has the same rights, it just up to you whether his actions are acceptable or not for you! Only you can decide that.