Yes, he's immature.
Yes, he's selfish.
If he's not really interested in seeing the kids, then keep them to yourself. They need mom, especially at these young ages. If he doesn't "get it" you can't force him to magically wake up.
What kind of boundaries do you want?
And how old is he? He sounds young or very immature. (No offense to the young peeps out there who are VERY mature and I know there are a good amount out there.)
And finally (most important) what is best for the kids? What is he teaching them by only half paying attention?