Maybe the question here is not whether it should be a concern, but whether it should be your concern. It sounds like you might be getting yanked into something that is really not your business. Detachment can be really hard, especially when those around you don't help you with it.
There is really no point in theorizing whether his facial swelling is the result of his drinking, a bad night's sleep, or too many salty snacks. If he is truly your ex, it isn't really your problem anymore.