You are certainly not the bad guy here, and to pretend that the bottles didn't exist would simply be enabling his alcohol use - I'm assuming you found them by accident, and not because you went looking for them (which IS codie!). You can, with confidence, boot all that guilt out of the window.
A sense of responsibility for him, and the resulting guilt, has kept you chained to this loser so far, and will continue to do so for as long as you let it. Addicts and alcoholics typically latch on to very conscientious, giving, guilt-ridden people because they are very easy to manipulate.
When you say "I also let him know that I was not living my life living with someone that was not honest and that I could not trust and I had just about reached my breaking point", yet continue to do so, you are unintentionally giving the message that his behaviour's OK. He will be looking at what you actually DO, rather than what you say. On the subject of trust, the one thing you can reliably trust an addict/alcoholic to do is use, or drink. Anything beyond that is unrealistic, and is setting yourself up for a merry-go-round of raised hopes then bitter disappointment.
I do not know how I'd handle your situation. I might just think what he would do if, say, I was injured, killed or incapacitated tomorrow - would he find a way of coping?