Google "landlor tenant hotline" for your state and give them a call. Sounds like your slime lord is an a$$. I would Give him a certified letter stating that he mentioned you were not responsible for the mold, and that you will take legal action to retrieve your deposit. If he is found to be wrong and your deposit is not returned within 14 days; you are entitled to double that according to the Landlord Tenants act. Let him know you know this. Hopefully he'll roll over. Again; I would run this by a state specific RE hotline as I'm not sure if there are any state laws that apply. My hunch is that this guy does this to everyone. You could threaten to report him to the BBB, post on local websites. Sounds like you already made a decision to call him on his bluff!