I used to work in vitamin shop. Melatonin was never promoted as a calming thing. If I recall what it does is trick your brain into thinking it's time to sleep but does not alter the kind of sleep you'll get. People take it when they travel to reset their sleep schedule. I used it for a while when my work schedule changed, and I didn't experience anything relaxing about it. It's not something you take throughout the day.
If you're thinking supplements, ask your psychiatrist if they have any advice on GABA or kava. You want to ask them because GABA interacts with some drugs, as does kava I think. But those are what I recall being "recommended but not FDA approved."