I think it really boils down to the expectations you set for yourself and your goals kevlar. "Not fitting in" all happens inside your head. It really has nothing to do with other people or what they think/feel/know -those are things completely beyond your control anyway.
Have you ever considered counseling? One of the things I learned from my therapist was the concept of mindfulness and meditation. One of the key things mindfulness does is to allow you to do exactly what you can't seem to do - just let things go. One of the analogies I love is the "river of thoughts". Imagine yourself standing by a swiftly flowing river or stream. All the water is your thoughts - and they will never stop flowing just like the water in the river will keep going. One of the choices you have is to simly stand on the bank and watch the thoughts flow. Or you can put your toes in the water and deal with a little bit of it at a time, and they will still be there when you come back next tiem. If you decide to dive in head-first to try and deal with them all at once, you will likely be drowned - just like in real life.