My mother has Asperger's. She's 85, and also has Alzheimer's now, which is an interesting mix to say the least.
I always knew that my mom was "off"--we all did. She was in her "own little world" most of the time, pursuing the focused interests that most people with Asperger's have, which definitely interfered with her parenting. And while she wasn't intentionally mean or vindictive, she was socially unaware--so she often said tactless and hurtful things but could not understand why others were upset. To her, if her comments were empirically true there was no reason she shouldn't say them.
I have mixed feelings about being raised by a person with Aspergers. For the most part it was difficult, because she was truly "different", very detached, and often behaved in ways that made me think she did not care about me. It would have helped if I had known that her behavior had nothing to do with me, that it was caused by the Aspergers, but we had no diagnosis until I was well into middle age. When I did find out, I had a huge sense of relief that there was a reason for her behavior--other than me being unloveable.