There's absolutely nothing wrong with being a reader, Adam. I, too, enjoy reading, but when drinking, I just don't do it, either because I'm passed out, or realize that I'll never remember what I read, anyway.
If reading is what you love doing, then I say read away! Who cares what others think!