You're missing out ! I started doing it when I realized that a book is nothing sacred, just thoughts the author where having. So by taking notes, it's like having a conversation with her. It has the benefit of also teaching you to be more "skeptical" of what you are reading, always a plus.
I'm probably a weirdo, but the "write in books" is a massive no no for me. I can't imagine writing in a book (unless it's a total turd, but I assume we're talking about good, insightful books).
I do take notes, but always on separate sheets that I keep in the book. Just no way I'll write on the book itself.