I think this is an interesting issue. Personally, I don't ever recall being given abuse as a result of my birding, although I have noticed that people often look at me with a very puzzled, almost intimidated, expression. I do a lot of my birding by public transport, so often end up carrying bins/tripod etc through built up areas. Even in relatively gentrified areas, people still obviously stop and stare. I remember going for the Amwell Little Butning last month, and a fairly elderly man stopped walking and stared at with me with what looked like an expression of genuine horror, which I thought was odd seeing as it seemed a relatively affluent area near a nature reserve! It may because I'm (relatively) young, quite skinny, and probably don't fit most peoples' conception of a bird/nature person should look like.
But more generally, as someone up thread correctly said, it's amazing how much anxiety unconventional behaviour can generate, and birding is not *that* unconventional. And yet even so, it habitually generates amusement, confusion and sometimes mild hostility. I think the confusion around birding is symptomatic of a more general uneasiness that the great British public seem to have with anything that seems remotely unconventional.