HermitIbis makes some good points here.
I would also add the question of whether you want to be a birder who goes out and carries a camera for opportune situations to capture images or whether you are more a photographer of birds where you go out with intention of getting some images. Of course, both could apply at different times as they do for me.
I therefore choose 100-400 zoom. There are times where a bigger lens would come in handy but I think this would always be the case as I find I'm often wanting to get closer than the situation would allow. In some cases I carry digiscope camera as well for some opportune record shots where this might be true.
If I am honest though, lugging a big lens and tripod around is not for me.
Another factor to consider is if you have a reasonable DSLR camera that allows cropping of your image when processing, you can still get some good detailed images. My Canon 7D falls into this category as it has a crop sensor.
Lastly, expectations are important and if you want head shots of birds or images that are publishable quality every time, then indeed be patient. :-)
Check my Flickr link - virtually all taken with 400mm lens and no more. No prize winners assuming I would want this, but they work for me.