Like many have said, viewing conditions and the type of birding you do first and foremost determine what is going to be the useful magnification range, and that in turn determines what is the optimum scope aperture.
I largely agree with Alexis's advice when it comes to birders who live in the continental US or central Europe and who don't do a lot of sea watching, winter birding or other long distance stuff. However, as a birder living in and birding where the sun rises and sets slowly and never gets that high in the sky, I get stable air a bit more often, often view over great distances, and consequently prioritize larger apertures and higher magnifications than most of the posters here.
For my kind of birding, size definitely matters, and from the way edj. phrases his question I take it that he does not mean whether size matters for portability or cost, which are sort of obvious points, but for image quality and ability to ID birds. For this latter, my history with scopes started with a mediocre 60 mm scope (Mirador) with a fixed 27x eyepiece but on a pretty good tripod, and evolved to the same ED 78 A Nikon Alexis uses, first with a 38x Wide eyepiece and later with a 25-75x zoom. The step up from the 60 to 78 together with a large jump in optical quality (ED glass, much better corrected optics) was quite dramatic, and the bigger scope gave hugely better views under every and all conditions.
My next scope was a Nikon ED 82 A, which was a worthwhile step up from the 78, but not particularly dramatic. Here, most of the improvement came from the 82 being a lower-aberration sample than the 78, and the rest of the improvement came from better contrast, brightness and less stray light problems due to improved baffling and multi coatings in the newer 82. The 4 mm increase in aperture does help in resolving power, but this is a step small enough to easily vanish among other variables, such as sample differences.
My present scope is a 95, and it also happens to be better corrected for optical aberrations than any of my previous scopes, or, for that matter, just about any other scope I have ever tested. This one is quite a bit better than the 82 or 78, both at lower magnifications and high. A well-corrected large scope provides a better image also in poor seeing, and the difference in image quality is obvious even at magnifications of 30x or so. We rather often get air stable enough that I get a visible benefit from the maximum magnifications of my zoom (72x), and this is not just at early mornings or evenings or overcast days, but also during passing clouds on partly cloudy days.
I just acquired a dedicated teleconverter for my scope that gives it a magnification range of 50-120x. For birding, with a 95 mm scope, I still think that 120x is probably well over the sensible diminishing returns threshold, but my initial experience has been that I will be using mags up to at least 100x for actual benefit at times. The scope with the extender weighs close to 2.5 kg and sits on a pretty hefty CF tripod with a good fluid head, but with a scope harness it is nicely portable. This setup gives me two options: 50-120x for good to excellent seeing and large distances, and 30-72x for mediocre to poor seeing. Changing between the two setups takes less than a minute, and both are equally easy to use.
Kimmo