Might also depend a bit on how you're panning with the bird. If the bird is moving through your frame, and you are just dipping down to the horizon with the bird, the relative lack of movement in the frame is going to likely cause the background detail to take over the focusing job, especially if the bird is a small part of the frame. However, I find if you are panning with a bird fairly quickly with a long focal length, I can leave all focus points active with continuous focus mode - as you pan with the bird, the background is rendered mostly blur and the camera can't pick up details, whereas the bird if being well panned stays relatively still in relation to the frame, keeping the focus on him.
Cameras will vary as well - some AF systems have more advanced subject tracking capabilities, and some lenses will be faster in focusing along with a closing subject. But even with a basic entry level camera and average focus system, it should still be possible to keep focus on a moving bird by panning quickly with it (obviously if he's coming head-on at you, then he's not varying position enough to the background, and multiple focus points will struggle - in those scenarios, switching to a single focus point or smaller group of central points would be better).