I think everyone's got it so far. Fast shutter speed, higher ISO if needed, lots of patience, and fire away as many frames as you can. Even a good photographer probably needs dozens or hundreds of shots to get one good one...versus maybe 1 in 4 or 1 in 5 for a larger bird. Good panning technique, possibly trying prefocus, especially when dealing with the small birds who love to follow patterns. Observe for a while, and often you'll find them circling through the same area several times, so you might be able to stay prefocused for that spot or distance, and be ready for when they come back that way again. Spray and prey, sadly, is the order of the day!
BTW - if you're lucky enough to catch them against a blue sky or neutral background, you can likely as not get away with multipoint continuous AF and the camera should have no problem finding the little dark bird...but get him in front of some cluttered background, and it gets a lot harder to AF - the multipoint systems often have too much of interest to latch onto other than your birdie, and the spot focus is often too much a challenge to keep the spot on the fast and frenetic little guy. I often try manual focus in these situations...and usually do try a little prefocus - I pick an object at roughly the focal distance where I've seen the bird flying, somewhere ahead of him, then as he moves into the area I am trying to catch him, I'll start panning and he's a little easier to find since your focus is probably pretty close...slight adjustments while firing off a burst are easy, since in manual the camera doesn't need to wait for focus confirm to shoot. I'll sometimes run through a range of focus, hoping that somewhere in the sequence I might have landed a proper focus.
I need much more patience and practice with little birds in flight - my biggest impediment is weather - so far, I've mostly tried to shoot little birds this summer, when temps are running close to 100 f and humidity nearly the same...so I just don't have the patience or endurance to stand there taking 200 frames...I try a single burst of 15-20 frames, then walk away to the next shot - only knowing when I got home if any of them worked. This winter, in cooler weather, hopefully I'll get more chances.