The Rufous's unique combination of long migration and short nesting season makes it seem as though they nest well south of their actual breeding range. Here in southeastern Arizona, there's only six to seven weeks between the last northbound Rufous (early May) and the earliest southbound birds (late June). The further north you go, the smaller the "gap" would be. They're usually common in the southern Rockies by late July, when the similar-looking female Broad-taileds are still trying to nest.
Based on banding studies, the average Rufous spends less than a week at any one stopover site, so there's a lot of turnover through the migration season!