They'll probably be gone by the end of the first week of June. Depending on where you are in the UK, it could be anytime from very end of May to middle of June.
Have a peek in the box now, when the birds are away. They've probably only just started laying (or may still be building the nest). Is the nest lined with feathers and/or hair/wool? Can you see any eggs? Have a very gentle feel at the bottom of the nest cup, as the first eggs are buried until incubation starts. How many eggs? If none, but the nest is lined, they'll probably have started laying within a week. If there are eggs, then they'll be laying one a day. Assume there are 10 eggs, so that's 10 days for laying. Then it's 13 days for incubation. Then 3 weeks for the chicks to fledge. So that's 7 weeks to be safe, from the date of the first egg.
Just remember not to keep checking aroudn fledging time to see if they've gone. You could make them fledge early if they're disturbed at that stage. You'll know when they've left, as the parents will not be around bringing in food and there will be no noise from chicks in the box. Give it another day to make sure they've stopped coming, then have a quick look. Once they've left, they wont come back to roost or anything, so you can get on with your job.