I have a number of beautiful photos of small birds coming to a pool to bath and drink, while I sat about a metre from the pool, making no effort at all to conceal myself. I think the behaviour of the creature they see is definitely more important than the clothing.
I have a bright red shirt, and some of the best of my photos without a hide, have been taken while wearing it. On the other hand, I have frequently frightened birds away when taking photos from the same position, but concealed by a well camouflaged green tent hide. The slightest movement will cause them to flee while I am hidden, but out in the open, I can very slowly raise a hand to scratch my nose without daunting them.
I'm only guessing, because I couldn't possibly know what a bird was thinking, but I have a theory that my red shirt tells the small birds that I have no reason to attempt to hide from them. Which would also tell them, of course, that I am not a predator. On the other hand, a furtive movement from a place of concealment (i.e. the tent) is predator behaviour, and if they detect it, they will flee. I also get the distinct impression that these small birds are able to see far better through finely woven garden mesh than humans can. Which brings us back to what fugl said about whether camouflage that works for humans also works for a bird's eyesight - which is very much superior to our own.