I'll admit that I sometimes forget I'm wearing CU gear, and I'll be focusing on whatever problem of the day in my head - then someone shouts "go Buffs!" at me. My first inclination in that situation is, in fact, confusion. My next reaction is an internal "well, yeah, go Buffs." And, however long later, I realize that I'm wearing a shirt or hat and then I finally realize what's going on.
If I haven't missed the moment, I'll return a "go buffs!" - but if it's a quick passing and I've missed the moment, then yeah, "I shouted 'go buffs!' and all I got was a confused look," could very well be me you were shouting it too.
But, to answer the initial post, the best place I've gotten a "Go Buffs" so far was on La Rambla in Barcelona.