Let's stick to college ball here shall we. I say this because they are two different types of fans. Bronco nation kind of dwarfs Buffs football I think is a fair way of looking at it.
Everytime the Buffs play someone at home and let's say a crowd of 38,000 or so shows up, what are the excuses we hear? Same thing you just said about so cal. People don't care, they are outdoors doing something or so the urban legend goes, people move to Colorado for the outdoors. That's what CU football is competeing against not the opponent of the day. How does CU get that person to come to the game verses whatever was the pull to Colorado be it hiking, photography, biking, fishing, camping, see where I'm going with this. This is how it is and always has been for the Buffs.
I'm not busting on you or anyone else about this. Other than expansion I'm liking what's being said, and wish AllBuffs had an envoy to the athletic Dept. they could use some of you on here.
You can't ignore the Bronco problem because it contributes to the issues you describe. A lot of football fans in Colorado do their "other stuff" (hiking, skiing, camping, fishing, etc.) on Saturday, so that they can tailgate and watch the Broncos on Sunday. I think people in Colorado do care about football, they just currently care about the Broncos more.