I think it's supposed to indicate that most people in Cardiff have heard of Torchwood, but nobody really knows what they do so they're seen as an annoyance. They never get the credit they deserve but they get blamed for a lot of things.

Of course that makes Gwen seem even more stupid because everyone knows about Torchwood except her.