Let me say first of all that Don Imus is an insensitive and outrageous person, that he sets out to be, and that he's made a career out of saying things that people will find offensive.
Let me say second of all that I'm not a fan of Don Imus.
Now let me admit that these two things have nothing to do with each other. Let me further admit that I am a fan of a show that routinely says things that would get Imus not only fired, but severely fined, and probably hunted down and beaten to death by an angry mob as well. And yes, I often find this show offensive as well. But most of the time it's entertaining.
Not only do I support the right of people to say objectionable things, I believe it's much better to have them saying those things than to have them thought but not spoken.
To give just one example: Would people who have decided to boycott Imus because he's a racist/sexist bastard prefer it if they were still supporting a racist/sexist bastard out of ignorance?
(I refer to Imus as a racist/sexist bastard in a hypothetical sense, here: As far as I can tell, he's mostly just clueless.)
My position on this is primarily informed by my libertarian instincts, particularly on censorship. I'm against it. I'm opposed to having a set of sensibilities, be they individual or collective, forcibly imposed on anybody.
Fat hairy men walking down the street wearing nothing but sandals? Putting Prince on television? Voting for George W. Bush in 2004? I find these things offensive, yes. I think they're an affront to decency. I wouldn't advocate that anyone be punished by the government just for offending me, though. In fact, I tend to think people who do these things will tend to suffer for their behavior without being directly punished.
When offensive conduct becomes destructive conduct, we can talk about getting the government involved. But it's the destructive element, not the offensive element, that controls. Child pornography isn't criminal because it's porn, it's criminal because of what's being done to the child. Advocating violence isn't criminal because of the opinions expressed, it's criminal if it's likely to cause people to do violence.
Decency isn't something a government can impose. It's learned behavior. And if we're always sheltered from indecent acts, we'll never recognize the line.