If I knew no one else would ever use my computer or have any direct access to it, I'd feel really confident about not having anti-virus software running in the background.
But if you have a shared computer or a computer on a network, and some of the people you share your computer/network with are somewhere on the verge of computer literacy, I'd keep it running.
Most people know enough to download screensavers, games, and other crap, and how to install that crap. What they don't know is how to recognize dubious sites, dubious programs, dubious attachments, etc.
One example is when I found my computer infected by a virus, I knew I hadn't infected my own computer, so I asked my mom to scan hers. Turns out my mother had opened an attachment on her computer, it didn't do anything (or so she thought) so she ignored it, and it infected the rest of our computers because we had mapped network drives for easy file sharing across computers.
You're only as secure as your weakest defense.