Yes, it's important to make sure your firewall program is always running, but some would argue that its not the job of the firewall program to do that.
-cthorpe
Any security system that can be terminated without authorization is
no security system at all. I really don't see how anyone can seriously advance the argument that it's not the responsibility of a security product to protect itself. That would be like having the police tell you they'll protect you, but only if the criminals say it's OK, otherwise they're leaving.
In my opinion, the firewalls that have branched out into HIPS functions like process termination protection can be more trouble than they are worth as they have to dig themselves so deep into the OS to provide those functions.
-cthorpe
Very true. But heuristic-based subsystems do have their place in a security plan. Part of the problem is that they're attempting to protect an operating system that is fundamentally insecure. HIPS may be yet another case of "putting lipstick on the pig," but until somebody in Redmond decides to audit and (potentially) rewrite something like 50+ million lines of source code, I'd like to have all options available.
That set of tests and the results posted are misleading.
-cthorpe
I am not all a fan of Matousec, because I don't fully trust his methods to be as relevant, as he would like us to think, but I don't think his affiliate deals are a problem, on this matter. One is a magazine, hakin9, and the other is the Anti Rootkit site. How can such affiliates be a problem?
-Curt
I think both of you have brought up a
very valuable point.
It is always a good idea to treat any product comparison as "food for thought" rather than revealed truth. In short, take them for what they're worth - somebody's opinion.
Any "shootout" or "challenge" is based on a set of criteria. And there are as many opinions about how to test things as there are people to do the testing. There are a good number of firewall comparison tests up on the web - many legitimate, others less so. You should try to read and understand as many as possible rather than accept any one at face value. Some testing bias will always be inevitable. Legitimate studies go to great lengths to keep it to a minimum.
I don't believe the Matousec report is unduly biased to favor their affiliate's products. Relative rankings aside, their top choices pretty much reflect industry consensus. So they're not alone in their opinion. Nor do I believe they're guilty of a seriously flawed testing methodology. Even experts will disagree on how best to evaluate firewall technology.
I do think Matousec deserves some credit for being very up front about their test methodology and criteria. That's one of the reasons why someone can have an intelligent basis for disagreeing with their conclusions!
(Disclaimer: I'm not affiliated with Matousec or any other security product or service. However, if Samuel Adams, Fender, or Ampeg would be interested in talking to me, I'm available!) 
