More Consistent Standards Needed
Lawrence M Walsh, CEO and chief analyst at New York-based business strategy firm the 2112 Group, said independent third-party testing by media and/or labs is an important part of the security industry, but that the market needs objective reviews to provide customers with validation or relative expectations for how products will perform when deployed. He said:
The problem is we, as an industry, lack a common testing baseline. We need a set of common standards that everyone can expect labs to follow. This doesn’t necessarily hamstring testers, as they can improve upon the standards and add their own secret sauce. A standard should set minimum expectations, and that’s something sorely needed in security product testing.
At the same time, he said, vendors shouldn’t be allowed to dictate how the standards are written:
ICSA Labs collaborates with vendors on its testing standards, though it remains the final arbiter of the test criteria. That always bothered me, because I felt the vendors could influence the criteria to narrow the parameters to avoid truly horrific results. I don’t think vendors can be excluded from the standards process, but they can’t have control or too much say in it, either. ICSA Labs did a good job separating the two, but can we say the same for all? Testing should reflect real-world conditions, not just ideal circumstances under controlled conditions.
> Read the full article, What makes for truly independent security product testing?, at nakedsecurity.sophos.com