How “Real Life” Reinforces the Need for a Resiliency Score
by Kristi ThieleMost industries have policies in place to help the customer or consumer feel more confident—and know more—about the quality of the product they are buying. Often, this requires some kind of rating, certification, or examination of a product or service against a well-defined set of standards. Such ratings or scores allow the customer or consumer to make an informed decision and understand any risk that might be associated with their selection.
For example, as a Systems Engineer at BreakingPoint, I have the opportunity to fly to different parts of the country and the world. When selecting a place to eat, there are several conscious and unconscious decisions that happen in my head. For example, I might use my iPhone and the UrbanSpoon application to see what others have said about a restaurant. If no one has indicated that they like it, then I accept the responsibility if I select that place.
At the same time, I realize that each person is entitled to their own opinion and I take an UrbanSpoon score or review for what it is—an opinion that has no one regulating or monitoring the criteria for the scoring. However, I may put more weight into a review by a well-known critic, since they have a reputation to uphold and will attempt to use similar criteria every time they review a restaurant. I also like it when I see health inspector ratings, as this again indicates conformity with a repeatable and regulated set of standards.
New York City has taken this a step further, now requiring each food establishment (temporary locations and food trucks excluded) to post a letter grade provided by the health inspector so it is visible to people passing by. The State of South Carolina and the City of Austin have similar grading programs.
For consumer products, ranging from your phone to large appliances and cars, we look to ratings from organizations such as Consumer Reports and J.D. Power and Associates. These give us the opportunity to see how similar products are rated side by side. Vendors even use positive ratings and awards in their advertisements to help build consumer confidence in products. We’ve all heard car companies brag about their five-star crash-test ratings, and it’s common sense to look for certifications such as the UL logo before we purchase appliances.
As an IT security professional for over a decade, I can’t tell you how many times I have wondered if there wasn’t some way to examine a product datasheet or website and know that there had been some validation that the product would work as advertised. Yes, I know that there are testing organizations that publish some information -- but there are caveats to that testing that could fill up a future blog post. In any case, the lack of good information and confidence in datasheets has forced most buyers of IT products to do their own equipment validation. The ad hoc nature of this validation only reinforces the need for an objective, independent method of scoring products.
In my role I’m constantly talking about scoring network and data center equipment, especially after we at BreakingPoint launched our Resiliency Score Lab. When I walk into customer environments now, one of the first things we do is to set up this standards-based validation so that the customer can score infrastructure equipment such as routers, switches, IPS devices, and more. These companies know that the score from one device can be compared against the score of another device, “apples to apples,” so that they can really know how the devices measure up to the claims on the data sheets.
Imagine looking at vendor documentation and knowing that an older model scores at 85 while a larger, newer model scores at 90. That would help you make an informed decision, right? (Note that the decision could be that an 85 is fine for your purposes—so the older, less expensive model could fill your needs without breaking your budget.) Now imagine a solution that would allow you to score devices that you’re deploying, then repeat those assessments each time there is an OS update or configuration change. Imagine the impact of knowing that you deployed a device that performed at 85, but that the newest OS release improved the score to a 90.
Perhaps then you could get a little extra sleep at night—or have time to go check out those grade-A restaurants.
