Setting An Automation Standard: BreakingPoint and NTAF
by Kris RaneyBreakingPoint is a founding member of the Network Test Automation Forum (NTAF). NTAF is a consortium of enterprise organizations that have large network and security validation labs, and vendors who sell equipment into those labs. In short, it's a group of our customers and our competitors. This group has come together to define a standard [PDF link] to allow interoperation among that equipment and with the automation frameworks that already exist in many labs. Today we wanted to provide an update on our work with NTAF.
Automation and TCL
Like most tools that live an a lab, the BreakingPoint Storm CTM™ provides a TCL-based API that allows users to bypass the graphical user interface (GUI) and tie the tool directly into their lab automation. Having an easy-to-use GUI like ours is great 99% of the time, but most large labs also have processes that need to be repeated often, and precisely. The BreakingPoint Storm CTM GUI helps with this by allowing users to save their configurations and reuse them, but ultimately most labs have an automation framework that they need the BreakingPoint Storm CTM to tie into.
For years, the de facto standard for this sort of automation has been a TCL API. BreakingPoint has always made a strong effort to have a good and complete TCL API. We strive to be as complete as possible when it comes to having the functionality of our UI available for automation in the TCL API. That said, making TCL the central point of automation does have some drawbacks:
- You’re locked into using TCL as the implementation language for your test automation.
- When integrating different versions of tools or tools from different vendors, it can be a challenge to set up a TCL shell that has exactly the right set of extensions and the right versions of those extensions so that everything can work at the same time.
- Writing all of that TCL automation takes a lot of in-house development and resources.
Setting An Automation Standard
There are four key aspects of the NTAF effort that I think will make it a huge win. In fact, they’re the main reasons why we work closely with the group:
- It moves the point of automation from a programming API (in TCL) to a network-based API.
- It makes use of the existing XMPP protocol, with XML-based messages.
- It requires the interfaces provided by tools to be self-describing.
- It's being developed using a "show me how it works" approach, not a theoretical debate.
Moving to a network API has a number of advantages:
- It frees people creating automation to use whatever programming language and development tools suit their environment and needs.
- It removes the requirement that the automation layers for different tools that need to work together have to run on the same physical hardware.
- It makes it more feasible for a third party to develop an automation tool that could be purchased, rather than having to develop all automation in-house.
- It's more in line with what people have come to expect from other kinds of software and equipment.
Meeting the Challenges of Creating a Network API
Of course, there are a number of challenges that come along with making a network API. That's why it's so important that the NTAF standard will be built on Extensible Messaging and Presence Protocol (XMPP). XMPP is a network protocol originally designed for use as an instant messenger protocol. It has proven to be very flexible and useful, and over time hundreds of extensions to it have been defined and standardized. Also, it has been deployed and is actively in service in a number of places.
Using XMPP, rather than starting from scratch, means that a lot of the tricky parts of making a network API have already been solved. For example, it knows how to deal with firewalls or other network setups that create complications. Being able to deal with these things is vital, because in many cases an engineer in India may need to use equipment that's in a lab in the UK. Even within a single company, the labs themselves may be firewalled off from the rest of the company. XMPP is already in use in situations like these, so those of us within NTAF won't have to solve these problems all by ourselves, and users won't have to wait for NTAF to get around to all the details before they have something that works in their particular environment.
Not only that, but using an existing protocol means there are already a number of open-source tools and libraries available that understand the protocol. That means that the standard can be adopted more easily, and will show up in real labs much sooner.
The third item above, self-describing interfaces, is no less important. What it means is that it's not enough for a tool to provide an API to talk to. There is a small set of APIs that every tool must implement that make it possible for other tools to ask "what can you do?" and "how do I ask you to do it?"—a process called discovery. That means that two tools that have never been tested together can still learn about each other and then begin to interoperate.
It's the final item, though, that I think is most important. Once we had a draft standard, the members of NTAF began implementing prototypes to try it out in practice. Late last month NTAF met so that these prototypes could be demonstrated to the board of directors.
NTAF Draft Standard In Action
During our prototype demonstration, members chose the implementation language that made sense for them, and there were several—including TCL, Java, Python, and an implementation that would work for any Windows CLR language. During the demo, tools were interacting that were located in two separate cities in California, two separate cities in Texas, and a server in the cloud. All of them were communicating through a variety of corporate firewalls, VPNs, and network security. Implementations from companies, including competitors, were brought together for the first time at the beginning of that week, and were functionally talking together by the end of a single meeting. By the time the demo came on Friday, they were working quite well together. The demo showed multiple different combinations of cross-vendor interaction.
Of course, work remains to be done. A few details of the standard were found that could use some improvement—and that's the great thing about having made real implementations. The standard isn't being made in a theoretical vacuum, and these things will be fixed.
The goal remains to finalize the details and ratify the first version of the NTAF standard by the end of Q1 2011, so that these improvements can be rolled into products that are actually shipping—and thus start improving the lives of our customers—as soon as possible. For anyone responsible for automation within their labs, it's something to look forward to.
