Over the last few weeks I’ve been working on my Web load testing utility West Wind WebSurge. One of the key components of a load testing tool is the ability to capture URLs effectively so that you can play them back later under load. One of the options in WebSurge for capturing URLs is to use its built-in capture tool which acts as an HTTP proxy to capture any HTTP and HTTPS traffic from most Windows HTTP clients, including Web Browsers as well as standalone Windows applications and services.
To make this happen, I used Eric Lawrence’s awesome FiddlerCore library, which provides most of the functionality of his desktop Fiddler application, all rolled into an easy to use library that you can plug into your own applications. FiddlerCore makes it almost too easy to capture HTTP content!
For WebSurge I needed to capture all HTTP traffic in order to capture the full HTTP request – URL, headers and any content posted by the client. The result of what I ended up creating is this semi-generic capture form:
In this post I’m going to demonstrate how easy it is to use FiddlerCore to build this HTTP Capture Form.
If you want to jump right in here are the links to get Telerik’s Fiddler Core and the code for the demo provided here.
Note that FiddlerCore is bound by a license for commercial usage – see license.txt in the FiddlerCore distribution for details.
FiddlerCore is a library that simply plugs into your application. You can download it from the Telerik site and manually add the assemblies to your project, or you can simply install the NuGet package via:
PM> Install-Package FiddlerCore
The library consists of the FiddlerCore.dll as well as a couple of support libraries (CertMaker.dll and BCMakeCert.dll) that are used for installing SSL certificates. I’ll have more on SSL captures and certificate installation later in this post.
But first let’s see how easy it is to use FiddlerCore to capture HTTP content by looking at how to build the above capture form.
Capturing HTTP Content
Once the library is installed it’s super easy to hook up Fiddler functionality. Fiddler includes a number of static class methods on the FiddlerApplication object that can be called to hook up callback events as well as actual start monitoring HTTP URLs.
In the following code directly lifted from WebSurge, I configure a few filter options on Form level object, from the user inputs shown on the form by assigning it to a capture options object. In the live application these settings are persisted configuration values, but in the demo they are one time values initialized and set on the form. Once these options are set, I hook up the AfterSessionComplete event to capture every URL that passes through the proxy after the request is completed and start up the Proxy service:
CaptureConfiguration.IgnoreResources = true;
CaptureConfiguration.IgnoreResources = false;
string strProcId = txtProcessId.Text;
strProcId = strProcId.Substring(strProcId.IndexOf('-') + 1).Trim();
strProcId = strProcId.Trim();
int procId = 0;
if (!int.TryParse(strProcId, out procId))
procId = 0;
CaptureConfiguration.ProcessId = procId;
CaptureConfiguration.CaptureDomain = txtCaptureDomain.Text;
FiddlerApplication.AfterSessionComplete += FiddlerApplication_AfterSessionComplete;
FiddlerApplication.Startup(8888, true, true, true);
The key lines for FiddlerCore are just the last two lines of code that include the event hookup code as well as the Startup() method call. Here I only hook up to the AfterSessionComplete event but there are a number of other events that hook various stages of the HTTP request cycle you can also hook into. Other events include BeforeRequest, BeforeResponse, RequestHeadersAvailable, ResponseHeadersAvailable and so on.
In my case I want to capture the request data and I actually have several options to capture this data. AfterSessionComplete is the last event that fires in the request sequence and it’s the most common choice to capture all request and response data. I could have used several other events, but AfterSessionComplete is one place where you can look both at the request and response data, so this will be the most common place to hook into if you’re capturing content.
The implementation of AfterSessionComplete is responsible for capturing all HTTP request headers and it looks something like this:
private void FiddlerApplication_AfterSessionComplete(Session sess)
// Ignore HTTPS connect requests
if (sess.RequestMethod == "CONNECT")
if (CaptureConfiguration.ProcessId > 0)
if (sess.LocalProcessID != 0 && sess.LocalProcessID != CaptureConfiguration.ProcessId)
if (sess.hostname.ToLower() != CaptureConfiguration.CaptureDomain.Trim().ToLower())
string url = sess.fullUrl.ToLower();
var extensions = CaptureConfiguration.ExtensionFilterExclusions;
foreach (var ext in extensions)
var filters = CaptureConfiguration.UrlFilterExclusions;
foreach (var urlFilter in filters)
if (sess == null || sess.oRequest == null || sess.oRequest.headers == null)
string headers = sess.oRequest.headers.ToString();
var reqBody = sess.GetRequestBodyAsString();
// if you wanted to capture the response
//string respHeaders = session.oResponse.headers.ToString();
//var respBody = session.GetResponseBodyAsString();
// replace the HTTP line to inject full URL
string firstLine = sess.RequestMethod + " " + sess.fullUrl + " " + sess.oRequest.headers.HTTPVersion;
int at = headers.IndexOf("\r\n");
if (at < 0)
headers = firstLine + "\r\n" + headers.Substring(at + 1);
string output = headers + "\r\n" +
(!string.IsNullOrEmpty(reqBody) ? reqBody + "\r\n" : string.Empty) +
Separator + "\r\n\r\n";
BeginInvoke(new Action<string>((text) =>
The code starts by filtering out some requests based on the CaptureOptions I set before the capture is started. These options/filters are applied when requests actually come in. This is very useful to help narrow down the requests that are captured for playback based on options the user picked. I find it useful to limit requests to a certain domain for captures, as well as filtering out some request types like static resources – images, css, scripts etc. This is of course optional, but I think it’s a common scenario and WebSurge makes good use of this feature.
AfterSessionComplete like other FiddlerCore events, provides a Session object parameter which contains all the request and response details. There are oRequest and oResponse objects to hold their respective data. In my case I’m interested in the raw request headers and body only, as you can see in the commented code you can also retrieve the response headers and body. Here the code captures the request headers and body and simply appends the output to the textbox on the screen. Note that the Fiddler events are asynchronous, so in order to display the content in the UI they have to be marshaled back the UI thread with BeginInvoke, which here simply takes the generated headers and appends it to the existing textbox test on the form. As each request is processed, the headers are captured and appended to the bottom of the textbox resulting in a Session HTTP capture in the format that Web Surge internally supports, which is basically raw request headers with a customized 1st HTTP Header line that includes the full URL rather than a server relative URL.
When the capture is done the user can either copy the raw HTTP session to the clipboard, or directly save it to file. This raw capture format is the same format WebSurge and also Fiddler use to import/export request data.
While this code is application specific, it demonstrates the kind of logic that you can easily apply to the request capture process, which is one of the reasonsof why FiddlerCore is so powerful. You get to choose what content you want to look up as part of your own application logic and you can then decide how to capture or use that data as part of your application.
The actual captured data in this case is only a string. The user can edit the data by hand or in the the case of WebSurge, save it to disk and automatically open the captured session as a new load test.
Other Posts you might also like