I depend heavily on using Fiddler or Charles for helping me debug Http issues in general and both work great for typical Web development debugging in the browser. This is especially useful for AJAX applications since you can't 'see' the output that travels over the wire directly most of the time.

Note that if you are debugging on the local machine, you can't debug Http requests against Localhost (127.0.0.1 - the loopback address) because there's actually no network traffic generated. Instead you need to use your local machine name which  will make the proxy kick in. The default proxy settings in .NET are such that any proxy redirection in the system is picked up by your .NET application and so you should see any HTTP traffic going through the proxy.  This is true both in the Web Browser and for standalone apps calling Web Services so if you're using a Web Service client you need to test, just make sure you point it at your local machine name instead of localhost and fire away.

This works fine and you can see the network HTTP traffic for service requests, however, I've noticed (recently it seems) that this only works for the first call. Any subsequent requests invariably end up failing with a network error when Fiddler is attached. The error I get is:

The underlying connection was closed: A connection that was expected to be kept alive was closed by the server.

Remove the proxy and all is well. Apparently this is an issue that is related to the proxy is handling the keep alives on the proxy connection. A related issue was covered here in regards to Authentication:

http://blogs.newsgator.com/inbox/2006/09/fiddler_and_net.html

 Comparing the headers the only difference I see is that Fiddler use a Proxy-Connection: Keep-Alive header which is likely the cause of the problem and while that solves my immediate problem, I'm wondering if there's some way to override this on the .NET connection end. Realistically we need to be able to control the connection so that this sort of proxy sceanario can work, but looking through the client proxy options - down to the bindings on the client, I don't see a way to control the keep-alives. If the client can surpress keep-alives altogether (Connection: Close) this likely would not be a problem. However, I don't think that there's actually a way to make this happen. I even went as far as adding explicit headers to the WCF client:

NorthwindServiceClient client = new NorthwindServiceClient();

using (OperationContextScope scope = new OperationContextScope(client.InnerChannel))

{

    HttpRequestMessageProperty req = new HttpRequestMessageProperty();

    req.Headers.Add("Connection", "Close");

    OperationContext.Current.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = req;

    Customer[] custList = client.GetCustomerList("");

    MessageBox.Show(custList.Length.ToString());

    Customer cust = client.GetCustomer("ALFKI");

    MessageBox.Show(cust.CompanyName);

}

The ability to have almost full control over the client headers (and a number of other things) using the OperationContextScope is very useful. Unfortunately however, in this case it didn't work because apparently Connection and Keep-Alive are headers that can't be overridden. Grumble...

So after looking in the Rules file in Fiddler again I found the following commented script block:

// Uncomment to reduce incidence of "unexpected socket closure" exceptions in .NET code.  
// Note that you really should also fix your .NET code to gracefully handle unexpected connection closure.
//
 if ((oSession.responseCode != 401) && (oSession.responseCode != 407)){
   oSession.oResponse["Connection"] = "close";
 }

The above is commented in Fiddler's CustomRules.js  in the OnBeforeResponse method. Uncommenting the code above makes the successive Web Service calls work with Fiddler attached. As the comment mentions above this isn't ideal but it works - you may have to resort to a few additional conditions with NTLM Authentication as mentioned in the newsgator blog entry above.

Yuch... it's ugly and it does make me wonder though  what the problem is here. Debugging proxy connections in live applications is a real bitch because you can rarely duplicate it so it'd be nice to know exactly what's causing the problem here because it'll likely cause problem with a live proxy in some situations as well. Or better yet somehow have the client allow setting the connection status explicitly. Unfortunately it looks like that's not really possible however as it's overridden explicitly by WCF.

Incidentally Charles works properly and maybe that's just the easiest route. Charles works nicer with SOAP requests anyway as  it has options to auto-format XML and works against localhost without explicitly specifying a machine name. But old habits die hard...