Contact   •   Products   •   Search

Rick Strahl's Web Log

Wind, waves, code and everything in between...
ASP.NET • C# • HTML5 • JavaScript • AngularJs

Tracing WCF Messages


I'm working on a small problem that involves connecting to a Web Service and using WS-Security and I'm using WCF. It's not going well and so I've been trying to trouble shoot exactly what's going on. WCF is pretty good about reporting error messages and generally pointing you in the right direction but in this case I've had little luck.

The problem is that the service calls are failing and I can't really tell what's happening exactly because WCF is actually injecting itself in the middle. Because the messages go over SSL I've not been able to do an Http trace either and due to the way that WS-Security works SSL can also not be turned off.

So my first thought was to just enable tracing and look at the messages.

What I'm trying to generate is this:

<?xml version="1.0" encoding="utf-8"?>

<soap:Envelope

xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"

xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns:xsd="http://www.w3.org/2001/XMLSchema">

  <soap:Header>

    <wsse:Security xmlns:wsse="http://schemas.xmlsoap.org/ws/2002/07/secext" soap:mustUnderstand="0">

      <wsse:UsernameToken>

        <wsse:Username>SWT-I75O</wsse:Username>

        <wsse:Password type="wsse:passwordText">$WTCO75O</wsse:Password>

      </wsse:UsernameToken>

    </wsse:Security>

  </soap:Header>

  <soap:Body>

    <Message_Input xmlns=http://Message.com/interfaces>

      <PingRequest/>

    </Message_Ping_Input>

  </soap:Body>

</soap:Envelope>

so I'm enabling message security over the connection with:

<basicHttpBinding>

  <binding name="Message_Binding">

 

    <security mode="TransportWithMessageCredential">

      <transport clientCredentialType="None" proxyCredentialType="None"

          realm="" />

      <message clientCredentialType="UserName" algorithmSuite="Default" />

    </security>

  </binding>
</basicHttpBinding>

which is supposed to use message level security or ws-security. Running the client thusly though:

[TestMethod]

public void PingTest()

{

    Message_Input input = new Message_Input();           

    input.PingRequest = "Test Value";

 

    Message_BindingClient client = new Message_BindingClient();

 

    //client.Endpoint.Binding = new WSHttpBinding("Message_wsBinding");

    client.Endpoint.Binding = new BasicHttpBinding("Message_Binding");

 

    client.ClientCredentials.UserName.UserName = "XXX";

    client.ClientCredentials.UserName.Password = "xxx";

 

    Output_Output output = client.Ping(input);

 

    Assert.IsTrue(output != null, "Ping call failed. Result is null");

}

Call works and hits the server but it's returning a message error.

So the first thing I thought I should do is try to enable tracing. So I added the appropriate Message Logging config entries to the client .config file:

<system.serviceModel>

  <diagnostics>

    <messageLogging

        logEntireMessage="true"

        logMalformedMessages="true"

        logMessagesAtServiceLevel="true"

        logMessagesAtTransportLevel="true"

        maxMessagesToLog="3000"

        maxSizeOfMessageToLog="2000"/>

  </diagnostics>

...
<system.serviceModel>

<system.diagnostics>

  <sources>

    <source name="System.ServiceModel.MessageLogging">

      <listeners>

        <add type="System.Diagnostics.DefaultTraceListener" name="Default">

          <filter type="" />

        </add>

        <add initializeData="c:\traces.xml" type="System.Diagnostics.XmlWriterTraceListener"

          name="messages">

          <filter type="" />

        </add>

      </listeners>

    </source>

  </sources>

</system.diagnostics>


This enables a message level listener that outputs to a text file and when I run my client I get both input and output messages.

Looking at the input message however, I found that it wasn't generating at all what I was expecting it to:

<ApplicationData>

   <TraceData>

    <DataItem>

       <MessageLogTraceRecord Time="2007-12-09T15:33:02.2280000-10:00" Source="ServiceLevelSendRequest" Type="System.ServiceModel.Dispatcher.OperationFormatter+OperationFormatterMessage" xmlns="http://schemas.microsoft.com/2004/06/ServiceModel/Management/MessageTrace">

        <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">

           <s:Header>

            <Action s:mustUnderstand="1" xmlns="http://schemas.microsoft.com/ws/2005/05/addressing/none">document/http://facsia.gov.au/interfaces:Ping</Action>

            <ActivityId CorrelationId="1168ef26-007c-485d-b430-473a35791395" xmlns="http://schemas.microsoft.com/2004/09/ServiceModel/Diagnostics">b48e36bf-34e9-49a3-a2ba-c17e7b30bac9</ActivityId>

           </s:Header>

           <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">

            <FaCSIADiagnosticInterface_Ping_Input xmlns="http://facsia.gov.au/interfaces">

               <PingRequest>Test Value</PingRequest>

            </FaCSIADiagnosticInterface_Ping_Input>

           </s:Body>

        </s:Envelope>

       </MessageLogTraceRecord>

    </DataItem>

   </TraceData>

</ApplicationData>

Notice that the header here is not actually showing any username and password headers going to the server. However the message does show soap headers which indicates it's not sending a plain Soap envelope.

But this didn't look right to me. Everything I read about transport level security seems to indicate that the code I have should work, so why no security headers in the trace.

To be sure I decided maybe I need to really check the raw HTTP input and output, but this proves difficult to do since the request runs over secured HTTP. Fiddler doesn't work with SSL very well, but Charles does - sort of.

Charles lets you trace SSL messages by basically proxying the HTTPS request internally with a private certificate and then forwarding the request to the actual server. This works in some cases - if you're using a browser because the browser lets you bypass what effectively amounts to an invalid certificate.

However, using .NET Networking of any sort (Web Service Clients, HttpWebRequest etc.) by default will not work because requests balk at certificate errorrs of any kind. This is probably as it should be but it sure bites when you want to debug a request.

The trick to make this work is to turn off Certificate validation for test scenarios. I wrote about this a while ago - the solution is to create a specific certificate policy that just accepts any certificate.

There are two steps: First create the certificate policy class:

internal class AcceptAllCertificatePolicy : ICertificatePolicy

{

    public AcceptAllCertificatePolicy()       

    {        }       

    public bool CheckValidationResult(ServicePoint sPoint,

        X509Certificate cert, WebRequest wRequest, int certProb)       

    {            // *** Always accept           

        return true;       

    }

}

then go ahead an use it with the client:

client.Endpoint.Binding = new BasicHttpBinding("Message_Binding");

 

// client.ChannelFactory.Endpoint.Address = new EndpointAddress(https://localhost/wconnect/testpage.wwd);

ServicePointManager.CertificatePolicy = new AcceptAllCertificatePolicy();

 

client.ClientCredentials.UserName.UserName = "XXX";

client.ClientCredentials.UserName.Password = "xxx";


Output_Output
output = client.Ping(input);

When I test this request now with Charles hooked up for Http tracing, the request goes through and I can examine the raw XML messages.

So surprise, surprise the Raw message looks very different from the Message Logging message:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"

            xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">

  <s:Header>

    <ActivityId CorrelationId="9785ff18-120a-4d53-900d-0d1071f470a8"

                xmlns="http://schemas.microsoft.com/2004/09/ServiceModel/Diagnostics">595ef112-9f76-4ae1-87a2-12849cd6b1d3</ActivityId>

    <o:Security s:mustUnderstand="1"

                xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">

      <u:Timestamp u:Id="_0">

        <u:Created>2007-12-10T01:36:46.122Z</u:Created>

        <u:Expires>2007-12-10T01:41:46.122Z</u:Expires>

      </u:Timestamp>

      <o:UsernameToken u:Id="uuid-5b72d9d6-e51f-42bf-8bb2-4d6e4e66535d-1">

        <o:Username>WWW</o:Username>

        <o:Password o:Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">www</o:Password>

      </o:UsernameToken>

    </o:Security>

  </s:Header>

  <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">

    <Message_Input xmlns=http://Message.com/interfaces>

      <PingRequest>Test Value</PingRequest>

    </Message_Input>

  </s:Body>

</s:Envelope>

and it in fact looks like WCF is sending a proper message to the server. Not only that but it even appears that the server is returning a valid SOAP response, however, WCF still fails with a message level error.

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">

  <SOAP-ENV:Body>

    <ns:Message_Output xmlns:ns="http://Message.com/interfaces">

      <ns:PingRequest>

        2007-12-10T13:37:46

      </ns:PingRequest>

    </ns:Message_Output>

  </SOAP-ENV:Body>

</SOAP-ENV:Envelope>

The error message is:

System.ServiceModel.Security.MessageSecurityException: Security processor was unable to find a security header in the message. This might be because the message is an unsecured fault or because there is a binding mismatch between the communicating parties.   This can occur if the service is configured for security and the client is not using security..

[Updated with information from the comments - 11/10/2007]

So what is it?

Good question. The server is accepting the input and returning a SOAP response, but WCF doesn't like the result. It appears WCF is looking for additional headers in the message and failing.

As it turns out that's EXACTLY what the problem is. Thanks to one of the commenters and a previous post on the MSDN forums I was able to get this working by using the following code:

[TestMethod]

public void PingTest()

{

    Message_Ping_Input input = new Message_Ping_Input();

    input.PingRequest = "Test Value";

 

    MessageClient client = new MessageClient();

 

    client.ClientCredentials.UserName.UserName = "XXX";

    client.ClientCredentials.UserName.Password = "xxx";

 

    // *** Don't include TimeStamp header - so server won't expect it back

    // *** This allows results returned without any Soap Headers

    BindingElementCollection elements = client.Endpoint.Binding.CreateBindingElements();

    elements.Find<SecurityBindingElement>().IncludeTimestamp = false;

    client.Endpoint.Binding = new CustomBinding(elements);

 

 

    Message_Ping_Output output = client.Ping(input);

 

    Assert.IsNotNull(output, "No data structure returned");

    Assert.IsNotNull(output.PingRequest, "No ping data value returned");

}

 

 

The issue is that WCF expects a TimeStamp Soap header in the response. If you look at the outbound response and the Soap headers you'll see that there's a timestamp there. The timestamp is expected to be returned on the return Soap response.  Note that this is not a requirement of WS-Security so WCF is doing something 'special' here that is in effect breaking this service call.


The above code modifies the Binding configuration by explicitly removing the Timestamp from the outbound call which removes the requirement for the server to return it. And this makes WCF happy and the call goes through.

Make Donation
Posted in WCF  Web Services  


Feedback for this Post

 
# re: Tracing WCF Messages
by Vlad Navazhylau December 09, 2007 @ 7:14pm
I am not an expert but here is my 2 cents.

your request message schema is "http://message.com/interfaces"

but response message schema is "http://Message.com/interfaces"

note capital "M"

I believe it may matter, but again I am not an expert.
# re: Tracing WCF Messages
by Rick Strahl December 10, 2007 @ 11:04am
Hi Vlad - sharp eye <s>, but that's my fault. I changed the namespaces in the messages for the post manually and I screwed them up. The namespaces are correct though as they're pulled out of the WSDL by WCF.

The issue is why WCF is failing on what looks to be a valid response.
# re: Tracing WCF Messages
by Rainer December 10, 2007 @ 11:54am
Hi Rick,

Maybe this link helps you out:
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1040876&SiteID=1

Look at the bottom of the page
# re: Tracing WCF Messages
by Rick Strahl December 10, 2007 @ 2:13pm
@Rainer - thanks man! That's it!

To summarize the problem apparently is that by adding WS-Security WCF will add a timestamp header to the request, which it's expecting back in the response. The server in this case is NOT returning the header in the response and so the request fails.

The work-around is to adjust the Binding to remove the timestamp from the outbound message. The code I ended up with is this:

[TestMethod]
public void PingTest()
{
    Message_Ping_Input input = new Message_Ping_Input();
    input.PingRequest = "Test Value";

    MessageClient client = new MessageClient();

    client.Endpoint.Binding = new BasicHttpBinding("Message_Binding");  // specific binding


    client.ClientCredentials.UserName.UserName = "XXX";
    client.ClientCredentials.UserName.Password = "xxx";

    // *** Allow acceptance of all certificates even if invalid (Test only!)
    ServicePointManager.CertificatePolicy = new AcceptAllCertificatePolicy();

    // *** Don't include TimeStamp header - so server won't expect it back
    // *** This allows results returned without any Soap Headers
    BindingElementCollection elements = client.Endpoint.Binding.CreateBindingElements();
    elements.Find<SecurityBindingElement>().IncludeTimestamp = false;
    client.Endpoint.Binding = new CustomBinding(elements);


    Message_Ping_Output output = client.Ping(input);

    Assert.IsNotNull(output, "No data structure returned");
    Assert.IsNotNull(output.PingRequest, "No ping data value returned");
}


and that does the trick. Neat trick to be able to CreateBindingElements() make a change and then simply reassign the binding from the elements. The same technique can be potentially applied to other things.


WCF is great in its flexibility but man is it hard to find stuff like this. I've been stuck on this simple thing for a few days now and ultimately the original poster waited for quite some time for a response - from Microsoft. There are so many freaking options that are so vaguely defined and documented that unless you have studied the WSDL/SOAP specs (and who has with all the latest specifications) it's nearly impossible to find this stuff.

The engine has all the configurability in the world but the last 3 times I've tried to use WCF in Web Service interop with non .NET services I've run into similiar stumbling blocks right from the start. It seems that WCF's schema validation and spec interpretation in many cases is overly strict causing some startup grief like this.

Not sure how to solve this problem - I suppose once the technology gains wider use searching will bring up more results along these lines hopefully. But it's difficult even to define proper search parameters and not get a bunch of noise as I found out this weekend.
# re: Tracing WCF Messages
by David Barrett January 03, 2008 @ 8:01am
Great post, Rick! Thanks for sharing. I, too, was running into the issue of not seeing the 'true' message going across the wire, so this was very helpful.

By the way, you *can* see the message in WCF logging -- in the WCF configurator, turn on the transport enum in the Log Level in addition to service messages. I guess this stuff gets added at the transport layer, so you don't see it until after the message is sent by the client and before it's actually put onto the wire.

But once you add it there, you can see the messages in the log twice -- once without the headers as you show above, and then immediately following as a 'wire' message -- with the headers.
# re: Tracing WCF Messages
by Steve Moseley December 15, 2008 @ 1:52pm
Is there a way to catch the actual message sent on the wire and display on web page like the trace.axd HttpHandler?

I have a need to see the message in a test environment I do not have access to.
# re: Tracing WCF Messages
by Lana December 23, 2008 @ 10:45am
Thank you for the article. It was so much helpful.
# re: Tracing WCF Messages
by andrew December 29, 2008 @ 7:28am
This is exactly the type of debugging I need to do, but how to setup charles to do this?

I have a "Map Remote" setup to forward traffic received on

https://127.0.0.1:8080/TestService/Ping

to https://realservice/TestService/Ping

But I do not see anything show up when I attempt to connect to TestService.

Is there some charles setup thing I am missing?
# re: Tracing WCF Messages
by Ruslan October 13, 2009 @ 12:01am
Hi Rick,

Thanks for the article, it's very useful.

I noticed you had mustUnderstand="0" in the initial message security header, but in the result mustUnderstand="1".
At the moment I'm trying to resolve such problem - replace mustUnderstand="1" with mustUnderstand="0". I created a ClientMessageInspector, but there is no security header in the intercepted message.
Maybe you've met with similar problem and can advise me something.

Thank you.

Environment I have:
WCF Client,
Non WCF web service,
basicHttpBinding,
security mode="TransportWithMessageCredential"
# re: Tracing WCF Messages
by Adam November 05, 2009 @ 6:36am
Thanks for the article. The timestamp issue was killing me.
# re: Tracing WCF Messages
by Ciprian Fusa January 26, 2010 @ 2:40am
You can use a custom binding to achieve the same thing:

<system.serviceModel>
  <bindings>
    <customBinding>
      <binding name="myCustomBindingConfig">
        <security authenticationMode="UserNameOverTransport" includeTimestamp="false"></security>
        <textMessageEncoding messageVersion="Soap11"></textMessageEncoding>
        <httpsTransport />
      </binding>
    </customBinding>
  </bindings>
  <client>
    <endpoint address="" binding="customBinding" bindingConfiguration="myCustomBindingConfig" 
     contract="" name="" />
  </client>
</system.serviceModel>


An then you'll only need to do:

MessageClient client = new MessageClient();
client.ClientCredentials.UserName.UserName = "XXX";
client.ClientCredentials.UserName.Password = "xxx";
# re: Tracing WCF Messages
by T-Bone March 03, 2010 @ 6:11am
Ruslan, did you ever manage to get Mustunderstand ="0" in the security header?
I have the same problem and also tried the client message inspector with the same results.
# re: Tracing WCF Messages
by Adi Kurniawan July 29, 2011 @ 11:45pm
Thank you. Thank you. Thank you.
This article is very helpful. I've had the same problem and tried to solve it for the last 3 days until I found this article.
# re: Tracing WCF Messages
by Bob March 12, 2012 @ 11:05am
Thanks, this helped me connect to a Peoplesoft web service from a WCF client. -Bob
# re: Tracing WCF Messages
by mike March 30, 2012 @ 2:06pm
you said you wanted to make the header like this

<wsse:Security xmlns:wsse="http://schemas.xmlsoap.org/ws/2002/07/secext"

but your result have this
<o:Security s:mustUnderstand="1"

xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">

how do you change the namespace of the security header?

thanks
# re: Tracing WCF Messages
by Bo April 16, 2012 @ 7:14am
I had the same issue, and adding

elements.Find<SecurityBindingElement>().EnableUnsecuredResponse = true;

worked for me.

I tried IncludeTimestamp = false and it worked too. So seemed either flag will instruct the WCF client not to expect a security header in response. However, I think the EnableUnsecredResponse may be a better choice.
# re: Tracing WCF Messages
by Chinh Do June 14, 2012 @ 7:21am
Hi Rick: Thanks a lot for this. I spent the last hour looking for this and without your post, who knows how much more time I would need to spend! :-) Chinh
# re: Tracing WCF Messages
by Gopal Narayan July 08, 2013 @ 10:08pm
I usually never post on forums, I'm compelled to do this because I was trying to consume a third party web service solely based on wsdl and not other help whatsoever and I was getting "WS Security header not found" message.

I pulled my hair for over a week trying to do various things and this article nailed it.

Shame on Microsoft, they do great things but screw up BIG time with crap like this!!

Thanks for the article, I'd buy you a beer if I meet you some day!!
# re: Tracing WCF Messages
by Manjit August 15, 2014 @ 9:01am
Hi Rick, thanks for the article. Saved me incredible amount of time.
 


West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2014