Contact   •   Products   •   Search

Rick Strahl's Web Log

Wind, waves, code and everything in between...
ASP.NET • C# • HTML5 • JavaScript • AngularJs

.NET WebRequest.PreAuthenticate – not quite what it sounds like


I’ve run into the  problem a few times now: How to pre-authenticate .NET WebRequest calls doing an HTTP call to the server – essentially send authentication credentials on the very first request instead of waiting for a server challenge first? At first glance this sound like it should be easy: The .NET WebRequest object has a PreAuthenticate property which sounds like it should force authentication credentials to be sent on the first request. Looking at the MSDN example certainly looks like it does:

http://msdn.microsoft.com/en-us/library/system.net.webrequest.preauthenticate.aspx

Unfortunately the MSDN sample is wrong. As is the text of the Help topic which incorrectly leads you to believe that PreAuthenticate… wait for it - pre-authenticates. But it doesn’t allow you to set credentials that are sent on the first request.

What this property actually does is quite different. It doesn’t send credentials on the first request but rather caches the credentials ONCE you have already authenticated once. Http Authentication is based on a challenge response mechanism typically where the client sends a request and the server responds with a 401 header requesting authentication.

So the client sends a request like this:

GET /wconnect/admin/wc.wc?_maintain~ShowStatus HTTP/1.1
Host: rasnote
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en,de;q=0.7,en-us;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

and the server responds with:

HTTP/1.1 401 Unauthorized
Cache-Control: private
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.5
WWW-Authenticate: basic realm=rasnote"
X-AspNet-Version: 2.0.50727
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
WWW-Authenticate: Basic realm="rasnote"
X-Powered-By: ASP.NET
Date: Tue, 27 Oct 2009 00:58:20 GMT
Content-Length: 5163

plus the actual error message body.

The client then is responsible for re-sending the current request with the authentication token information provided (in this case Basic Auth):

GET /wconnect/admin/wc.wc?_maintain~ShowStatus HTTP/1.1
Host: rasnote
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en,de;q=0.7,en-us;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: TimeTrakker=2HJ1998WH06696; WebLogCommentUser=Rick Strahl|http://www.west-wind.com/|rstrahl@west-wind.com; WebStoreUser=b8bd0ed9
Authorization: Basic cgsf12aDpkc2ZhZG1zMA==

Once the authorization info is sent the server responds with the actual page result.

Now if you use WebRequest (or WebClient) the default behavior is to re-authenticate on every request that requires authorization. This means if you look in  Fiddler or some other HTTP client Proxy that captures requests you’ll see that each request re-authenticates: Here are two requests fired back to back:

TwoRequests

and you can see the 401 challenge, the 200 response for both requests.

If you watch this same conversation between a browser and a server you’ll notice that the first 401 is also there but the subsequent 401 requests are not present.

WebRequest.PreAuthenticate

And this is precisely what the WebRequest.PreAuthenticate property does: It’s a caching mechanism that caches the connection credentials for a given domain in the active process and resends it on subsequent requests. It does not send credentials on the first request but it will cache credentials on subsequent requests after authentication has succeeded:

string url = "http://rasnote/wconnect/admin/wc.wc?_maintain~ShowStatus";
HttpWebRequest req = HttpWebRequest.Create(url) as HttpWebRequest;
req.PreAuthenticate = true;
req.Credentials = new NetworkCredential("rick", "secret", "rasnote");
req.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
WebResponse resp = req.GetResponse();
resp.Close();

req = HttpWebRequest.Create(url) as HttpWebRequest;
req.PreAuthenticate = true;
req.Credentials = new NetworkCredential("rstrahl", "secret", "rasnote");
req.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
resp = req.GetResponse();

which results in the desired sequence:

PreAuthenticated

where only the first request doesn’t send credentials.

This is quite useful as it saves quite a few round trips to the server – bascially it saves one auth request request for every authenticated request you make. In most scenarios I think you’d want to send these credentials this way but one downside to this is that there’s no way to log out the client. Since the client always sends the credentials once authenticated only an explicit operation ON THE SERVER can undo the credentials by forcing another login explicitly (ie. re-challenging with a forced 401 request).

Forcing Basic Authentication Credentials on the first Request

On a few occasions I’ve needed to send credentials on a first request – mainly to some oddball third party Web Services (why you’d want to use Basic Auth on a Web Service is beyond me – don’t ask but it’s not uncommon in my experience). This is true of certain services that are using Basic Authentication (especially some Apache based Web Services) and REQUIRE that the authentication is sent right from the first request. No challenge first. Ugly but there it is.

Now the following works only with Basic Authentication because it’s pretty straight forward to create the Basic Authorization ‘token’ in code since it’s just an unencrypted encoding of the user name and password into base64. As you might guess this is totally unsecure and should only be used when using HTTPS/SSL connections (i’m not in this example so I can capture the Fiddler trace and my local machine doesn’t have a cert installed, but for production apps ALWAYS use SSL with basic auth).

The idea is that you simply add the required Authorization header to the request on your own along with the authorization string that encodes the username and password:

string url = "http://rasnote/wconnect/admin/wc.wc?_maintain~ShowStatus";
HttpWebRequest req = HttpWebRequest.Create(url) as HttpWebRequest;

string user = "rick";
string pwd = "secret";
string domain = "www.west-wind.com";

string auth = "Basic " + Convert.ToBase64String(System.Text.Encoding.Default.GetBytes(user + ":" + pwd));
req.PreAuthenticate = true;
req.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
req.Headers.Add("Authorization", auth);
req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)"; WebResponse resp = req.GetResponse(); resp.Close();

This works and causes the request to immediately send auth information to the server. However, this only works with Basic Auth because you can actually create the authentication credentials easily on the client because it’s essentially clear text. The same doesn’t work for Windows or Digest authentication since you can’t easily create the authentication token on the client and send it to the server.

Another issue with this approach is that PreAuthenticate has no effect when you manually force the authentication. As far as Web Request is concerned it never sent the authentication information so it’s not actually caching the value any longer. If you run 3 requests in a row like this:

        string url = "http://rasnote/wconnect/admin/wc.wc?_maintain~ShowStatus";
        HttpWebRequest req = HttpWebRequest.Create(url) as HttpWebRequest;

        string user = "ricks";
        string pwd = "secret";
        string domain = "www.west-wind.com";

        string auth = "Basic " + Convert.ToBase64String(System.Text.Encoding.Default.GetBytes(user + ":" + pwd));
        req.PreAuthenticate = true;
        req.Headers.Add("Authorization", auth);
        req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
        WebResponse resp = req.GetResponse();
        resp.Close();


        req = HttpWebRequest.Create(url) as HttpWebRequest;
        req.PreAuthenticate = true;
        req.Credentials = new NetworkCredential(user, pwd, domain);
        req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
        resp = req.GetResponse();
        resp.Close();

        req = HttpWebRequest.Create(url) as HttpWebRequest;
        req.PreAuthenticate = true;
        req.Credentials = new NetworkCredential(user, pwd, domain);
        req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
        resp = req.GetResponse();

you’ll find the trace looking like this:

Manually

where the first request (the one we explicitly add the header to) authenticates, the second challenges, and any subsequent ones then use the PreAuthenticate credential caching. In effect you’ll end up with one extra 401 request in this scenario, which is still better than 401 challenges on each request.

Getting Access to WebRequest in Classic .NET Web Service Clients

If you’re running a classic .NET Web Service client (non-WCF) one issue with the above is how do you get access to the WebRequest to actually add the custom headers to do the custom Authentication described above? One easy way is to implement a partial class that allows you add headers with something like this:

public partial class TaxService 
{ protected NameValueCollection Headers = new NameValueCollection(); public void AddHttpHeader(string key, string value) { this.Headers.Add(key,value); } public void ClearHttpHeaders() { this.Headers.Clear(); } protected override WebRequest GetWebRequest(Uri uri) { HttpWebRequest request = (HttpWebRequest) base.GetWebRequest(uri); request.Headers.Add(this.Headers); return request; }
}

where TaxService is the name of the .NET generated proxy class. In code you can then call AddHttpHeader() anywhere to add additional headers which are sent as part of the GetWebRequest override. Nice and simple once you know where to hook it.

For WCF there’s a bit more work involved by creating a message extension as described here: http://weblogs.asp.net/avnerk/archive/2006/04/26/Adding-custom-headers-to-every-WCF-call-_2D00_-a-solution.aspx.

FWIW, I think that HTTP header manipulation should be readily available on any HTTP based Web Service client DIRECTLY without having to subclass or implement a special interface hook. But alas a little extra work is required in .NET to make this happen

Not a Common Problem, but when it happens…

This has been one of those issues that is really rare, but it’s bitten me on several occasions when dealing with oddball Web services – a couple of times in my own work interacting with various Web Services and a few times on customer projects that required interaction with credentials-first services. Since the servers determine the protocol, we don’t have a choice but to follow the protocol. Lovely following standards that implementers decide to ignore, isn’t it? :-}

Make Donation
Posted in .NET  CSharp  Web Services  


Feedback for this Post

 
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by John Sheehan February 18, 2010 @ 2:34pm
Good article. Nice overview of the problem. I ran into a similar issue with RestSharp which Daniel Crenna (TweetSharp) pointed out and helped resolve to make RS use this method by default with basic auth.
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by Matt February 18, 2010 @ 9:42pm
Brilliant article. This is something that's caught me in the past - you explain it so well I've added a link back to this on my own blog for future reference. Thanks.
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by Josh Stodola February 19, 2010 @ 7:54am
Great detailed explanation, as always!
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by JW March 18, 2010 @ 12:02pm
So glad I found this. I have this issue calling a JAX-WS on a weblogic server, not exactlly odd ball in my mind so am I doing something wrong on the ejb side?
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by Jon Hanna July 18, 2010 @ 8:32am
Ironically, this is the exact opposite of the problem with using XHR for ajax, json or similar uses of javascript in Safari. It does (or did, they may have fixed it since) send the Basic headers on the first attempt, with no means to override this behaviour.

This is perfect in the sort of case you are talking about (webserver uses Basic auth and [incorrectly] doesn't challenge if they aren't sent) but is disasterous in cases where a more secure authentication method like Digest or NTLM over an unecrypted connection (quite a good idea when the only sensitive details are the auth details, as encrypted connections have lots of downsides along with their obvious upside). In such a case the correct behaviour works fine, but the behaviour of Safari sends your password in the clear, before then authenticating correctly in response to the server's challenge.

Having found no better solution for a given application but to file a bug report against Safari and block it from the functionality in question (and thank my stars that it was a hobby project rather than anything critical), I'm very glad that WebRequest has the behaviour you complain of here.

The behaviour of WebRequest is correct, and you had to hack around the incorrect behaviour of a server. Doing the same thing when not needed by a malfunctioning server would introduce a security risk.
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by Kashif July 28, 2010 @ 10:57am
I am working on a CJ webservice , it's working fine in our development XP machine having IIS 6 installed. But same code return 401 UnAuthorized on production Server ( IIS 7.0). I don't understand the issue. This third party service is based on Authorization key only and working fine on development machine.
By the way Production server is shared so we have very limited option to diagnose the problem. Any feedback is much appreciated.
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by Oisin G. February 24, 2012 @ 12:08pm
I've just found this and I find myself having to go the WCF route. I have a WCF service out of my control that's using MTOM streaming AND basic authentication. However, WCF clients won't allow basic authentication in this situation because it's a one-way post of data (which is an implementation detail of the WCF client.) WCF has hard checks to prevent you from enabling transport security in this case. My only choice seems to be to configure the proxy to use no authentication but inject basic auth headers myself. Is it so f***ed up that WCF as a service allows you to enable MTOM and streaming along with basic authentication, but no WCF clients can use it? Java clients don't have a problem as those poor guys have to inject basic auth headers most of the time themselves anyway. Sigh.
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by Phillip Lanza September 19, 2012 @ 9:28am
Thanks for the informative post. That's almost exactly what I needed to fix a problem that I was having with a legacy importer app that one of my company's clients uses. I changed the auth type from "basic" to "digest" and that fixed the 401 response error. Thanks again!
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by bex August 14, 2013 @ 12:06pm
Great article! It helped me out when I was trying to get the WebLogic login page to stop showing. I did something slightly different, tho: I'm doing a 'PING' request first, and then my actual request. Same as a round-trip, but then I get a JSESSIONID in the response that I can cache in the cookie for future requests.
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by shabir September 29, 2013 @ 12:33am
Hi
Simply awesome.even i thought lots of time that what this WebRequest.PreAuthenticate is doing .It was very confusing? looking into your post i feel glad.I really appreciate the way you present things. You are simple awesome..

Regards
Shabir Hakim
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by Elad Nava October 15, 2013 @ 10:52am
Great article! I was looking for a solution to this issue for a long time!


The problem I was experiencing, is that a proxy that requires authentication returned an unexpected error in case I didn't supply the authentication header in the first request.
# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like
by Bidyut September 11, 2014 @ 2:59pm
Out of all the trash available on the WebRequest authentication topic, at various sites , this is the most complete example. It is clear to understand with coherent script flow. I found the following script block to be most useful

"string auth = "Basic " + Convert.ToBase64String(System.Text.Encoding.Default.GetBytes(user + ":" + pwd));
req.PreAuthenticate = true;
req.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;"
 


West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2014