Querystring and form data parsing and editing is something that I seem to run into from time to time in non-Web applications. Actually, it’s easy enough to do simple parsing in Web applications that use System.Web using the HttpUtility.ParseQueryString() method or simply using HttpWebRequest’s methods. If you’re inside of an ASP.NET Web app, see the end of this article.
But if you’re not working within the scope of System.Web, there’s not a ton of support for manipulating form data or query string values via code. Heck it’s a pain even in ASP.NET Web API or SignalR if you’re self or OWIN hosting, where there’s no real interface to create or even easily read raw form and query string data. For client side applications in particular the lack of this functionality can be a pain. I’ve been guilty of adding System.Web from time to time just for this functionality, which is not a good idea due to the sheer size of System.Web.
It makes you wonder why this sort of functionality isn’t provided natively in the base framework in System.NET since it’s needed for all sorts of client side HTTP scenarios, from constructing client side requests with HttpWebRequest to server side manipulation of URLs, or even using the frameworks that don’t have built in parsing and update support. Even the new HttpClient class doesn’t have good support for form data creation although it can at least be done. Either way I’ve run into having to manipulate url encoded often enough that it’s gotten me to do something about it.
To wit, currently I’m working on West Wind WebSurge and one useful feature improvement request I’ve had is to add the ability to override provided query string parameters with custom query string key/value pairs provided by the user when a test is run. Internally I use HttpWebRequest to fire previously captured requests, and each request is filtered/modified based on a number of request modifiers that can be set. For querystring values this is useful for changing fixed ids or ‘license plate’/token query string parameters when the URL executes to match a specific user context. You can also change other things like Cookies or Authentication headers, but that’s not part of this discussion here.
A UrlEncodingParser Class
I’ve needed to parse, write or update querystrings or form data a few times before and because it’s not exactly rocket science I’ve usually in-lined a bit of code that handles the simple things I needed to do – it worked but it’s ugly. Well, no more! This time I decided to fix this once and for all by creating a small helper class that handles this problem more generically.
The class does the following:
- Takes raw Query String or Form data as input
- Optionally allows a full URL as input – if a URL is passed the query string data is modified
- Allows reading of values by key
- Allows reading of multiple values for a key
- Allows modifying and adding of keys
- Allows for setting multiple values for a single key
- Allows writing modified data out to raw data or a URL if a URL was originally provided
This is perfect for URL injection or for applications that need to build up raw HTTP form data to post to a server.
Using UrlEncodingParser
This class is based on the NameValueCollection class, which is also used by System.Web’s various key value collections like QueryString, Form and Headers. One of the unique things about this collection class is that it’s optimized for fast retrieval and explicitly supports multiple values per key.
My simple implementation of UrlEncodingParser subclasses NameValueCollection and adds the ability to parse an existing urlencoded string or full URL into it, allows for modification including adding multi-values for keys, and then can output the urlencoded data including a full URL if one was passed in using ToString().
Because of the reuse of NVC using UrlEncodingParser should feel familiar. Here’s is an example:
[TestMethod]
public void QueryStringTest()
{
string str = "http://mysite.com/page1?id=3123&format=json&action=edit&text=It's%20a%20brave%20new%20world!";
var query = new UrlEncodingParser(str);
Assert.IsTrue(query["id"] == "3123");
Assert.IsTrue(query["format"] == "json","wrong format " + query["format"]);
Assert.IsTrue(query["action"] == "edit");
Console.WriteLine(query["text"]);
// It's a brave new world!
query["id"] = "4123";
query["format"] = "xml";
query["name"] = "<< It's a brave new world!";
var url = query.ToString();
Console.WriteLine(url);
//http://mysite.com/page1?id=4123&format=xml&action=edit&
//text=It's%20a%20brave%20new%20world!&name=%3C%3C%20It's%20a%20brave%20new%20world!
}
This code passes in a full URL, checks the input values, then modifies and adds one, then writes out the modified URL to a new string. I’m using a URL here which allows preserves the original base URL and simply appends the new/modified query string. But you could also pass the raw URL encoded data/querystring in which case you get just that data back.
The parser also supports multiple values per key, since that’s a supported feature for Form variables at least (not for query strings though).
[TestMethod]
public void QueryStringMultipleTest()
{
string str = "http://mysite.com/page1?id=3123&format=json&format=xml";
var query = new UrlEncodingParser(str);
Assert.IsTrue(query["id"] == "3123");
Assert.IsTrue(query["format"] == "json,xml", "wrong format " + query["format"]);
Console.WriteLine(query["text"]);
// multiple format strings
string[] formats = query.GetValues("format");
Assert.IsTrue(formats.Length == 2);
query.SetValues("multiple", new[]
{
"1",
"2",
"3"
});
var url = query.ToString();
Console.WriteLine(url);
Assert.IsTrue(url ==
"http://mysite.com/page1?id=3123&format=json&format=xml&multiple=1&multiple=2&multiple=3");
}
Show me the Code
The implementation of this simple class is straightforward, although I ended up experimenting a bit with various dictionary types before I realized that I had to support multiple values per key in order to support Form data, which led me to the NameValueCollection class. The beauty of that is that very little code is required as the key/value management is completely handle by the base – I only had to add parsing a couple of specialty overrides.
Here’s the complete code (you can also find the code on Github):
/// <summary>
/// A query string or UrlEncoded form parser and editor
/// class that allows reading and writing of urlencoded
/// key value pairs used for query string and HTTP
/// form data.
///
/// Useful for parsing and editing querystrings inside
/// of non-Web code that doesn't have easy access to
/// the HttpUtility class.
/// </summary>
/// <remarks>
/// Supports multiple values per key
/// </remarks>
public class UrlEncodingParser : NameValueCollection
{
/// <summary>
/// Holds the original Url that was assigned if any
/// Url must contain // to be considered a url
/// </summary>
private string Url { get; set; }
/// <summary>
/// Always pass in a UrlEncoded data or a URL to parse from
/// unless you are creating a new one from scratch.
/// </summary>
/// <param name="queryStringOrUrl">
/// Pass a query string or raw Form data, or a full URL.
/// If a URL is parsed the part prior to the ? is stripped
/// but saved. Then when you write the original URL is
/// re-written with the new query string.
/// </param>
public UrlEncodingParser(string queryStringOrUrl = null)
{
Url = string.Empty;
if (!string.IsNullOrEmpty(queryStringOrUrl))
{
Parse(queryStringOrUrl);
}
}
/// <summary>
/// Assigns multiple values to the same key
/// </summary>
/// <param name="key"></param>
/// <param name="values"></param>
public void SetValues(string key, IEnumerable<string> values)
{
foreach (var val in values)
Add(key, val);
}
/// <summary>
/// Parses the query string into the internal dictionary
/// and optionally also returns this dictionary
/// </summary>
/// <param name="query">
/// Query string key value pairs or a full URL. If URL is
/// passed the URL is re-written in Write operation
/// </param>
/// <returns></returns>
public NameValueCollection Parse(string query)
{
if (Uri.IsWellFormedUriString(query,UriKind.Absolute))
Url = query;
if (string.IsNullOrEmpty(query))
Clear();
else
{
int index = query.IndexOf('?');
if (index > -1)
{
if (query.Length >= index + 1)
query = query.Substring(index + 1);
}
var pairs = query.Split('&');
foreach (var pair in pairs)
{
int index2 = pair.IndexOf('=');
if (index2 > 0)
{
Add(pair.Substring(0, index2), pair.Substring(index2 + 1));
}
}
}
return this;
}
/// <summary>
/// Writes out the urlencoded data/query string or full URL based
/// on the internally set values.
/// </summary>
/// <returns>urlencoded data or url</returns>
public override string ToString()
{
string query = string.Empty;
foreach (string key in Keys)
{
string[] values = GetValues(key);
foreach (var val in values)
{
query += key + "=" + Uri.EscapeUriString(val) + "&";
}
}
query = query.Trim('&');
if (!string.IsNullOrEmpty(Url))
{
if (Url.Contains("?"))
query = Url.Substring(0, Url.IndexOf('?') + 1) + query;
else
query = Url + "?" + query;
}
return query;
}
}
Short and simple – makes you wonder why this isn’t built into the core framework, right?
This code is self-contained so you can just paste it into your app, or you can get it as part of the Westwind.Utilities library from Nuget.
Applying it in my App
So inside of WebSurge I need to do URL replacement and it’s a cinch to do now by simply reading the original URL and its query string parameters and updating it with values from the list that the user provided.
The helper function that does this looks like this:
private string ReplaceQueryStringValuePairs(string url, string replaceKeys)
{
if (string.IsNullOrEmpty(replaceKeys))
return url;
var urlQuery = new UrlEncodingParser(url);
var replaceQuery = new UrlEncodingParser(replaceKeys);
foreach (string key in replaceQuery.Keys)
{
urlQuery[key] = replaceQuery[key];
}
return urlQuery.ToString();
}
Notice that this routine passes in a full URL and the URL is preserved by the parser which is a nice bonus feature and avoids having to deal with the logic of extracting and then re-appending the query string reliably with or without the ? required, which makes the app level code much cleaner.
If using System.Web, you can use HttpUtility
As I mentioned at the beginning if you’re inside of the context of a Web application you can easily use the HttpUtility class and it the ParseQueryString() method which provides you with a NameValueCollection that provides most of the same functionality. It won’t parse existing URLs and return them to you, but it will let you manage the actual UrlEncoded data.
Here’s an example of manipulating raw data similar to what I showed earlier:
var str = "id=123312&action=edit&format=json";
var query = HttpUtility.ParseQueryString(str);
query["Lang"] = "en";
query["format"] = "xml";
Console.WriteLine(query.ToString());
// id=123312&action=edit&format=xml&Lang=en
You can also create an empty collection that you can add to with:
var query = HttpUtility.ParseQueryString("");
It’s just a bummer that these general formatting routines are tied up in System.Web, rather than in System.Net with all the rest of the URI related formatting where it belongs. Well, maybe in the future.
In the meantime the above helper class is a way to easily add this functionality to your non-Web apps. Hope some of you find this useful.
Resources
Other Posts you might also like