Rick Strahl's Weblog  

Wind, waves, code and everything in between...
.NET • C# • Markdown • WPF • All Things Web
Contact   •   Articles   •   Products   •   Support   •   Advertise
Sponsored by:
West Wind WebSurge - Rest Client and Http Load Testing for Windows

Building a GZip JavaScript Resource Compression Module for ASP.NET


:P
On this page:

Script compression using GZip is a fairly important issue and as I’m building my client library and the library is growing I’ve been meaning for some time to build this Compression module. I figured this would be a quick thing to do, but it ended up taking me a good part of a day to get this all worked out and working correctly this weekend. Here’s what I ended up with.

 

The idea was to have this work with minimal overhead and minimal configuration and the compression should work if the module is enabled, and if it isn’t the code should fall back to using the standard Web Resource mechanism.

 

I had mentioned this before – my hope was that there would be SOME way to hook into the HTTP pipeline without any web.config changes, but alas I couldn’t find a way to hook either a module or handler entirely programmatically anywhere but in HttpApplication.OnInit. So Bertrand was right when he said he was ‘almost 100%’ certain it couldn’t be done <g>… Consider that an enhancement request (and I bet it's one Microsoft wishes themselves they'd have made for ASP.NET 2.0 <g>).

 

So the next best thing that I thought of was to use a Module and tie it into my ClientScriptProxy utility I created for compatibility with MS Ajax. MS Ajax as you know, actually supports script compression, and you get it for free with any scripts you load throug, but only if a ScriptManager is on the page, which also incurs the overhead of the MS Ajax client libraries.

 

So, since ClientScriptProxy is a front end for both ClientScriptManager and ScriptManager I already have a hook that allows me to create the appropriate Resource urls on the fly. To make this work I created a modue and overrode the RegisterClientScriptResource method in the ClientScriptProxy. The Module needs to have a flag to let the method know wether it’s loaded so it can decide whether to load MS Ajax ScriptManager resources or my compressed resources or if neither of those are available continue to serve standard WebResource.axd links.

 

 I created the module which needs to be installed in web.config as usual, and it has a public static wwScriptCompressionModuleLoaded property which is set when the module starts up:

 

public class wwScriptCompressionModule : IHttpModule

{

 

    public static bool wwScriptCompressionModuleActive = false;

 

    public void Init(HttpApplication context)

    {

        wwScriptCompressionModuleActive = true;

        context.PostResolveRequestCache += new EventHandler(this.PostResolveRequestCache);

    }…

}

 

This gives a flag that can be used to check whether the module is installed. Then in the ClientScriptProxy class I can check for this flag and if the module is present use it to serve script resources instead. ClientScriptProxy checks for an MS Ajax script manager on the page first and if so uses it to register the resource, then checks for the script module and if that’s not there just uses ClientScript.

 

The code for this looks like this:

 

/// <summary>

/// Returns a WebResource or ScriptResource URL for script resources that are to be

/// embedded as script includes.

/// </summary>

/// <param name="control"></param>

/// <param name="type"></param>

/// <param name="resourceName"></param>

public void RegisterClientScriptResource(Control control, Type type, string resourceName)

{

    if (this.IsScriptManagerOnPage(control.Page))

    {

        // *** NOTE: If MS Ajax is referenced, but no scriptmanager is on the page

        //           script no compression will occur. With a script manager

        //           on the page compression will be handled by MS Ajax.

        if (RegisterClientScriptResourceMethod == null)

            RegisterClientScriptResourceMethod = scriptManagerType.GetMethod("RegisterClientScriptResource",

                 new Type[3] { typeof(Control), typeof(Type), typeof(string) });

 

        RegisterClientScriptResourceMethod.Invoke(null,

                       new object[3] { control, type, resourceName });

        return;

    }

 

#if IncludewwScriptingModuleSupport

    // *** If wwScriptCompression Module through Web.config is loaded use it to compress

    // *** script resources by using wcSC.axd Url the module intercepts

    if (wwScriptCompressionModule.wwScriptCompressionModuleActive)

    {

        if (type.Assembly == this.GetType().Assembly)

 

            RegisterClientScriptInclude(control, type, resourceName, "wwSC.axd?r=" +

                                        Convert.ToBase64String(Encoding.ASCII.GetBytes(resourceName)));

        else

            RegisterClientScriptInclude(control, type, resourceName, "wwSC.axd?r=" +

                                        Convert.ToBase64String(Encoding.ASCII.GetBytes(resourceName)) +

                                        "&t=" +

                                        Convert.ToBase64String(Encoding.ASCII.GetBytes(type.Assembly.FullName)));

    }

    else

#endif

        // *** Otherwise just embed a script reference into the page

        control.Page.ClientScript.RegisterClientScriptResource(type, resourceName);

}

 

The module then hooks a call to wwSC.axd which is trapped by the module and compresses the script code. Note that there’s a slight optimzation here if the assembly is the same as the one that contains the resource module it doesn’t require passing the type information which makes for a shorter URL.

 

Building the script compression module is straight forward enough – the GZip code is built into the .NET framework and is almost childishly easy to implement. Still there’s a fair amount of code in the module that deals with various scenarios:

 

  • Checking whether GZip is supported by the client
  • Caching script content both for GZip and straight through
  • Sending back optimized script code in non-debug mode

 

Here’s what the ScriptCompression module looks like:

/// <summary>

/// Module that handles compression of JavaScript resources using

/// GZip and some basic code optimizations that strips full line

/// comments and whitespace from the beginning and end of lines.

///

/// This module should be used in conjunction with

/// ClientScriptProxy.RegisterClientScriptResource which sets

/// up the proper URL formatting required for this module to

/// handle requests. Format is (with base64 encode values):

///

/// wwScriptCompression.ashx?r=ResourceName&t=FullAssemblyName

///

/// The type parameter can be omitted if the resource lives

/// in this assembly.

/// <remarks>

/// * JS resources should have \r\n breaks not \n

/// * Script Optimization strips any comment lines starting with //

///   but not any other comments

/// </remarks>

/// </summary>

public class wwScriptCompressionModule : IHttpModule

{

 

    public static bool wwScriptCompressionModuleActive = false;

 

    public void Init(HttpApplication context)

    {

        wwScriptCompressionModuleActive = true;

        context.PostResolveRequestCache += new EventHandler(this.PostResolveRequestCache);

    }

    public void Dispose()

    {

    }

   

    private void PostResolveRequestCache(object sender, EventArgs e)

    {

        HttpContext Context = HttpContext.Current;

        HttpRequest Request = Context.Request;

 

        // *** Skip over anything we don't care about immediately

        if (!Request.Url.LocalPath.ToLower().Contains("wwsc.axd"))

            return;

 

        HttpResponse Response = Context.Response;

        string AcceptEncoding = Request.Headers["Accept-Encoding"];

 

        // *** Start by checking whether GZip is supported by client

        bool UseGZip = false;

        if (!string.IsNullOrEmpty(AcceptEncoding) &&

            AcceptEncoding.ToLower().IndexOf("gzip") > -1 )

            UseGZip = true;

 

        // *** Create a cachekey and check whether it exists

        string CacheKey = Request.QueryString.ToString() + UseGZip.ToString();

       

        byte[] Output = Context.Cache[CacheKey] as byte[];

        if (Output != null)

        {

            // *** Yup - read cache and send to client

            SendOutput(Output, UseGZip);

            return;

        }

 

        // *** Retrieve information about resource embedded

        // *** Values are base64 encoded

        string ResourceTypeName = Request.QueryString["t"];

        if (!string.IsNullOrEmpty(ResourceTypeName))

            ResourceTypeName = Encoding.ASCII.GetString(Convert.FromBase64String(ResourceTypeName));

 

        string Resource = Request.QueryString["r"];

        if (string.IsNullOrEmpty(Resource))

        {

            SendErrorResponse("Invalid Resource");

            return;

        }

        Resource = Encoding.ASCII.GetString(Convert.FromBase64String(Resource));

       

        // *** Try to locate the assembly that houses the Resource

        Assembly ResourceAssembly = null;

 

        // *** If no type is passed use the current assembly - otherwise

        // *** run through the loaded assemblies and try to find assembly

        if (string.IsNullOrEmpty(ResourceTypeName))

            ResourceAssembly = this.GetType().Assembly;

        else

        {

            ResourceAssembly = this.FindAssembly(ResourceTypeName);

            if (ResourceAssembly == null)

            {

                SendErrorResponse("Invalid Type Information");

                return;

            }

        }

 

        // *** Load the script file as a string from Resources

        string Script = "";

        using (Stream st = ResourceAssembly.GetManifestResourceStream(Resource))

        {               

            StreamReader sr = new StreamReader(st);

            Script = sr.ReadToEnd();

        }

 

        // *** Optimize the script by removing comment lines and stripping spaces

        if (!Context.IsDebuggingEnabled)           

            Script = OptimizeScript(Script);               

                   

        // *** Now we're ready to create out output

        // *** Don't GZip unless at least 8k

        if (UseGZip && Script.Length > 6000)

            Output = GZipMemory(Script);

        else

        {

            Output = Encoding.ASCII.GetBytes(Script);

            UseGZip = false;

        }

 

        // *** Add into the cache

        Context.Cache.Add(CacheKey, Output, null, DateTime.UtcNow.AddDays(1), TimeSpan.Zero, CacheItemPriority.High,null);

       

        // *** Write out to Response object with appropriate Client Cache settings

        this.SendOutput(Output, UseGZip);

    }

 

   

    /// <summary>

    /// Returns an error response to the client. Generates a 404 error

    /// </summary>

    /// <param name="Message"></param>

    private void SendErrorResponse(string Message)

    {

        if (!string.IsNullOrEmpty(Message))

            Message = "Invalid Web Resource";

 

        HttpContext Context = HttpContext.Current;

 

        Context.Response.StatusCode = 404;

        Context.Response.StatusDescription = Message;

        Context.Response.End();

    }

 

    /// <summary>

    /// Sends the output to the client using appropriate cache settings.

    /// Content should be already encoded and ready to be sent as binary.

    /// </summary>

    /// <param name="Output"></param>

    /// <param name="UseGZip"></param>

    private void SendOutput(byte[] Output, bool UseGZip)

    {

        HttpResponse Response = HttpContext.Current.Response;

 

        Response.ContentType = "application/x-javascript";

        if (UseGZip)

            Response.AppendHeader("Content-Encoding", "gzip");

 

        if (!HttpContext.Current.IsDebuggingEnabled)

        {

            Response.ExpiresAbsolute = DateTime.UtcNow.AddYears(1);

            Response.Cache.SetLastModified(DateTime.UtcNow);

            Response.Cache.SetCacheability(HttpCacheability.Public);

        }

 

        Response.BinaryWrite(Output);

        Response.End();

    }

 

 

    /// <summary>

    /// Very basic script optimization to reduce size:

    /// Remove any leading and trailig white space from lines

    /// and any lines starting with //.

    /// </summary>

    /// <param name="Script"></param>

    /// <returns></returns>

    public static string OptimizeScript(string Script)

    {

        string[] Lines = Script.Split(new string[1] { "\r\n" }, StringSplitOptions.RemoveEmptyEntries);

        StringBuilder sb = new StringBuilder();

        foreach (string Line in Lines)

        {

            string LineContent = Line.Trim();

 

            // *** Remove full comment lines

            if (LineContent.StartsWith("//"))

                continue;

            sb.AppendLine(LineContent);

        }

        return sb.ToString();

    }

 

 

    /// <summary>

    /// Finds an assembly in the current loaded assembly list

    /// </summary>

    /// <param name="TypeName"></param>

    /// <returns></returns>

    private Assembly FindAssembly(string TypeName)

    {

        foreach (Assembly ass in AppDomain.CurrentDomain.GetAssemblies())

        {

            if (ass.FullName == TypeName)

                return ass;

        }

 

        return null;

    }

   

    /// <summary>

    /// Takes a binary input buffer and GZip encodes the input

    /// </summary>

    /// <param name="Buffer"></param>

    /// <returns></returns>

    public static byte[] GZipMemory(byte[] Buffer)

    {

        MemoryStream ms = new MemoryStream();

 

        GZipStream GZip = new GZipStream(ms, CompressionMode.Compress);

 

        GZip.Write(Buffer, 0, Buffer.Length);

        GZip.Close();

 

        byte[] Result = ms.ToArray();

        ms.Close();

 

        return Result;

    }

 

 

    public static byte[] GZipMemory(string Input)

    {

        return GZipMemory(Encoding.ASCII.GetBytes(Input));

    }

 

}

 

The code’s a bit light on error handling, which for the moment is on purpose <s>. I’m not sure if it’s worth to catch errors explicitly and throw back a 404 error on failure or whether to let a standard 500 error message bubble back instead. Either way the script isn’t going to get served and at least with the error there’s a way to see what happened (either in the page in local debug or in an error log). <shrug>

 

The module caches the script content and it does so separatatng the Gzipped code and the plain version for each script served if requested. I haven’t done any heavy duty testing but it seems that with caching performance should be no problem.

 

As to compression: My main script library is 39+k and the GZip compression reduces and optimization reduces that down to a little over 9k. Without the text optimization it’s about 10.5k so the script compression reduces another 10%. The module is also used for a number of smaller support scripts.

 

Note the module is specific to script compression – it’s not meant to compress anything else although it could be modified (or maybe better copied as a generic text compression module) to do so fairly easily. I don’t see much need for this though beyond script code. Images are already internally compressed and I don’t find myself feeding any text content through resources. Possibly CSS but that’s very rare so I haven’t bothered… The main thing that seems to be growing in size is scripts, hence this module.

 

I happy to have finally put this into my lib. Actually I wish I had done this a little sooner yet, because I could have coded the library with comments <s>. I hate having non-commented code – especially gooey JavaScript code that I won’t remember WTF I did another month from now <s>. The comment stripping is certainly going to be nice going forward. Of course if leave ATLAS to do the script compression that won't work, but ATLAS supports feeding script.debug.js so it's possible to do this by pre-processing possibly at compile time. Hmmm... that might be another thing to support - project for another day <s>.

 

The code is part of the latest wwHoverPanel download and you can find both the ClientScriptProxy and the wwScriptCompression module in that code. The ClientScriptProxy is not required for use, but you will need to hook up some mechanism to actually generate the script resource URLs as shown above.

 

Posted in Ajax  ASP.NET  West Wind Ajax Toolkit  

The Voices of Reason


 

Speednet
January 23, 2007

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

Rick,

Great stuff!

Here's an enhanced version of the OptimizeScript() function I whipped up to utilize Regular Expressions, which will be faster, more flexible, maintainable, etc., etc.

It can be boiled down to a couple of lines, but since you mentioned that you love comments, it is a little beefier! ;-)

It also could be made much more aggressive, but this is a good balance if SPEED vs. aggressiveness, and still leaves the file as readable on the client. All of the Regex tests are very quick comparisons.

-Todd

public static string OptimizeScript(string Script)
{
    // Remove all instances of /* .... */ spanning multiple lines if necessary
    string optimized = Regex.Replace(Script, "/\*.*?\*/\s*\n*", "", RegexOptions.Singleline);

    // Remove comments and whitespace, line-by-line:
    // "^\s*//.*$\n?" - Single-line comment, also look for leading whitespace and trailing newline
    // "^\s*$\n" - Blank line (with or without whitespace) and trailing newline
    // "^\s+" - Leading whitespace
    // "\s+$" - Trailing whitespace
    optimized = Regex.Replace(optimized, "^\s*//.*$\n?|^\s*$\n|^\s+|\s+$", "", RegexOptions.Multiline);

    return optimized;
}

Speednet
January 23, 2007

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

A couple of small corrections to my last post, my apologies in advance.

The first correction is that the Regex patterns need a "@" before them, so that the backslashes are not interpreted as escape sequences.

The second correction is an enhancement to the first Regex pattern, which guards against the possibility that the first "/*" it finds is actually part of a character string and should be left alone. It also guards against embedded escape sequences in character strings. It is extremely robust, yet coded to run very quickly, as it parses big chunks of the string.

Here are both corrected lines of code:

string optimized = Regex.Replace(Script, @"([^\"\'/]+|\"[^\"\\]*(?:\\.[^\"\\]*)*\"[^\"\'/]*|'[^'\\]*(?:\\.[^'\\]*)*'[^\"\'/]*)|/\*.*?\*/", "$1", RegexOptions.Singleline);
optimized = Regex.Replace(optimized, @"^\s*//.*$\n?|^\s*$\n|^\s+|\s+$", "", RegexOptions.Multiline);

Bill Pierce
January 23, 2007

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

Hey Rick,
I feel some wheel-reinvention here, can you help me out? IIS 6 has built in compression based on file extensions. Is there a reason you don't just add axd to the IIS6 configuration?

Ken Egozi
January 23, 2007

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

afaik - using IIS compression on dynamic content results in non cached output.
Furthermore, on shared-hosting scenarios, the hosting company won't willingly enable it to avoid server load.

This implementation is somewhat smart in it's caching approach. I use something quite similar, however without the MS-AJAX magic since I work with Castle MonoRail.

Rick Strahl
January 23, 2007

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

Bill in addition to what Ken said you don't want compression on every resource. Images are already compressed so compressing them is not likely to yield any improvements. There's no good way that I could find to use WebResource.axd short of completely replacing it... Also, configuring IIS 6 at least for compression is a pain in the ass. This will be much easier in IIS 7, but until that's out in the wild I still think that using module code is the better choice.

Ken, the way the headers are set up I think IIS 6 will also kernel cache the resources.

Rick Strahl
January 24, 2007

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

Ken just as side note - there's no dependency on MS Ajax in this code. The MS Ajax code above is only CHECKING for MS Ajax optionally and using it if a PageManager exists, but that can be removed if not required - that code requires some additional code that's not shown here anyway.

Rick Strahl
January 25, 2007

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

Thanks Todd for the RegEx optimizations. I'll add this when I get a little more time to test...

Rick Strahl's Web Log
February 06, 2007

# More on GZip compression with ASP.NET Content - Rick Strahl's Web Log

Now that GZip is natively available in .NET 2.0 it's very easy to compress your ASP.NET content with GZip compression.

Shan Plourde
February 08, 2007

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

Just a quick comment on IIS GZip compression - it can be configured to only compress by certain file extensions. I'm using it on my company's Intranet. I'm sure in hosted environments this type of thing makes a ton of sense

Alex
February 20, 2008

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

hmm, aren't we supposed to remove our PostResolveRequestCache event handler in IHttpModule.Dispose? All IHttpModule code-samples on the Net (including this one) never implement the Dispose method, I wonder why... Anyway, great post. I'm surfing this blog for many hours now and it's a real gem!! THANKS Rick.

Rick Strahl
February 20, 2008

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

@Alex - yes we probably should but it doesn't matter much because modules get loaded for the lifetime of the ASP.NET AppDomain. By the time it gets released everything shuts down completely and the GC will clear out anything left in memory anyway. IOW, it's not really necessary.

DotNetShoutout
November 19, 2008

# Building a GZip JavaScript Resource Compression Module for ASP.NET

Your Story is Submitted - Trackback from DotNetShoutout

Sameer
June 02, 2009

# re: Building a GZip JavaScript Resource Compression Module for ASP.NET

Dont you guys think you are reinventing the wheel again? I understand Rick's JS minimization is very minimal and safe but if you want to do comprehensive regexp minimization shouldn't you just use something like JSMin (http://www.crockford.com/javascript/jsmin.html) ?

West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2024