Rick Strahl's Weblog  

Wind, waves, code and everything in between...
.NET • C# • Markdown • WPF • All Things Web
Contact   •   Articles   •   Products   •   Support   •   Advertise
Sponsored by:
West Wind WebSurge - Rest Client and Http Load Testing for Windows

Retrieving Web Resources and Content Types in Code


:P
On this page:

Last night I was working on updating some code in my library that handles script compression in relation to my ClientScriptProxy class. The ClientScriptProxy is a wrapper around ClientScript and ScriptManager that provides access to the right one depending on which is present or as an alternative uses custom script service to serve compressed scripts even in absence of a ScriptManager on a page if ScriptManager is not available (which for me is pretty much always <s>).

One thing I've been doing more of recently is to embed more CSS resources as resources and it would be kind of cool to get compression to work with these resources as well. Previously my script compression code only defaulted to JavaScript resources simply because that is the most common scenario for resources and CSS typically isn't all that large.

But one thing to make this work is to figure out the content type of the resource. 'Standard' .NET resources do not have any associated content type - they are just binary streams that can be read out of an assembly, which in turn makes it a little difficult to dynamically figure out what type of resource you are dealing with. However, ASP.NET's resources - resources that can be also be served through ASP.NET's WebResource mechanism - require additional information in the form of a WebResource attribute that does provide content type information that's tied to the resource name.

So any resource that is to be used by ASP.NET also needs a an assembly level attribute defined once somewhere in the code for the assembly.For example, here are some of my WebResource definitions for the West Wind Ajax toolkit:

[assembly: WebResource("Westwind.Web.Controls.Resources.jquery.js", "application/x-javascript")]
[assembly: WebResource("Westwind.Web.Controls.Resources.ww.jquery.js", "application/x-javascript")]
[assembly: WebResource("Westwind.Web.Controls.Resources.calendar.gif", "image/gif")]
[assembly: WebResource("Westwind.Web.Controls.Resources.ui.datepicker.js", "text/javascript")]
[assembly: WebResource("Westwind.Web.Controls.Resources.ui.datepicker.css", "text/css")]
[assembly: WebResource("Westwind.Web.Controls.Resources.wwscriptlibrary.js", "application/x-javascript")]

Each of thes e attributes matches the actual resource file that is also embedded into the assembly (using Embedded resource as the file type). Resources are named based on the project base namespace (set in the project's property settings) plus the project relative path. So in my project all resources are stored in a /resources folder below the project root with a default namespace of Westwind.Web.Controls.

EmbeddedResource

Normally to access the resources you use the Page.ClientScript or a ScriptManager instance to directly embed the resource into the page or (in the case of CSS typically) you can retrieve the URL to embed into the page:

string resName = "Westwind.Web.Controls.ww.jquery.js";
Page.ClientScript.RegisterClientScriptResource(typeof(ControlResources),resName);
string url = Page.ClientScript.GetWebResourceUrl(typeof(ControlResources), resName);

And that works well enough if that's what you're after.

Retrieving the Resources And the Content Type

In my situation though I'm effectively replacing some of the WebResource functionality specifically for script and css files since I want compression (both Gzip and in the case of script minimizing) but without requiring ScriptManager. So I need to be able to read the resource stream AND also be able to retrieve the content type from the Web Resource (if a Web Resource was passed) at least.

The process of reading the resource and getting the content type are two completely separate operations. As far as I could tell there is no public ASP.NET API that helps with reading Web resources IF you want to read them directly in code.

The process of getting the content type and reading the output looks something like this snippet:

// Look up the WebResource Attribute
WebResourceAttribute[] attr = resourceAssembly.GetCustomAttributes(typeof(WebResourceAttribute),false) 
as WebResourceAttribute[]; if (attr != null) cacheItem.ContentType = attr.Where(at => at.WebResource == resource).FirstOrDefault().ContentType;
// default to javascript if (cacheItem.ContentType == null) cacheItem.ContentType = STR_JavaScript_ContentType; // Load the script file as a string from Resources string script = ""; using (Stream st = resourceAssembly.GetManifestResourceStream(resource)) { StreamReader sr = new StreamReader(st, Encoding.Default); script = sr.ReadToEnd(); }
// Optimize the script by removing comment lines and stripping spaces
// Only applies to JavaScript
if (cacheItem.ContentType == STR_JavaScript_ContentType && !Context.IsDebuggingEnabled)
    script = OptimizeScript(script);

// *** Now we're ready to create out output
// *** Don't GZip unless at least 4k
if (useGZip && script.Length > 4096)
{
    cacheItem.Content = GZipMemory(script);
    cacheItem.IsCompressed = true;
}
else
{
    cacheItem.Content = Encoding.UTF8.GetBytes(script);
    cacheItem.IsCompressed = false;
}

// *** Add into the cache
Context.Cache.Add(cacheKey, cacheItem, null, DateTime.UtcNow.AddDays(1), 
TimeSpan.Zero, CacheItemPriority.High, null); // *** Write out to Response object with appropriate Client Cache settings this.SendTextOutput(cacheItem.Content, cacheItem.IsCompressed, cacheItem.ContentType);

Gotta love LINQ for making reflection queries like this easy (thanks to Dimebrain for the nudge via Twitter). So basically I have to retrieve the WebResourceAttributes separately and then retrieve the resource data using standard Assembly.GetManifestResourceStream. Not rocket science but a little concerned about performance since this might get hit fairly frequently. To avoid the repeated resource and reflection hits content - including the content type - is cached in the HttpContext cache which stores a special WebResourceItem as a cache entry that contains the content, the content type and whether the content was gZipped. All those elements need to be available if the content is served again.

This is good to know for other purposes as well. It might also be useful to build an app start up task that can optionally loop through all the resources in a script file and dump out the resources into a file directory of the app and then serve the resources from there - the advantage of this approach is that requests wouldn't have to go through ASP.NET at all.

To Resource or Not to Resource

I've brought up the subject of using Control resources on several occasions and I'm still not convinced of any one particular approach. When I build components that rely on support resources - say a component that relies on jQuery I tend to build resources into the assembly so there are no dependencies with the component loading jQuery, but with options to either specify an external file or not have the component load anything at all.

I think that this approach is pretty much a requirement or otherwise you might run into conflicts of versions or multiple loads of some libraries. Personally in my own apps I have most 'core' resources auto-loaded via resources and only support libraries (say jQUery UI components) loaded manually. Resources have big advantages in that the latest version is always available - no more copying around of resoures and making sure the latest version is loaded in many projects which really can get old.

What's your approach? Always embedding scripts manually even when building components? How do you manage dependencies and keeping scripts updated?

You can find the ScriptCompression and ClientScriptProxy code online in my Subversion repository  (Guest and no password)

On a side note: Does anybody know how to configure Apache and Subversion (in a VisualSvn distro) to make anonymous access work and yet still allow authenticated access for updates? I seem to only get one or the other to work using the require configuration switch but not both. I have either open access (for read and write for all) or ALWAYS require a username without anonymous access) .

Posted in ASP.NET  

The Voices of Reason


 

Matt Brooks
September 08, 2008

# re: Retrieving Web Resources and Content Types in Code

Hi Rick,

I tend to keep script files as script files and have them served statically by the web server. We host most of our web applications in-house and tend to do 'full-publish' update so this doesn't pose troublesome for us. However, I think the resource-based approach does lend it self nicely to dynamic/custom processing, such as the compression or 'minifying' as you've pointed out.

I appreciate this is only demonstration code but would it be better for the lines

if (attr != null)
    cacheItem.ContentType = attr.Where(at => at.WebResource == resource).FirstOrDefault().ContentType;


to actually read

if (attr != null)
    cacheItem.ContentType = attr.Where(at => at.WebResource == resource).First().ContentType;


to prevent the possibility of a NullReferenceException and get an easier to diagnose InvalidOperationException?

Matthieu
September 08, 2008

# re: Retrieving Web Resources and Content Types in Code

I think you must have the "third" module for authentication with Apache/SVN :

http://svnbook.red-bean.com/nightly/en/svn.serverconfig.httpd.html#svn.serverconfig.httpd.authz.perdir.ex-3
in the section "Per-directory access control"

and the sample of the policy file is here :
http://svnbook.red-bean.com/nightly/en/svn.serverconfig.pathbasedauthz.html

at the bottom of this page you got another sample:
http://www.csoft.net/docs/svndav.html.en

Rick Strahl
September 08, 2008

# re: Retrieving Web Resources and Content Types in Code

@Marc - Thanks for the feedback. The issue for me is that I have a bunch of controls that rely on resources. So I typically have at least two script files one of which is one that I build and modify (my internal library) so it changes frequently. it's been realy hard for me to keep this script file in sync across about 10 different applications. I often make changes when I am working on an app and promtly forget ending up doing frequent diffs in BeyondCompare - very time consuming. Having a single file source as a resource makes this much easier. But there are caveats too - script loading order can get hosed and dupes can arise especially when loading things like 'standard' resources like jQuery and making sure it laods first before any plug-ins (like my lib for example). No easy solutions, but for my lib plus the components I use it works well because 90% of the time I only load those two components and that's it.

As to the LINQ code - you're quite right, but the null check for the array should take care of that as null is returned if there are no resources. Still changed to First which is definitely more correct... Ah choices.

Rick Strahl
September 08, 2008

# re: Retrieving Web Resources and Content Types in Code

@Mattieu - THANK YOU! Yes that does the trick. With VisualSVN the following works for me:

<Location /svn/>
  DAV svn

  SVNListParentPath on
  SVNParentPath "c:/SvnWestWind/"
  SVNIndexXSLT "/svnindex.xsl"
  SVNPathAuthz on

  AuthName "Subversion Repositories"
  AuthType Basic
  AuthBasicProvider file
  AuthUserFile "c:/SvnWestWind//htpasswd"
  AuthzSVNAccessFile "c:/SvnWestWind//authz"

  Satisfy Any
  Require valid-user
</Location>


The key is the Satisfy Any before the Require valid-user which allows anonymous access. Seems to work - so the password's no longer required.

I think I'll repost this tomorrow so I can remember and find this again <s>.

DotNetKicks.com
September 08, 2008

# Retrieving Web Resources and Content Types in Code

You've been kicked (a good thing) - Trackback from DotNetKicks.com

Sergio Pereira
September 08, 2008

# re: Retrieving Web Resources and Content Types in Code

My choice between not embedding, embedding, and how to embed varied with the deployment scenario of the project. For a single (or a few) apps hosted in-house the static files still feel simple and manageable enough. If there too many apps or of the app/component is to be distributed to other teams or outside, then I try to embed them. It also happened once that we had several different components that used the same embedded resources (say jQuery and some icons), duplicating them in more than one assembly, with different URLs, causing the same files to be loaded (and potentially cached) twice by the browser. To overcome that we've put the shared resources in a separate assembly, referenced by the other components. Like you, many times we use the webResource attribute directly to do compression, concatenation, replacement of special syntax (think CSS resources with references to embedded images), localization, etc. In one case we ended up replacing the default WebResource attribute with our own, with more options.

Dave Ward
September 08, 2008

# re: Retrieving Web Resources and Content Types in Code

In my websites, I use static files.

In my server controls, I use embedded resources. I think a control having unnecessary external dependencies defeats the whole idea of encapsulating its functionality in the control.

Bertrand Le Roy
September 08, 2008

# re: Retrieving Web Resources and Content Types in Code

Maybe you're also doing that but it's not clear from just the code in the post: you might want to take advantage of output caching as well, which would be a lot more efficient than data caching.

Rick Strahl
September 08, 2008

# re: Retrieving Web Resources and Content Types in Code

@Bertrand - How do you use output cache from code? Isn't that effectively the same thing? Or are you talking about about the Response.CachePolicycache settings? I'm using public and server for the caching although frankly I am not 100% certain what the server portion of it does (kernal cache?). Is this what you're talking about? Otherwise think I'd need a hint...

The whole code is visible here:
http://www.west-wind.com:8080/svn/jQuery/jQueryControls/Support/ScriptCompressionModule.cs

and the code in question would be the SendTextOutput() method:

private void SendTextOutput(byte[] Output, bool UseGZip, string contentType)
{
    HttpResponse Response = HttpContext.Current.Response;
    Response.ContentType = contentType;
    Response.Charset = "utf-8";

    if (UseGZip)
        Response.AppendHeader("Content-Encoding", "gzip");

    //if (!HttpContext.Current.IsDebuggingEnabled)
    //{
    Response.Cache.SetCacheability(HttpCacheability.Public);
    Response.ExpiresAbsolute = DateTime.UtcNow.AddDays(1);
    Response.Cache.SetLastModified(DateTime.UtcNow);                
    //}

    Response.BinaryWrite(Output);
    Response.End();
}

Mike Gale
September 08, 2008

# re: Retrieving Web Resources and Content Types in Code

Depends on your audience and your situation.

I'm all for supporting the guy who knows what he's doing.

I've used controls that really annoy me, for example:
1) The Javascript resource has a problem, I know that if I loaded it manually I could most likely get it fixed.
2) The bundled CSS doesn't do something fundamental like changing text size when the user wants that. Or more decoratively it does not change when a site gets a "font makeover". Or worse still it uses a font-family-cascade that's clueless.

I want control and a simple architecture.

That tends to mean no resources.

If your target audience includes those who wouldn't have any of these problems maybe go for switchable resources, that you can configure into the game.

I recommend not shutting out the people who know how to tune things up, want to be in control of their application and get annoyed where they're locked out.

(For internal purposes it's a different ball of wax that depends...)

Rick Strahl
September 08, 2008

# re: Retrieving Web Resources and Content Types in Code

@Mike - Definitely agree with all of your points.

When I built resources into a control I always try to make sure that whatever resources are used are configurable. In fact, I'm rebuilding my control library at the moment and one thing that each control has is a Resource section where the default is WebResource, and the other options are to specify a url to the resource manually or leave it blank so you are responsible to load the resource on your own. This basically gives you all the options needed and allows mixing and matching.

It might also be good to maybe allow some 'application' level setting at least for things like CSS and scripts. So you can set in one place - always load from Resources or always load from ~/scripts for example so the configuration can be made in one place for the entire app. This gets a little difficult to document though <g>...

Matt Brooks
September 09, 2008

# re: Retrieving Web Resources and Content Types in Code

Rick,

You've mentioned a few times now that you disable ViewState on most of your pages. Does that mean you're re-binding data-bound controls on every page load? If so, are you querying the datastore for the same data (that was used on the previous page load) on every page load? Or are you caching the data so that the datastore doesn't have to be hit again?

In my experience disabling ViewState and re-binding on every page load often means you actually have to re-bind twice on post-backs. Once to populate the control with the same data from the previous page load and then again in response to the user's action that caused the post-back, e.g. page changed, sort changed, etc.

Rick Strahl
September 09, 2008

# re: Retrieving Web Resources and Content Types in Code

@Matt - yes I do tend to rebind data. Maybe that's a hold over from the pre-ASP.NET days when this was just the way it was done. In most cases I find this more efficient though than sending a boatload of ViewState to the client and then back up from the client on a POST back.

As to the double loading of data - if possible i try to bind data in OnPreRender() after events have fired and have some sort of data context (a data reader, L2S query or whatever) that gets set up during page process or is empty in which case the last data is loaded and then rendered.

This doesn't always work and yes in some situations this means rebinding twice. In some situations I'll also turn VS back on for certain controls if it makes sense (say the query(ies) are really expensive to get a relatively small chunk of data like a page in a list).

It can be more work, but in my experience it still results in more responsive pages than pages with tons of Viewstate.

Paul Mrozowski
September 09, 2008

# re: Retrieving Web Resources and Content Types in Code

I tend to want both. When I'm starting out with a control I want it to "just work" - no screwing around with trying to get pathing correct for the CSS or JS files. Once it's working I then tend to then want to switch to using the files externally so I can enable client-side caching on them to help speed things up; I don't even want them hitting the server just to ask "has it changed?" most of the time.

Bradvin
September 10, 2008

# re: Retrieving Web Resources and Content Types in Code

why do you not gzip files if they are less than 4K?

Rick Strahl
September 10, 2008

# re: Retrieving Web Resources and Content Types in Code

@Bravdin - not worth it. There's overhead in GZip on the server and if files are small there's really not a big of a reason to compress. Maybe 4k is a little too big - maybe 2k or even 1k is more apppropriate.

Steve Hobbs
September 17, 2008

# re: Retrieving Web Resources and Content Types in Code

Hi Rick,

With regards to your gripes with ViewState, have a look at http://www.codeproject.com/KB/aspnet/ViewStatePersistance.aspx, which explains how one can persist the viewstate on the server, which saves you from having to send the viewstate to the client and back again.

Obviously only useful depending on how large the viewstate is, but you can engineer it to apply to all pages or just a single page, and is a fairly pluggable solution. A little more work would be required if you're working with a server farm obviously, but so far it has worked very well for my single server-hosted scenarios.

# Webresource Bookmarks

Bookmarked your page with keywords webresource!

West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2024