I've been thinking a bit about JavaScript inclusion in the last few weeks as I've been working on quite a few different Web projects that use a fair bit of JavaScript functionality. One thing that I'm still trying to feel good about is how JavaScript is included into pages. There are a number of different ways that script embedding can be done especially with ASP.NET and none of them are really perfect.
So what's the Problem?
The issue is plain and simple, how do you deal with JavaScript inclusion into the page given that you have a fair number of JavaScript files that are bound to change frequently and across multiple projects/applications? My assumption here is that I'm using a few libraries and utility classes both public (jQuery and jQuery plug-ins mainly) and some internally developed support libraries. All in all many pages I deal with have typically 4 - 8 related JavaScript resources in AJAX centric applications with about 15-20 JavaScript resources overall involved in a typical project. In addition there are a few ASP.NET custom controls that depend on some of these Ajax libraries and components.
My project load currently runs about 10 concurrent projects both internal and external and all of them are making use of these same script resources. Keeping versions of both my internal JavaScript libraries as well as external libraries in sync and up to date for each of these applications both in development and online gets to be challenging especially if the files are not necessarily delivered as a unified whole (ie. often I pick and choose which plug-ins to use).
So as you can probably see, it's pretty easy to be in 'script file hell' - I find myself using Beyond Compare quite frequently to compare what's changed across different projects and update scripts precariously. I don't think there are easy solutions to this problem other than being very organized and managing versions as anally as possible but doing so can be a real drag on productivity as I move from project to project wondering if I have the correct current version.
There are many different ways that JavaScript resources can be loaded into pages from the basic manually managed header embedding of each script to using full ASP.NET resource management of all script files used. And plenty of other options in between. Try as I might I have not found a solution that really makes me feel all warm and fuzzy, so I'm curious what you are doing and what works best for you.
But let's review some of the approaches that are available to script embedding. For a specific example, let's take the more generic case of the jQuery library. Here are some of the options you have to get jQuery into your page:
- Include in a scripts folder and link via script include as jQuery.js
- Include in a scripts folder and link via script include as jQuery-1.2.6.js (ie. version specific)
- Link to the version specific file at jQuery.com (typically jQuery-1.2.6.js)
- Link to the latest version at jQuery.com (ie. jQuery-latest.js)
- Link to some other script hosting site (ie. Google's Ajax Lib API) via script loading
- Embed scripts into ASP.NET Resources
- Use ScriptManager to load scripts either from Folder or Resources
- Use a custom script manager
All of these are options but they all have a few shortcomings as well.
Local Script Files
The most common and easiest to understand approach is to simply embed local script links into the page. Add scripts into the header with <script src="scripts/jquery.js"> and off you go. In many situations this approach is perfectly fine, especially if script files are static and don't change frequently.
<head runat="server">
<title>Configuration Settings</title>
<link href="Standard.css" rel="stylesheet" type="text/css" />
<script src="scripts/jquery.js" type="text/javascript"></script>
</head>
The issue with locally stored script files is that they they need to be updated if files changed. In a single application this is not a big issue. But if you're managing a large number of projects, changes to resusable scripts both of your or third party script libraries can easily become a burden and an exercise in version management. It'd be great if source control would solve this problem, but because resources are often mixed and matched storing a fixed set of script files under source control also doesn't really address this scenario either and wouldn't work very well anyway if the project is already under Source Control.
In addition, if you start using script code as part of related ASP.NET controls that might build on top of client script components and that code has some dependencies on specific versions you can run into subtle versioning problems that are difficult to detect and test for. But even for relatively simple things like the static core jQuery library this can get sticky as new versions are frequently distributed.
Versioning JavaScript files in general is a sticky issue. If you have local files, how do you version these JS resource? For example, many jQuery examples are shown using the version specific file like so:
<script src="~/scripts/jquery-1.2.6.js" type="text/javascript"></script>
This addresses versioning alright but then gets tedious when a new version gets released and all pages referencing this version needs to be updated. Again, with a single file this isn't terrible but now think about 10-20 resources used in an application that are all versioned this not really a good option. The other choice is to use a non-versioned file and just update the files - this is easier for updates but in turn can result in subtle feature incompatibilities if newer esions break old builds/dependencies.
I personally lean towards the latter approach of using non-versioned filenames and take my chances hoping that changes are backwards compatible. In my own libraries even a breaking change is easily caught because I know what was changed intimately, but with third party libraries this can be tricky. For example, while using the jQuery.ui beta versions there many subtle breaking changes that were difficult to catch once the code was migrated to the release version because 95% of the code worked great. Testing JavaScript code consistently - especially UI code - is difficult at best and it's easy to have a subtle bug to not show up until the app is in production for a while.
One Library Path for All Scripts
Another option for 'local' script files might be to store all scripts in one place - say off the root of the Web site similar to the way ASP.NET 1.1 used to store resources in a /WebClient folder - and access them through this single script location. This solves the problem of having to copy versioned files around for each application and having a single store to update third party libraries and modify internal libraries. If there's a single store it's also easier to put the files under source control and manage them effectively.
<script src="/scripts/jquery.js" type="text/javascript"></script>
But this approach also has a few problems. First it's not a generic solution because it requires that you use the same convention across applications. Might work fine for your own applications where you set policy, but if you're working to somebody else specs it may not be possible to dump scripts into a specific consistent location.
It also may not necessarily be Ok to update scripts for every application at once, especially if scripts have breaking changes that might not work with older code not adapted to it. If you work with script files that are dependent on one another and these files are updated out of band it's easy to run into version problems. This is always a concern and not having local copies or linking to a single non-versioned script file is prone to potential version issues. Having local files in each app allows isolation from this version issue that can crop up with a single shared library folder. This issue is probably not a major concern
I played around with this approach some time ago, but tossed the idea because of issues
External Hosting
Another centralized approach is to use external hosting for certain script resources. This is more applicable for public librarires like jQuery, Prototype etc. and some high profile plug-ins available for them. External hosting also offloads the bandwidth for downloading scripts from your site to whatever public site hosting the scripts. Additionally, chances are that these script resources may be already cached in user browsers as the public resources are more widely used by users in general browsing which provides additional performance gains.
For example, you can load the latest jQuery version from:
<script src="http://www.jquery.com/src/jquery-latest.js" type="text/javascript"></script>
or a version specific one from the
Google jQuery code repository:
<script src="http://jqueryjs.googlecode.com/files/jquery-1.2.6.js" type="text/javascript"></script>
Most other major libraries also provide similar direct links to libraries so that you can offload loading from your site.
Google recently also started hosting various JavaScript libraries with the Google's Ajax Lib API. This API allows you to load script via JavaScript code in the page and allows you to specify specific version numbers as part of the API call. This is a clever way to get a better handle on versioning as you can potentially use single script file to in your app to load various script files with 'factory' function calls that load each library with the appropriate version. FWIW, it doesn't require Google's API to do this; you can take a similar approach by yourself by creating a small loader script that can be updated for each application (see below on how this can be done with jQuery for example).
The disadvantage for external hosting is of course that you are dependent on an external provider to be up and running 100% of the time and to have adequate bandwidth to serve the script files.The provider can decide any time to remove files or APIs so at that point you may end up with bad links in your code. This is where an API - whether your own or Google's can actually buy some advantage because it can be changed in one place.
Another concern is that scripts loaded across sites can also trigger warnings in the browser in some situations like SSL connections (to non SSL script resources) or strict browser requirements for cross domain loading of any files.
Dynamic Loading
As mentioned above you can also load script dynamically from other script code. For example, it might be useful to load one script from another script to provide dependency loading. In jQuery for example you can do:
/// <reference path="jquery.js" />
$.getScript("scripts/basepage.js");to pull in additional script code.
One potential advantage of this approach is that you can build a simple loading api that turns scripts into resources with maybe some constants that determine which scripts to load and from where, consistently across an application. Rather than linking all scripts in pages you can call a single script file that handles loading all other scripts.
However, when loading script dynamically it's important to be very careful and ensure that the code in the dependency is not accessed before the script has actually loaded. This can have tricky side effects. In one app I was working on just today pages would call into a loaded script function and it would work just fine the first time the page loaded or when a full refresh occurred, but not when the page was just reloaded via click (ie. most resources are already cached). The difference is that the other scripts are fully cached and so load time is significantly faster and the code executes before the second script page loads. In this case IE fails to see the function on page startup (even in $(document).ready() in jQuery) while FireFox, Opera and Safari properly delay the loading code.
Most JavaScript libraries allow working around this via events fired when the script loading completes. In jQuery there's a handler that can be passed as a second parameter that's a handler:
$.getScript("scripts/wwJquery.js",function() { showStatus("Ready");});
which will reliably cause dependent code to run. But it's not always feasible to tie code to a specific library having been loaded. Often you have to wait on multiple libraries. Oddly I've seen a host of problems with this approach both with jQuery, the Google API while script loading from the page's <script> tags seems to work just fine. Detecting script completion across browsers is tricky especially in IE it appears.
I have this approach on my list of things to play around with someday, but given the mixed results I've seen with load order failures I'm incline to pass this option up completely.
ASP.NET Resources
When using ASP.NET it's also possible to rein in JavaScript resources by building them into ASP.NET compiled assemblies as Resources. Using this approach allows the resources to be stored in a single location that is centrally managed. Web Applications can then access these resources by calling Page.ClientScript.RegisterClientScriptInclude(), Page.ClientScript.RegisterClientScriptResource() or the methods of the same name on the ASP.NET ScriptManager control.
When using resources in a separate non-Web project, the project can be added other solutions and so can be shared across multiple projects which can both read the resources and modify any of them and have the changes reflected in any other application that includes the resource (or more often control ) assembly.
I use ASP.NET resources a lot for a variety of purposes from images to scripts to css files and it's a great way to put everything into one place. But resources are not without their problems either. The worst issue by far is that they produce hideously long URLs that are non-transparent in the page and you can't easily tell what each script file points at. They also take up a fair bit of space (a typical WebResource.axd url can be 150 chars long). Additionally the API to actually load these script files is a code only API that needs to be handled in Codebehind of the page with no HTML page markup equivalent.
Resources are a must if you're building controls that have script dependencies in my opinion and that's how I've been using them. I use my own script library that has both client and server components and the server controls need to ensure that scripts are loaded appropriately into the page. Without resources these controls would have to explicitly require the developer to add scripts to the page which would be a bit hokey. So the control internally loads the appropriate resources itself and in the right order and so ensures that the component is self contained.
On the flip side using resources this way can be a problem if you also use the same library for other coding. The smaller problem is that it's very easy to double load resources when you forget that a component loads the resources internally. Further it's possible that the component loads the resources too late or in the wrong order from what the same page code might require if there are dependencies.
In worse cases there may be issues where there are load order problems where the page control order rather than manual ordering end up determining the order in which scripts load. Again, this can become tricky if you have libraries that depend on each other like say a jQuery and a jQuery plug-in, or my personal library that might depend also on jQuery in certain parts.
Resources also don't work well with Intellisense, except when using them with the ScriptManager control (see below). Otherwise the resource URLs aren't transparent so there's no easy way to embed scripts into other JavaScript resources as /// <script reference="" />.
The big problem is that ASP.NET natively doesn't have a native script loading API that works both for markup and code based script injection. Because the ClientScriptManager is a code only component it's very difficult to coordinate script loading both through markup and CodeBehind or component code and it can get very tricky to coordinate script dependencies loaded from resources if those same scripts also need to be loaded for other code.
ASP.NET Script Manager
Ah, but what about the new ScriptManager control that's part of Microsoft ASP.NET AJAX? It addresses a number of the issues that I've mentioned above. Indeed, ScriptManager provides a mirror interface for most of the ClientScript functions so you get the same functionality as the ClientScript for code manipulation and you get the actual control which can be stuck into a page and provide markup based script injection into the page. You can embed both physical url based scripts links as well as resource links so it's possible to consistently embed scripts from both. The control also detects script duplication and only embeds only a single script reference into the page so it's easier for Markup and Code injected scripts to not generate multiple script references. It's possible to add a script like jQuery into the page and ensure that it doesn't get embedded twice for example.
But ScriptManager is also very problematic. First it's a .NET 3.5 component so it's really and add-on and not universally available. There's also no easy way to detect whether ScriptManager is available in a page without using the ScriptManager in the first place (it's possible but it's a royal pain). ScriptManager also requires a control on the page - it's not a standalone API which complicates using it in generic solutions especially since only one ScriptManager is allowed per page. Finally last but not least the control arrogantly throws the Microsoft ASP.NET AJAX ClientLibraries into the page (at least 28k of compressed script for minmal support) with no option to turn that off. If you're not using ASP.NET AJAX like myself the last thing I want is to get a bunch of scripts I'm not using thrown into my page.
Custom Script Manager
My first thought when seeing ScriptManager's issue of AJAX script loading was to subclass the control. It's easy to turn off the AJAX script injection, but unfortunately ScriptManager can't be subclassed in such a way that it still provides Intellisense because apparently Visual Studio's Intellisense for script Intellisense is hard coded to look for ScriptManager's specific properties. Script Intellisense is quite useful so ontop of the other shortcomings of SM I decided to just build my own.
The control I ended up with is similar in concept, but with better focus on externally hosted scripts.
<ww:wwScriptContainer ID="scripts" runat="server" RenderMode="Header">
<Scripts>
<Script Src="Scripts/jquery.js" Resource="jquery"></Script>
<Script Src="Scripts/ui.core.js" AllowMinScript="true"></Script>
<Script Src="Scripts/ui.draggable.js" ></Script>
<Script Src="Scripts/wwscriptlibrary.js" Resource="wwscriptlibrary"></Script>
<Script Resource="ControlResources.Menu.js" ResourceControl="txtInput" />
</Scripts>
</ww:wwScriptContainer>
The control can either be embedded into the page or there's a .Current property that can be used from code that either finds an existing instance or creates a new control and adds it to the page dynamically which is useful for controls that want to embed resources. This gives a single API both for markup and code based embedding.
While working on this there were are a few other things that came up as useful: Optional automatic usage of minimized scripts (.min.js) extensions, abillity to embed all scripts into the page header (or inline, or as scripts) rather than adding into the page, the ability to order scripts by inserting at the end or beginning, as well as determining on a per script basis where it renders (Header, Script, Inline or InheritedFromControl) all of which gives some additional control both for markup and code behind. You can also embed resources but use reference a script on disk (for example in a central Web location) in order to still get Intellisense. The latter isn't as nice as ScriptManager's ability to just work off resource Urls, but at least it allows for some Intellisense of resource based scripts.
The format of the control is a bit ugly - I had to use the <scripts> and <script> tags in order to get Intellisense in pages to work because this is the only way I could figure out how to get Intellisense to work. Using custom prefixes (required for new controls) immediately breaks VS JavaScript Intellisense. The result is that I had to use HtmlControls for the script which means there's no Intellisense. Bummer, but the syntax is easy enough to look up.
This is a relatively new control for me and so far I've only used it with one project, but so far this looks promising. But of course this too is a non-standard way of working - it requires using this control instead of script embedding into headers, but at least the syntax of this control is pretty much based on standard Script tags. In fact, for basic scripts the syntax is identical (ie. like the draggable.js) above. If this sounds interesting to you let me know in comments and I can post more info on this control.
But this control is also not a generic solution. It works well for my internal purposes as well as for a few customers I'm working with but it's not totally satisfying either because of the inconsistencies in Visual Studio. And forcing developers to use a custom control is not my idea of a good design either, but for me at least this control addresses a few key scenarios like script compression and programmatic control over scripts and resources.
No Clear Winning Solution
So as you can see I have no clear answers that satisfy even my own requirements entirely. For the most part I'm still embedding scripts straight into the HTML header and hope for the best. The biggest issues I've run into have been conflicts between some of my controls that rely on script resources and scripts loaded through the header with duplicate loads. Most of this is mitigated by my controls which can disable script loading and defer to manually loaded scripts when needed (ie. all script resources are properties and can be loaded either from resources, a url or not all injected by the control). Tracking these things down though can be time consuming. I'm not there yet but I think using hte above control is likely to fix that problem, but for generic controls requiring this control container to be used is probably a hard sell.
Even the wwScriptContainer control doesn't address the versioning and update scenarios although it does help with managing resource and file based script injection both for markup and controls (at least for my own controls). So I'm still searching for sanity and some input on what works for others who are heavily using scripts.
So, dear reader, what are your thoughts on this issue? Are you also fighting with script management hell? Do you use scripts from raw files, or do you use resources? What about for controls that use these scripts? And how do you manage these scripts if they get used across solutions? There are lots of options available but which ones make the most sense to you and why?
I'd love to hear your thoughts.
Other Posts you might also like