FireFox 3.0 apparently is much more aggressive in caching content than previous versions of FireFox. While this is good for performance in many situations I've noticed that this is seriously becoming a problem in some of my applications where support files like scripts and CSS files get updated and FireFox doesn't recognize those changes. The problem appears to be that FireFox is not sending the required 304 request to check for last-modified headers that are typically used for static files.

The latest example is my (ancient) Message Board site which I recently updated by reducing the HTML and externalizing most of the inline styling into an external CSS file that previously existed. Today I'm waking up to 10 messages from visitors that the the formatting of the message board is completely mangled. Apparently the CSS file changes are not being picked up by FireFox 3.0 and so users see the new dynamic message board pages, rendered against the old CSS styles cached in the browser. Thankfully I did this over the weekend and so the cacophony of disgruntled visitors was only minor.

The updated CSS file is there on the server. IE, Opera and Safari and even FF 2 all work fine - only FF3 fails to refresh the CSS for clients because apparently it's not detecting the last-modified date for the static resource. Even pressing Refresh doesn't fix this and even FireFox's 'hard' Shift-Refresh doesn't do the trick if the CSS is already cached.

Part of the problem here is that the app is frames based. Shift-Refresh should in theory force a refresh of all resources (and it does usually), but in the case of this frames based layout it doesn't. The only way to force the CSS to refresh that I found is to Ctrl-Click one of the links and force into a new frame, then force a Shift-Refresh in the new window. Intuitive, eh? Oh, or you can clear the browser cache! Yeah - that's what I tell 'em! <g>

But it's not only this CSS that suffers. I've also had issues with AJAX related static JavaScript files served from the server that also didn't work with the same caching issues. This is even more insidious because you can't see the problem except that potentially your app will fail now because the application's client application data is not up to date.

Again only a Shift-Refresh usually solves this problem. While this is fairly painless to do in the dev environment (it's become a habit already to Ctrl-Click) this is a big problem in production environments since you certainly don't want to tell your visitors to 'just Ctl-Click and things will be cool'.

It appears that the problem is vaguely related to this post:

With Firefox 1 and 2, “Cache-control: no-cache” accomplished this goal. I’ve spend the last several hours trying to figure out how to prevent Firefox 3 from caching my pages, with no luck. It completely ignores “Cache-control: no-store, no-cache, must-revalidate” and every combination of values for the “Expired:” header that I can think of, in blatant violation of RFC 2616 (14.9 and 14.21).

This post also mentions how you can modify your cache settings. While this works this is hardly a solution for end users.

Last-Modified Headers not working properly in FF3

The issue described in the post is in regards to caching, which is a little different actually than what I'm experiencing here, which has to do with the last-modified header that static files use. Web Servers serve static files (like CSS and scripts) from disk and they tend to serve this data with a last-modified header which the browser is supposed to respect. When requesting data again for a page that when originally loaded the browser is supposed to fire a 304 request to see if the content has changed.

Here's what the 304 response looks like in Internet Explorer with Fiddler enabled. IE fires the hit to the CSS file every time as a 304 request which looks like this:

GET /wwThreads/wwthreads.css HTTP/1.1
Accept: */*
Referer: http://www.west-wind.com/wwThreads/head.htm
Accept-Language: en-us
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; FDM; .NET CLR 3.5.21022; .NET CLR 3.5.30729; .NET CLR 3.0.30618; .NET CLR 1.1.4322)
UA-CPU: x86
Accept-Encoding: gzip, deflate
If-Modified-Since: Sun, 31 Aug 2008 09:37:26 GMT
If-None-Match: "20d7b72d4dbc91:0"
Connection: Keep-Alive
Host: www.west-wind.com

HTTP/1.1 304 Not Modified
Last-Modified: Sun, 31 Aug 2008 09:37:26 GMT
ETag: "20d7b72d4dbc91:0"
Accept-Ranges: bytes
Server: Microsoft-IIS/7.0
X-Powered-By: ASP.NET
Date: Sun, 31 Aug 2008 20:04:11 GMT

If the content were modified the server would respond with a full 200 HTTP response. So if I go in and modify my CSS file and hit the page again, IE will properly detect the change.

FireFox however, doesn't appear to fire the 304 at all. I've hooked up Firebug and the CSS files don't show up at all. Not trusting the FireBug network tracing I also hooked up FireFox to Fiddler and lo and behold it too shows that FireFox is not firing the 304's to check for updated files. When loading pages - even with Ctl-Clicking - I don't see the CSS files loaded which explains why the pages render all mangled. In my frames page even browser Refresh or even a Shift-Refresh doesn't cause the frames to reload their content or fire 304s to check for updates - all the static content is simply not accessed at all.

For example, if I start up FireFox fresh and navigate to my site and hit one link here's what Fiddler shows:

Notice there are no other requests to any support resources - images, CSS etc. No checks for 304's. Here's that same trace with Internet Explorer:

:

and you'll notice all the 304 hits to verify resources. Now using Shift-Refresh in Firefox will force FireFox to refresh all those links explicitly and load them, but this isn't the expected behavior. The whole point of last-modified and 304s is to be able to detect changes to static resources and apparently FireFox doesn't play along here.

Frames

Ok, frames are so passé but as I mentioned the app I'm actually dealing with uses frames and the problem there is that a Shift-Refresh on the parent frame isn't enough to force the individual frames to refresh. In fact, I'm not really sure how to force FireFox to refresh an individual frame. The only way I've been able to do this is to load one of the frames by shift clicking into a separate window and then doing a shift-Refresh to force the stylesheet to reload. After that all's well because that updates the cache globally.

Refresh should be Refresh

What's worse though is that in FireFox 3's default configuration even clicking the Refresh button will not force 304s. Apparently the Refresh button behavior has been changed to only re-run the last request EXACTLY in the same mode it was run. So if you did a Shift-Refresh on the last hit, it will repeat that behavior. If you did a plain link click that will replay exactly and so on. There's no longer an explicit reloading of resources which is now delegated to Shift-Refresh and which no 'typical' user will think of.

The FireFox Refresh button's Refresh behavior is pretty lame. Having an easy way to force pages and all related content to reload is key in a browser environment where the underlying resources may frequently change on the server. Between the refresh and last-modified issues described FireFox 3.x is becoming a bit shaky for the stability, especially for applications that are rich in client side styling and script behavior. It's not uncommon for my apps to update CSS or script content several times a day and there's now a good possibility that these changes may not match changes in the underlying application that serves non-static content. This has bit me quite frequently now during development when refreshing pages and expecting to see update content only to find that things like Script files have not refreshed and a few times in production with customers complaining.

One temporary solution might be (thanks to Steve Black for the hint) to force the CSS references in pages to refresh by changing the URL slightly. Like

<link  href="wwThreads.css?i=1" rel="stylesheet" type="text/css"/>

The additional querystring will force even FireFox to refresh the data. But that's one heck of a hack. But I may have to resort to this because nothing else works reliably at this point.

So... has anybody else run into this in their applications? In searching around the Web it doesn't appear that there's much talk about this issue - I didn't find anything related to last-modified and a few discussions - which are related - to caching and back button and refresh behavior. It looks like Firefox is trying to be too agressive in caching. Sure, caching is cool for performance, but not if it's at the cost of stale content.