Contact   •   Products   •   Search

Rick Strahl's Web Log

Wind, waves, code and everything in between...
ASP.NET • C# • HTML5 • JavaScript • AngularJs

Fun with List Types for Caching Data


I often find myself using various collection types for 'cached' storage in Web applications. For example, in some applications there are lists of countries, lists of states, various other lookup lists that are initially stored in a database, but then are constantly reused. There's little point in continually reloading this data from the database so caching it as part of the application is useful.

Depending on how you like to work data is usually pulled from the database with a DataReader,  which can't be cached or a DataTable which can. DataTables are Ok, but they have a lot of bulk associated with them in addition to the data, which is something I usually want to avoid when I'm caching data.

So more commonly for cached data I use one of the collection types to hold the data read from a DataReader. This is something I don't do very frequently though and it always takes a small reminder to remember how you can databind this data using 'generic' names rather than specific field names as you might be used to with a DataReader or DataTable.

So let's take a typical situation of retrieving a state table. When I retrieve the table it's typically done once for the application. The application runs a SQL Query to pull the statelist from the database, then uses a DataReader and fills a Dictionary<string,string> with the values.

public static Dictionary<string,string> StateList

{
  get 
  {              
      if (_StateList == null)
          LoadStateList();              
 
      return _StateList; 
  }
  set { _StateList = value; }
}
private static Dictionary<string,string> _StateList = null;
 
 
public static void LoadStateList()
{
    if (_StateList != null)
        return;
 
    _StateList = new Dictionary<string, string>();
 
    busLookups Lookups = WebStoreFactory.GetbusLookups();            
    IDataReader Reader = Lookups.ExecuteReader("select cData1 as State, cData as StateCode from " + Lookups.Tablename + " WHERE type='STATE' ORDER BY 1");
 
    while (Reader.Read())
    {
        string StateCode = Reader.GetString(1);
        string StateName = Reader.GetString(0);
 
        _StateList.Add( StateCode,StateName );
    }
}

I prefer using static properties for the caching for permanent items rather than using say the Cache object, primarily because using a static property is more portable. You can use this code in any app and it's part of the business layer, so there's no specific dependency on ASP.NET. Also, for anything that's permanently cached there's no reason to go through a cache manager and static's work just fine for this.

So how do you databind a Dictionary? Rather than binding by explicit field names (ie. StateCode and State in this case) you have to bind by the collection type's field names, which in many of the common collection types is Key and Value:

protected void Page_Load(object sender, EventArgs e)

{
    //Hashtable sd = new Hashtable(); // StringDictionary sd = new StringDictionary();
    //StringDictionary sd = new StringDictionary();
 
    Dictionary<string, string> sd = new Dictionary<string, string>();
    sd.Add("HI", "Hawaii");
    sd.Add("OR", "Oregon");
 
    this.lstStates.DataSource = sd;
    this.lstStates.DataTextField = "Value";
    this.lstStates.DataValueField = "Key";
 
    this.lstStates.DataBind();
}
 

One of the interesting challenges is to determine what type of list collection to use. Do you use a Hashtable, a StringDictionary, a generic Dictionary... The choices are not always quite obvious as I found out today <s>. I was working on the code above to cache my statelist, but quickly realized that the code above actually fails because there are actually duplicate state codes for the various of the Armed Forces state keys.

Armed Forces Africa AE
Armed Forces Americas AA (except Canada)
Armed Forces Canada
AE Armed Forces Europe
AE Armed Forces Middle East
AE Armed Forces Pacific AP

Any dictionaries don't allow adding duplicate values so a dictionary is actually not the right choice unfortunately. Most of the dictionary types work in a similar fashion and use key value objects that you can bind to.

There's NameValueCollection() which does accept duplicate string keys, but NVC is a pain in the ass to use (iteration sucks) and I can't figure out a way to use it for databinding in the above manner (some searching also finds that there's apparently no solution).

But there are additional choices. It would seem that StringDictionary for example would be a natural choice for the above data right? After all I'm storing a key value pair which are both strings. Unfortunately StringDictionary changes the key value to lower case which precludes using StringDictionary in situations where character case has significance.

One nice thing about the way the databinding works in .NET is that you can easily use your own objects for databinding so if the stock collections don't work for you you can create your own class and do something like this: 

protected void Page_Load(object sender, EventArgs e)
    List<KeyValue> sd = new List<KeyValue>();
    sd.Add(new KeyValue("OR", "Oregon"));
    sd.Add(new KeyValue("HI", "Hawaii"));
 
    this.lstStates.DataSource = sd;
    this.lstStates.DataTextField = "Value";
    this.lstStates.DataValueField = "Key";
 
    this.lstStates.DataBind();
}
 
public class KeyValue
{
    public KeyValue(string Key, string Value)
    {
        this.Key = Key;
        this.Value = Value;
    }
 
    public string Key
    {
        get { return _Name; }
        set { _Name = value; }
    }
    private string _Name = "";
 
 
    public string Value
    {
        get { return _Value; }
        set { _Value = value; }
    }
    private string _Value = "";
 
}

Here a simple list collection is used and a custom object is configured. If all you do is databind the object then this appraoch works well, but you loose string index access as you would with a string dictionary or Dictionar<string,string>. So this sort of thing wouldn't work:

busLookups.StateList["HI"]; 

unless you implement the indexer yourself.

The custom class approach is especially nice if you need to cache more complex records that have more than two fields. You can also use the property names as field values for databinding in a Gridview or other list controls.

Lots of options for sure. And unfortunately I haven't found a good reference on when each of the various list/collection classes works best. The best reference I found is in this book:

Improving .NET Application Performance and Scalability (Patterns & Practices)
by Microsoft Corporation

Read more about this title...

But unfortunately it's a 1.1 specifc book, so it doesn't cover the generic collections. Incidentally that book has been quite useful for performance related issues, but it's badly in need of an update for .NET 2.0.

Make Donation


Feedback for this Post

 
# A Continuous Learner's Weblog: Links (3/21/2007)
by A Continuous Learner's Weblog March 22, 2007 @ 12:38am
# re: Fun with List Types for Caching Data
by kevin March 23, 2007 @ 6:30am
do you really place your data access code in the collection object?

I would think you'd separate that layer by abstracting it to something like a Utility.Controller.Country that handles the data access/cache access and then returns the object in all its glory.
# re: Fun with List Types for Caching Data
by Milan Negovan March 23, 2007 @ 10:47am
I hear you, Rick. I think collections in .NET are pretty messed up. The book you pointed out led me to express my opinion about collections here http://www.aspnetresources.com/blog/dotnet_collection_madness.aspx and http://www.aspnetresources.com/blog/dotnet_collection_madness2.aspx Hope you'll find my observations helpful.
# re: Fun with List Types for Caching Data
by Rick Strahl March 23, 2007 @ 12:17pm
Kevin, no not usually. This object actually sits on a business object which is filled by the actual business object. 'Normal' returns out of the business objects are either DataReaders or DataTables (typically), but you don't really see that here <s>...

The point of the collection here is the caching - you need to persist the data into something and using a collection is way more lightweight than say a DataTable to hold the data (which is another option). Since databinding works directly against the collection there's no reason to hold on to the overhead of a DataTable...
# re: Fun with List Types for Caching Data
by Rick Strahl March 23, 2007 @ 12:27pm
Milan great posts on Collections. Like you to this day, almost everytime I need a dictionary type Collection I take a wild guess at picking the best collection type (even with the 2.0 generic versions). Perhaps what bugs me even more than the lack of clear explanations is the fact the interfaces of these collection often varies so drastically. NameValueCollection is probably the best example of a truly hosed and unfriendly front interface.

As you know it gets even more confusing in 2.0 with the various generic collections thrown into the mix. Luckily it seems that the naming of these collections is a little cleaner (List, Dictionary) and from what cursory evidence I've seen is that these generic collections usually are more performant than the non-generic versions as they were re-written from scratch.

Still I was looking around quite a bit the other day trying to find a definitive comparison on the topic and came up blank. Oddly I didn't even find your post which isn't saying much for my search skills is it? <s>...
# A Continuous Learner's Weblog: March 2007
by A Continuous Learner's Weblog April 17, 2007 @ 10:22am
# DotNetSlackers: Fun with List Types for Caching Data
by DotNetSlackers Latest ASP.NET News April 19, 2007 @ 11:55am
# re: Fun with List Types for Caching Data
by Tawab May 08, 2009 @ 2:00pm
The List class actually works quite well for key/value collections containing duplicates where you would like to iterate over the collection. Example:

List<KeyValuePair<string, string>> list = new List<KeyValuePair<string, string>>();

// add some values to the collection here

for (int i = 0; i < list.Count; i++)
{
Print(list[i].Key, list[i].Value);
}
 


West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2014