Rick Strahl's Web Log

Wind, waves, code and everything in between...
ASP.NET • C# • HTML5 • JavaScript • AngularJs
Contact   •   Articles   •   Products   •   Support   •   Search
Ad-free experience sponsored by:
ASPOSE - the market leader of .NET and Java APIs for file formats – natively work with DOCX, XLSX, PPT, PDF, images and more

Variable Scoping in Anonymous Delegates in C#


Every once in a while when I write code that deals with Anonymous Delegates, I still kinda freak out when it comes to the behavior of closures and the variable scoping that goes along with it. For example, the following code uses a List<T>.Find() Predicate to search for items in the list and requires that a dynamic value is compared to:

public void AddScript(ScriptItem script)
{
    ScriptItem match = null;
 
    // *** Grab just the path
    if (!string.IsNullOrEmpty(script.Src))
    {
        script.FileId = Path.GetFileName(script.Src).ToLower();
 
        match = this.InternalScripts.Find(
            delegate(ScriptItem item)
            {
               ScriptRenderModes mode = this.RenderMode;  // demonstrate this pointer
               return (item.FileId == script.FileId);
            });
    }
 
    if (match == null)                
        this.InternalScripts.Add(script);
    else
        match = script;
} 

The Predicate<T> signature requires a boolean return value and an input parameter of any type (the <T>). The input parameter will be the item iterated over and is of the same time as the list. But the question then becomes: How do you compare that object against a dynamic value?

The freaky thing is the script variable in the AddScript function. It's passed in as a parameter to the outer function so it's effectively a local variable, but yet it is fully visible in the anonymous delegate's own private body. The delegate is in effect just another function, but what's different here is that the anonymous delegate inherits the variable scope of the containing method that is calling it.

To contrast if I were to call an explicit method to handle the Find filter:

match = this.InternalScripts.Find(Exists(script2));

and then create a method that matches this signature I can't access the script variable so this fails to compile:

bool Exists(ScriptItem script2)
{
    return (script2.FileId == script.FileId);   // FAILS obviously due to scoping
}

because script isn't available.

Behind the scenes the C# compiler creates a dynamic delegate class that includes fields for each of the local variables that need to be in scope, so when script  is referenced in the delegate's body it's really referencing the equivalent of this.script. The compiler fixes up the code so that the script property is set when before the method call is made so the delegate code 'appears' to be referencing a variable. If the this pointer is referenced in the delegate code, the compiler also creates a field that holds a reference to the calling class and then fixes up any calls to this to this member instead.

Here's what the compiler generated class looks like:

[CompilerGenerated]
private sealed class <>c__DisplayClass2
{
    // Fields
    public wwScriptContainer <>4__this;
    public ScriptItem script;
 
    // Methods
    public <>c__DisplayClass2();
    public bool <AddScript>b__0(ScriptItem item);
}

and the actual delegate method on the class:

public bool <AddScript>b__0(ScriptItem item)
{
    ScriptRenderModes mode = this.<>4__this.RenderMode;  
    return (this.script.Src == item.Src);
}

Notice that the this pointer is actually referencing a property that has the parent class assigned to it. It's all compiler sugar as well as Visual Studio's Intellisense that gives you the impression that you are in fact inside of closure method. Incidentally one thing that DOESN'T work is checking the this. pointer in the debugger while you are in the delegate code. This cannot be accessed in the debugger inside of the delegate code which would presumably cause some confusion of whether this should point at the delegate (and reveal the compiler magic) or the calling container's class.

Regardless of sleight of hand, the concept of closures is very cool and highly useful. I use it all the time in JavaScript, but damn, it still feels unnatural to me in C#. There's some debate over whether C# anonymous delegates are true lexical closures (take a look at this post and especially the comments), but for most situations that require variable scoping to be passed down to delegates for all intents and purpose the behavior is like that of a typical closure that inherits the calling context's variable scope.

This concept of anonymous delegates and closure scoping is EXACTLY what makes Lambda Expressions work in C# 3.0. Lambdas are just glorified Anonymous Delegates. In fact, the Find() predicate above can also be expressed as a Lambda expression:

match = this.InternalScripts.Find( item => item.FileId == script.FileId );

It's funny sometimes how you end up using some functionality without really understanding what goes on underneath the covers and one day you're using something completely unrelated (in this case JavaScript closures) that make everything click and fall into place. A little spelunking with Reflector later and you feel a heck of a lot more comfortable with several language features...

Posted in .NET  CSharp  

The Voices of Reason


 

Kevin Pirkl
April 27, 2008

# re: Variable Scoping in Anonymous Delegates in C#

Closures are a damn difficult thing to explain in text and an almost mythical a byatch to feel safe about when coding them in .Net. I like seeing your usage and explanation as I was plodding around trying to understand and do the same thing with .Net 3.5 and grok it entirely. JavaScript junkies like John Resig have long been touting Closures (he has a whole chapter dedicated to the concept in his next JavaScript book, not completed) but in the mean time this Morris Johns "JavaScript Closures for Dummies" might help to make it more clear. http://blog.morrisjohns.com/javascript_closures_for_dummies.html

Thanks for the sample Rick... Tre-Cool!

Edward J. Stembler
April 27, 2008

# re: Variable Scoping in Anonymous Delegates in C#

What "freaks" me out too is the fact that the anonymous type declared in my Lambda expression (which I thought was temporary) is actually still in scope after the Lambda call within my method.

IWorkbenchDescriptor workbenchDescriptor = workbenchDescriptors.FirstOrDefault(workbench => workbench.Name == workbenchName);

IWorkbench workbench = Activator.CreateInstance(workbenchType) as IWorkbench; // Compile-time error


The compiler reports the following error:

A local variable named 'workbench' cannot be declared in this scope because it would give a different meaning to 'workbench', which is already used in a 'child' scope to denote something else.

Rick Strahl
April 27, 2008

# re: Variable Scoping in Anonymous Delegates in C#

@Kevin - thanks. Looking forward to John's book actually. I've been looking for a more advanced general JavaScript book and the market on that end has been sparse.

@Edward - Good catch - I didn't notice this. That indeed is odd behavior especially given the way that the actual delegate is created as a separate object. The same thing applies in the example I posted above with an explicit anonymous delegate (which is after all what Lambdas are converted to):

                match = this.InternalScripts.Find(
                    delegate(ScriptItem item)
                    {
                        ScriptRenderModes mode = this.RenderMode;
                        return script.Src == item.Src;
                    });

                ScriptRenderModes mode = ScriptRenderModes.Script; // fails at compile time


this fails with the same compiler error. I suspect that's a compiler bug - that code should actually be compilable.

Vladimir Kelman
April 29, 2008

# re: Variable Scoping in Anonymous Delegates in C#

Edward and Rick, it sounds like a bug in C#. Isn't it supposed to support variable block scope, so that a variable local to anonymous delegate (or lambda) shouldn't affect anything outside the block where it is defined?
Designers even added block scope to JavaScript 1.7: http://developer.mozilla.org/en/docs/New_in_JavaScript_1.7#Block_scope_with_let.

What is a new book by John Resig? A new edition of "Pro JavaScript Techniques"? Is there any info about it on the Web?
There is one more coming JavaScript book to watch for: "JavaScript: The Good Parts" by Douglas Crockford http://developer.mozilla.org/en/docs/New_in_JavaScript_1.7#Block_scope_with_let

Vladimir Kelman
May 01, 2008

# Lambdas and Anonymous Delegates in C#: a block scope!

I was completely wrong! That happens when studying too many languages at the same time. JavaScript and F# allow to hide outer variable by declaring another variable with the same name in a nested scope. C# does not allow to do that!

"odd behavior especially given the way that the actual delegate is created as a separate object"
It actually shows that lexically lambdas and anonymous methods are treated as inline code placed into a nested block scope. It is consistent with the fact that they can access local variables from outer scope.
Behind the scenes compiler activity of creating actual delegate class and closure behavior is just behind the scenes and shouldn't affect lexical rules.

  /// <summary>
  /// C# 3.0 in a Nutshell, http://www.amazon.com/3-0-Nutshell-Desktop-Reference-OReilly/dp/0596527578/
  /// Page 46, "The scope of local or constant variable extends to the end of the current block.
  /// You cannot declare another local variable with the same name in the current block
  /// or in any nested blocks."
  /// That's an opposite to JavaScript and F# scoping rules.
  /// </summary>
  class Program {
    delegate int Adder();

    static void Main(string[] args) {
      int x;
      {
        int y;
        int z;
        int x;  // error, x already defined in outer scope (1*)
      }
      int y;  // error, y already defined in a child scope (2*)
      {
        int z;  // ok, no z in outer block
      }
      Console.WriteLine(z); // error, z is out of scope

      int t, u;
      Adder goodAdder = () => { return t++; };  //ok
      Adder badAdder = () => { int u; return t + u; };  // error, u already defined in outer scope
      Adder badToo = delegate { int u; return t + u; }; // the same error
    }
  }
}
// 1*: Compiler errors are as followed
// 1    A local variable named 'x' cannot be declared in this scope because it would give
// a different meaning to 'x', which is already used in a 'parent or current' scope to denote something else
//
// 2*: Compiler error is as followed
// A local variable named 'y' cannot be declared in this scope because it would give
// a different meaning to 'y', which is already used in a 'child' scope to denote something else

Martin
November 28, 2008

# re: Variable Scoping in Anonymous Delegates in C#

I've just come across a subtlety in this area, related to loops:
Basically my code was along the lines of:
private void DataBindToGroups()
{
    foreach (ClassA a in this.ListOfA)
    {
        ClassB b = new ClassB();
        b.Event += delegate(object sender, EventArgs e)
        {
            SomeAction(a);
        };
        this.ListOfB.Add(b);            
    }
}

However, when the Event was raised on _any_ of the ClassBs, the resulted was execution of SomeAction on the _last_ of the ClassA in my list, rather than the one which was in scope when the ClassB's event was attached to.

I found that I could get the correct behaviour simply by moving the delegate into a separate method, taking the required ClassA as a parameter:
private void DataBindToGroups()
{
    foreach (ClassA a in this.ListOfA)
    {
        ClassB b = new ClassB();
        SetEventHandler(b, a);
        this.ListOfB.Add(b);            
    }
}

private void SetEventHandler(ClassB b, ClassA a)
{
    b.Event += delegate(object sender, EventArgs e)
    {
        SomeAction(a);
    };
}

Nick
December 12, 2008

# re: Variable Scoping in Anonymous Delegates in C#

This shed's a great deal of light on the subject. I came across the concept of closure scope accidentally when looking at the parallel extensions for .NET. These will be available in C# 4.0 but they've released the extensions that allow you to use them in VS 2008.

It basically adds up to a for loop where each pass through the iteration is run on a separate thread. I didn't want to install the extensions and I thought "I should be able to do this with basic .NET 2.0 code", and I was, with an anonymous method:

void MultiThreadedForLoop(int start, int end, MultiThreadedForLoopDelegate d)
        {
            int threadCount = end - start;
            Thread[] t = new Thread[threadCount];
            for (int i = start; i < end; i++)
            {
                t[i - start] = new Thread(new ParameterizedThreadStart(d));
                t[i - start].Start(i);
            }

            for (int i = start; i < end; i++)
            {
                t[i - start].Join();
            }
        }


and it's called like this:

void MTMatrixMultiplication(double[,] a, double[,] b, double[,] c)
        {
            int s = a.GetLength(0);

            MultiThreadedForLoop(0, s, new MultiThreadedForLoopDelegate(delegate(object o)
                                                 {
                                                     int i = (int) o;
                                                     for (int j = 0; j < s; j++)
                                                     {
                                                         double v = 0;

                                                         for (int k = 0; k < s; k++)
                                                         {
                                                             v += a[i, k] * b[k, j];
                                                         }

                                                         c[i, j] = v;
                                                     }
                                                 }
                         ));
        }


but after I got it working, I was a little freaked out that I was able to access my outer function parameters inside my delegate function, which, if it were a separate function, couldn't possibly access these things.

I'd be curious to see how the C# 4.0 implements these things behind these scenes, and if they allow you to use an actual delegate/function.

Thanks for the explanation though, I'm not sure I totally understand it now, but I feel better about using it :)

Andy
November 06, 2009

# re: Variable Scoping in Anonymous Delegates in C#

Hey, I know this is old, but what did you use for your disassembler? I tried the red gate one, and that just turns lambda expressions into anonymous delegates, and the Microsoft one (ilsasm) seems to go a bit too far and is starting to look like assembly language! :s (Although in the tree view you can see the generated <>c__DisplayClass.

Otherwise, excellent post!!

Ta,

Andy

Andy
November 06, 2009

# re: Variable Scoping in Anonymous Delegates in C#

Never mind, I wasn't setting the decompile in the redgate software to IL! :)

See this blog: http://aspnet.4guysfromrolla.com/articles/080404-1.aspx

Sky
December 16, 2009

# re: Variable Scoping in Anonymous Delegates in C#

@martin r.e. non-intuitive behaviour when using an enumerator within a closure:

First of all, I know your post is a year old, but this can be an extremely frustrating 'feature' if not understood.

I first encountered this behaviour in JavaScript. I chased my own tail for quite some time nailing this down. I tried using an indexer instead of an enumerator with no joy.

The simple solution, in both javascript and C#, is to dereference the enumerator's current, in your case 'a', simply assigning a local variable in the closure....

    .....
    foreach (ClassA a in this.ListOfA)
    {
        var localA = a;
        .....
        SomeAction(localA);
        .....
    }
    .....


To ensure I recalled properly I wrote a proof and will probably make a blog post of it.

And, as you have found, escaping from the scope of the closure with a function call forces a dereference to the same end.

It seems that the actual assignments are deferred until completion of the closure and at that time the current of the enumerator (last) is in scope and is the target for all assignments.

p.s. A hint can be taken from the VS warning about "Access to modified closure" in these scenarios.

Hope this helps someone.

Sky

Sky
December 16, 2009

# re: Variable Scoping in Anonymous Delegates in C#

    /// <summary>
    /// A closure scoping study inspired by martins 
    /// comment on http://www.west-wind.com/weblog/ShowPost.aspx?id=330694
    /// 
    /// while fact checking my suspected explanation i found this, 
    /// http://stackoverflow.com/questions/512166/c-the-foreach-identifier-and-closures,
    /// with much better examples and explanations.
    /// 
    /// </summary>
    public class ClosureScopeStudy
    {
        public void AccessingAModifiedClosure_OR_WhatNotToDoAndHowToFixIt()
        {
            var listA = new List<A> {new A(), new A(), new A(), new A()};
            var listB = new List<B>();
 
            foreach (A a in listA)
            {
                // on each iteration 'a', which has been declared in 
                // the enclosing scope of the foreach,
                // is being reassigned to enumerator.next
 
                var b = new B();
                b.Event += delegate { SomeAction(a); };
 
                // Access to modified closure: ^^^
                // a reference to 'a' is captured by this
                // delegates closure.
                // when the closure loses scope the
                // reference has been set to the enumerator.last
 
                listB.Add(b);
            }
 
            listB.ForEach(Out);
            Console.WriteLine("Not what you expected, eh?\n");
 
            // try it again with a very subtle yet simple fix
 
            listA = new List<A> {new A(), new A(), new A(), new A()};
            listB = new List<B>();
 
            foreach (A a in listA)
            {
                // dereference the enumerator into the scope
                // of this block
                A localA = a;
 
                var b = new B();
                b.Event += delegate { SomeAction(localA); };
 
                listB.Add(b);
            }
 
            // 'a' is still being reassigned on each iteration, but 'localA'
            // is only ever assigned once and results in expected behavior.
 
            listB.ForEach(Out);
            Console.WriteLine("What you expected.\n");
 
 
            // same thing using a lambda
            listA = new List<A> {new A(), new A(), new A(), new A()};
            listB = new List<B>();
 
            foreach (A a in listA)
            {
                A localA = a;
 
                var b = new B();
                b.Event += ((sender, e) => SomeAction(localA));
                listB.Add(b);
            }
 
 
            listB.ForEach(Out);
            Console.WriteLine("Using a lambda.\n");
            Console.WriteLine("\nFinished. Press Enter");
            Console.ReadLine();
        }
 
        private static void Out(B b)
        {
            Console.Write("b.ID={0}, ", b.Id);
            b.OnEvent(EventArgs.Empty);
        }
 
        private static void SomeAction(A a)
        {
            Console.WriteLine("a.ID={0}", a.Id);
        }
 
        #region Nested type: A
 
        public class A
        {
            private static int _id;
 
            public int Id;
 
            public A()
            {
                Id = _id++;
            }
        }
 
        #endregion
 
        #region Nested type: B
 
        public class B
        {
            private static int _id;
 
            public int Id;
 
            public B()
            {
                Id = _id++;
            }
 
            public event EventHandler Event;
 
            internal void OnEvent(EventArgs e)
            {
                EventHandler handler = Event;
                if (handler != null) handler(this, e);
            }
        }
 
        #endregion
    }

Ashiq
November 19, 2010

# re: Variable Scoping in Anonymous Delegates in C#

This is what's so awesome about C#. It just does stuff the way you expect, and gets out of the way. You can even create local variables, create an in-line anonymous delegate, pass the local variable, and when the delegate is invoked, it "knows" about that -- as you've shown by decompiling, it actually stores the variables it needs. Smart!

ConversionMan
December 29, 2010

# re: Variable Scoping in Anonymous Delegates in C#

The anonymous delegate sure messed me up more than once, especially the variable scope inside loops. I think I finally got the concept of closure after working more with javascript. In Chrome, the developer toolbar shows "closures" in addition to local/global, which is a nice way of learning about scoping rules in closures.
 

West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2017