Rick Strahl's Weblog  

Wind, waves, code and everything in between...
.NET • C# • Markdown • WPF • All Things Web
Contact   •   Articles   •   Products   •   Support   •   Advertise
Sponsored by:
West Wind WebSurge - Rest Client and Http Load Testing for Windows

Thread Aborted exception with a Thread Timer fired in ASP.NET


:P
On this page:

So one of the things I’m doing in the Weblog is to handle backlink tracking in the background. There’s a scheduler that pushes all referred links that qualify into a database queue. I then have a routine off a timer that fires on a configurable interval to go and check the referred links. The routine works fine, but the I’m having some big issues with the timer.

 

What appears to be happening is that the timer fires starts processing but in the middle of processing somewhere the Thread is getting aborted I get a

 

Thread was being aborted 
 

What’s interesting is that this only happens on the live server – in my development situation this code runs fine, but of course the live server is pretty busy and there’s likely lots of other stuff going on in the same application unlike here were I’m not running under any load. I ran a quick load test here and let the queue fire and it also worked fine here.

 

What I’m doing is that I have an App class with a static property for my scheduler hanging off of it:

 

public class App

{

    public static busScheduler Scheduler

    {

        get { return _Scheduler; }

        set { _Scheduler = value; }

    }

    private static busScheduler _Scheduler = null;

 

    static App()

    {

        /// *** Load the properties from the Config file

        Configuration = new WebLogConfiguration();

        WWLOG_APPNAME = Configuration.WebLogTitle;

 

        // *** Start the scheduler that handles mail and trackback lookups

        Scheduler = new busScheduler();

        Scheduler.QueueProcess(30);

    }

}

 

The static constructor creates an instance of the scheduler and immediately schedules queue processing 30 seconds later. The queing process basically creates a new timer and sets it up to fire in x number of seconds:

 

/// <summary>

/// Queues up the next check in the specified RefreshFrequency in seconds       

/// </summary>

/// <param name="Seconds"></param>

public void QueueProcess(int Seconds)

{

    if (App.Configuration.LogErrors == ErrorLogTypes.SqlServer)

        WebRequestLog.LogCustomMessage(App.Configuration.ConnectionString, WebRequestLogMessageTypes.ApplicationMessage, "QueueProcess called");

 

    this.RefreshFrequency = Seconds;

    Timer t = new Timer( this.QueueProcess_Start, null, Seconds * 1000,Timeout.Infinite);

}

private void QueueProcess_Start(object state)

{

    if (App.Configuration.LogErrors == ErrorLogTypes.SqlServer)

        WebRequestLog.LogCustomMessage(App.Configuration.ConnectionString, WebRequestLogMessageTypes.ApplicationMessage, "Queue Process Start called");

 

    this.Process();

}

 

(note the timer is a System.Threading.Timer the WinForms Timer)

 

There’s a Process method that gets called which actually does get fired once and it then runs through the queue picking out requests and proceeds to process them. The code is nothing fancy but it does use the WebRequest object to read the target content. The generic processing code looks something like this:

 

/// <summary>

/// Processes all pending requests

/// </summary>

public bool Process()

{

    this.SetError();

 

    if (App.Configuration.LogErrors == ErrorLogTypes.SqlServer)

        WebRequestLog.LogCustomMessage(App.Configuration.ConnectionString, WebRequestLogMessageTypes.ApplicationMessage, "Scheduler Process called: Start");

 

    // *** Create a new instance each time so there's

    // *** not a threading problem

    busScheduler Sched = new busScheduler();

 

    if (Sched.GetPendingItems(null, "TItems") < 0)

    {

        // *** Queue up the next check

        if (this.RefreshFrequency != 0)

            this.QueueProcess(this.RefreshFrequency);

 

        if (App.Configuration.LogErrors == ErrorLogTypes.SqlServer)

            WebRequestLog.LogCustomMessage(App.Configuration.ConnectionString, WebRequestLogMessageTypes.ApplicationMessage, "Scheduler Process called: Complte - nothing to do.");

 

        return false;

    }

    

    foreach (DataRow Row in Sched.DataSet.Tables["TItems"].Rows)

    {

        Sched.Entity.SetDataRow(Row);

 

        try

        {

            if (Sched.Entity.Type == "TRACKBACK")

                this.HandleTrackback(Sched);

       

        }

        catch (Exception ex)

        {

            if (App.Configuration.LogErrors == ErrorLogTypes.SqlServer)

                WebRequestLog.LogCustomMessage(App.Configuration.ConnectionString, WebRequestLogMessageTypes.Error,

                                               "Failed to handle scheduling of type: " + Sched.Entity.Type + ".\r\n" + ex.Message);

        }

 

        // *** Clear out the scheduled item

        Sched.Delete(Sched.Entity.pk);

 

        // *** Make sure we give up SOME time slice

        System.Threading.Thread.Sleep(1);

    }

 

    // *** Queue up the next check

    if (this.RefreshFrequency != 0)

        this.QueueProcess(this.RefreshFrequency);

 

    if (App.Configuration.LogErrors == ErrorLogTypes.SqlServer)

        WebRequestLog.LogCustomMessage(App.Configuration.ConnectionString, WebRequestLogMessageTypes.ApplicationMessage, "Scheduler Process called: Complete");

 

    if (this.ErrorMessage != "")

        return false;

 

    return true;

}

 

Right now there’s only a single handler in this code, but in the future I suppose there’ll be more than one. This code gets fired off the timer, but it catches an exception that the thread was aborted somewhere in the Trackback processing.

 

I’ve added some logging code into the page here as you can see that writes out to my error/application message log and so I can see what’s happening. Locally this works just fine, but on the server I see one of two things:

 

1.

The timer is set and then the timer callback function NEVER even gets called. No errors, no failure no logging – nada, but nothing happens obviously.There are no errors (which I trap) but the timer just never reaches the callback code in QueueProcess_Start().

 

2.

The queue starts up and the timer does fire once. But somewhere in the processing the timer thread just aborts and I get the aforementioned exception.

 

Now, I figured this would work just fine and the timer seems like a perfect setup for this because it can handle the checking for the next hit. Offhand I can’t see what would cause this sort of behavior. What would cause a thread to be aborted? The processing code just runs through the database and makes WebRequest calls. It’s not calling back into ASP.NET at all – it’s all business logic, so there’s no reliance even on the HttpContext or anything related to it.

 

Timer threads should run off the ASP.NET thread pool, but I doubt there’s any problem with load on this server. The server is busy, but not that busy to chew through a hundred threads. IAC, even if ASP.NET was that busy I don’t think it would just try to kill a thread like that? What would cause a thread to abort like this? But it sure looks like something is calling Thread.Abort() from outside.

 

Anybody see a reason why this shouldn’t work that I’m missing?

 

I suppose the alternative is to spin up raw new thread and just manage the timing myself putting the thread to sleep for the refresh cyle period, but it seems inefficient to tie up a thread in this way for what will be mostly idle time.

 

Posted in ASP.NET  Threading  

The Voices of Reason


 

Eric W. Bachtal
September 27, 2006

# re: Thread Aborted exception with a Thread Timer fired in ASP.NET

I believe the problem is that your timer is being garbage collected before it can go off. You need to keep it alive, either with a reference or a strategically placed GC.KeepAlive.

Wilco Bauwer
September 27, 2006

# re: Thread Aborted exception with a Thread Timer fired in ASP.NET

I suspect that if you create a thread yourself, you'll end up with a similar behavior. Although I'm not exactly sure why you get a TAE so early, I would probably try to use one of the mechanisms provided by ASP.NET to deal with long-running tasks (if you really want to run them inside your webapplication... that's a different discussion though). They should be save to use with relation to appdomain recycles, etc. One of the things you could try is to use the async task stuff on the Page object and see if that helps at all.

Rick Strahl
September 27, 2006

# re: Thread Aborted exception with a Thread Timer fired in ASP.NET

Eric, the scheduler and the code it fires is a static property on an object so it should never go out of scope and not be garbage collected. App.Scheduler will always be there after the static constructor has fired. I don't think it's a scope issue...

Wilco, the problem is that this is a task that is not essentially related to the page or even a handler. It's a timed task that just runs and checks the queue occasionally. I remember Rob Howard mentioning this as a perf trick some time ago and I finally got a chance to try it out but no luck <s>.

I'm not terribly worried about an AppDomain recylce - if that happens the thread would indeed abort, but there's nothing lost if it doesn't as the scheduled task won't delete until it's complete and it'll just re-run on the next pass. I'm also logggin Application_Start/End and the Application isn't shutting down - the thread shutdown is independent of that.

I'm going to try a separate thread - it's possible that ASP.NET is doing some monitoring on threadpool threads that the Timer uses - the tasks that would run on this thread would be lengthy and might well exceed the ASP.NET request timeout.

Eric W. Bachtal
September 27, 2006

# re: Thread Aborted exception with a Thread Timer fired in ASP.NET

The scheduler and fired code lifetimes aren't the problem, the timer is. The timer holds a reference to the TimerCallback delegate which holds a reference to the Scheduler, and a reference to the timer is passed to some unmanaged kernel code to alert the timer when the time expires. The unmanaged reference to the timer won't keep the timer from being garbage collected and you aren't holding a reference to it anywhere either (it's scoped to the QueueProcess routine), so it's getting GC'd. When the unmanaged code tries to call the timer back, it's gone. The unhandled exception on the thread throws a thread abort. From the help:

"As long as you are using a Timer, you must keep a reference to it. As with any managed object, a Timer is subject to garbage collection when there are no references to it. The fact that a Timer is still active does not prevent it from being collected."

My recommendation is to make Timer t a member variable of busScheduler and see what happens.

Rick Strahl
September 27, 2006

# re: Thread Aborted exception with a Thread Timer fired in ASP.NET

Oh duh!!!! That makes perfect sense Eric... Didn't really think about the timer object itself being vital since I figured its offloading the timer management to Windows...

Can't check that now but I suspect that would have fixed the issue - I switched the code over to use a separate thread that just goes to sleep between intervals and that works just fine now. In the end this code is cleaner anyway...

Louis
November 20, 2006

# re: Thread Aborted exception with a Thread Timer fired in ASP.NET

3 days sitting on this problem, Never thought to put the Timer as a member variable of the class. Works exactly i wanted to now.

West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2024