Rick Strahl's Weblog  

Wind, waves, code and everything in between...
.NET • C# • Markdown • WPF • All Things Web
Contact   •   Articles   •   Products   •   Support   •   Advertise
Sponsored by:
West Wind WebSurge - Rest Client and Http Load Testing for Windows

Shutting up Invalid ViewState from SPAM bots


:P
On this page:

Somehow, sometime ago a spambot got into my West Wind Web Store and picked up a couple of pages and decided to just POST random (and randy <s>) SPAM to this URL to the tune of a few hundred POSTs a day by now. The funny thing about this is that the URL is just a product page that accepts only a single input quantity. For example:

http://www.west-wind.com/wwstore/item.aspx?sku=WWHELP40

There's absolutely no redeeming value to keep spamming this URL and posting data to it that goes absolutely nowhere.

Now the good news is that ASP.NET - because of Viewstate and Request Validation - will fail all of these requests and nothing is really lost other than the wasted bandwidth and processor time.

However, in all of my Web applications I have a global error handler hooked up to HttpApplication.Error that handles error logging, error notification and display of error messages in a centralized location. I have a fair amount of stuff in this app level handler code. Here's what it looks like to give you an idea:

       protected void HandleError()
        {

            Exception ex = Server.GetLastError();

            // *** Check for POST spam and write out IP blocking list            
            this.IsSpam(ex);

            WebErrorHandler Handler = null;

            // *** Try to log the inner Exception since that's what
            // *** contains the 'real' error.
            if (ex.InnerException != null)
                Handler = new WebErrorHandler(ex.InnerException);
            else
                // *** If no inner exception then we have access errors
                // *** or handler errors - log main exception
                Handler = new WebErrorHandler(ex);

            // *** Log the error if specified
            if (App.Configuration.LogErrors != ErrorLogTypes.None)
            {
                if (App.Configuration.LogErrors == ErrorLogTypes.XmlFile)  // XML Logging
                {
                    Handler.LogFileName = App.Configuration.XmlErrorLogFile;
                    Handler.LogErrorToXml(true);
                }
                else if (App.Configuration.LogErrors == ErrorLogTypes.SqlServer)  // Sql Server with WebRequestLog
                    Handler.LogErrorToSql(App.Configuration.ConnectionString);
            }

            // *** Retrieve the detailed String information of the Error
            string ErrorDetail = Handler.ToString();

            // *** Optionally email it to the Admin contacts set up in WebStoreConfig
            if (App.Configuration.SendAdminEmail)
                WebUtils.SendAdminEmail(App.Configuration.ApplicationName + "Error: " + Request.RawUrl, ErrorDetail);


            // *** Debug modes handle different error display mechanisms
            // *** Default  - default ASP.Net - depends on web.config settings
            // *** Developer  - display a generic application error message with no error info
            // *** User  - display a detailed error message with full error info independent of web.config setting
            if (App.Configuration.DebugMode == DebugModes.DeveloperErrorMessage)
            {
                Server.ClearError();
                MessageDisplay.DisplayMessage("Application Error", "<pre style='font: normal 8pt Arial'>" + ErrorDetail + "</pre>");
                return;
            }
            else if (App.Configuration.DebugMode == DebugModes.ApplicationErrorMessage)
            {
                string StockMessage =
                        "The Server Administrator has been notified and the error logged.<p>" +
                        "Please continue on by either clicking the back button or by returning to the home page.<p>" +
                        "<center><b><a href='" + App.Configuration.StoreBaseUrl + "'>West Wind Web Store Home Page</a></b></center>";

                // Handle some stock errors that may require special error pages
                HttpException HEx = ex as HttpException;
                if (HEx != null)
                {
                    int HttpCode = HEx.GetHttpCode();
                    Server.ClearError();

                    if (HttpCode == 404) // Page Not Found 
                    {
                        Response.StatusCode = 404;
                        MessageDisplay.DisplayMessage("Page not found",
                            "You've accessed an invalid page in this Web server. " +
                            StockMessage);
                        return;
                    }
                    if (HttpCode == 401) // Access Denied 
                    {
                        Response.StatusCode = 401;
                        MessageDisplay.DisplayMessage("Access Denied",
                            "You've accessed a resource that requires a valid login. " +
                            StockMessage);
                        return;
                    }
                }

                Server.ClearError();
                Response.StatusCode = 500;
                MessageDisplay.DisplayMessage("Application Error",
                    "We're sorry, but an unhandled error occurred on the server. " +
                    StockMessage);
                return;
            }

            return;
        }

This method is just called off the Application_Error event and it does basically all the work to log, notify and handle error display. Global error handling is really nice because with this code the entire application is covered and the code is nearly completely generic so it can be dropped into any app with the error handling component available. It's not totally generic though - there are a couple of app specific things - bascically the configuration settings (from App.Configuration) and a few custom things like checking for IsSpam to write out IP Addresses for later exclusion (more on that in a minute). Close enough though and it's usually one of the first things I do in an application. It's invaluable for getting useful information on your running application and I highly recommend some sort of error tracking with every Web app however trivial.

Error logging and notification is very useful but unfortunately, in the case of the SPAM bots firing a few hundred messages each day  it also causes several hundred error messages and more importantly several hundred email messages of crap to be sent out.

So although I'm loath to put error handling code into a specific page, I decided I had to do it to get control back of my error logs and email box <s>. As you probably know ASP.NET can also capture errors on a page/control level by overriding the OnError method.

protected override void OnError(EventArgs e)
{
    base.OnError(e);

    Exception ex = Server.GetLastError();
    if (ex.InnerException != null)
        ex = ex.InnerException;

    // *** Special handling to avoid SPAM messages
    if (ex is ViewStateException || ex is HttpRequestValidationException)
    {
        Response.StatusCode = 500;
        MessageDisplay.DisplayMessage("Application Error", 
                                      "Invalid input received for this request. Please retry it again with valid data.");
    }
}

It took me a few tries to get this right - I always forget that the 'real' exception is typically buried in the InnerException rather than the primary exception returned by GetLastError()  which only returns HttpException objects rather than say ViewStateException or HttpRequestValidationException.

Now I can get some piece of mind <s>...

As to the spam shit it still irks me to no end that these requests are being fired at my server consuming bandwidth and server resources. A while back I implemented some check code to  to capture IP addresses on these errors and started writing them out to a file:

        protected bool IsSpam(Exception ex)
        {

            bool blockIp = false;

            if (ex is HttpRequestValidationException || ex is ViewStateException)
            {
                if (Request.Path.ToLower().Contains("item.aspx"))
                    blockIp = true;
            }

            if (blockIp)
            {
                StreamWriter writer = new StreamWriter(Server.MapPath("~/admin/spamip.txt"), true);
                writer.WriteLine(Request.UserHostAddress);
                writer.Close();
            }

            return blockIp;
        }

In a matter of a couple of weeks I ended up with over 2000 Ip addresses which I went ahead and blocked. I even built a routine to automate this process through the Blocking IIS IP Addresses with ASP.NET that lets me add IP address. But they just keep coming even though pages are returning 500 errors. I also played with returning 404 errors for a while but none of that matters to the robots apparently. Once set nothing will stop this non-sense. Apparently IP blocking is also becoming useless with IP spoofing and more likely huge banks of IP addresses based in China serving just for that purpose.

Is there really any way to combat this? I suppose it's not worth it until it becomes a full scale DOS attack...

Oh well, one more thing we have to live with I guess, but man does that irk - all that wasted bandwidth and server resources and for what? It's not even exploiting anything useful... The world is full of M-O-R-O-N-S.

Posted in ASP.NET  

The Voices of Reason


 

Wilco Bauwer
July 31, 2007

# re: Shutting up Invalid ViewState from SPAM bots

Very annoying indeed. When I got fed up with it, I wrote a "ViewStateDosBanModule" (http://www.wilcob.com/Wilco/News/ViewStateDoSBanModule.aspx). "In a nutshell, the module keeps a list of requests which resulted in a ViewStateException. This list is stored in the cache with an absolute expiration time of 2 minutes. When a request is made by a client that already resulted in at least 5 ViewStateExceptions, the request is ended immediately."

Obviously this means a few of the exceptions will still get through, but at least I don't get batches of thousands of those exceptions anymore in the logs. I guess keeping a list of IPs would prevent that, but on the other hand, what if the IP is shared by many people (e.g. a large organization which had one exploited machine, or a university)?

McGurk
July 31, 2007

# re: Shutting up Invalid ViewState from SPAM bots

What's so bad about blocking huge swaths of Chinese IPs? Ever check to see how many sales came from those chunks? Its all cost/benefit. If you lose five sales' worth of income but regain a hundred sales' worth of bandwidth costs, then ban 'em.

Mads Kristensen
July 31, 2007

# re: Shutting up Invalid ViewState from SPAM bots

That's a clever approach Rick. In an HttpModule it would be very powerful and easy to distribute to all the the websites you have including this blog. It's definently something I will look at for BlogEngine.NET

GlenH
July 31, 2007

# re: Shutting up Invalid ViewState from SPAM bots

Maybe you should do a redirect to one of the other (random) banned IPs on your list. Maybe they can spam each other to death.

Might also consider just ending the response without returning anything to appear more like a black hole.

Rick Strahl
July 31, 2007

# re: Shutting up Invalid ViewState from SPAM bots

Well, looks like sanity has returned to my inbox this morning! No ViewState errors (no errors at all - yay!) Now I may actually note errors that occur for real and need attention <s>...

@Wilco - That's a great idea as well, although I don't think this would help much in this module particular scenario. These bots are smart enough to not send more than a 2 requests at a time and then wait for a half an hour or so for more. Further the IP Address vary on every request and they are widely ranged on top of it and don't seem to repeat often.

@McGurk - Blocking huge swaths - been thinking about it, but I do get a fair amount of legit traffic from Asia so it'd be tough. I haven't looked closely at the list of IPs captured but they are ALL OVER THE MAP. I don't think they are geographically isolated (ie. probably spoofed although I'm not sure how that's done).

@Mads - the problem for something generic is coming up with the right rules. I think what Wilco mentions is as close as it gets. In my case it's a very specific page that's easy to trap. But in other situations it may be much more difficult as ViewState errors may actually be 'valid' (ie. timed out pages etc) and you wouldn't just want to kill it. There are no easy answers I think...

Even a Module still hits ASP.NET too, so a lot of the overhead is already there although it certainly beats running all the way into a page <s>.

Peter Bromberg
July 31, 2007

# re: Shutting up Invalid ViewState from SPAM bots

Rick,
Its funny you mention this because I have similar global exception logging code to yours, and after I installed my RPC Ping server "fire and forget" code, I noticed some bogus requests started coming in and added a couple of IP addresses to my "IsBannedIP" method. This method is called very early in the Request pipline and simply stops them dead in their tracks before you even spin up a handler. It turned out that one of them was the googlebot! So, while I still have a few IPs in the list, I check them carefully.

Rick Strahl
July 31, 2007

# re: Shutting up Invalid ViewState from SPAM bots

@Peter - Google shouldn't trigger InvalidViewState or RequestValidation errors because they're not posting at least, so that should be easy to trap.

As to IP trapping I put it right into the IIS restricted list for the site
http://west-wind.com/WebLog/posts/59731.aspx, but that's only an option if you run your own server and have full rights of course. I do it as a separate step too - not as part of the request cycle because that's pretty overhead intensive (DirectoryServices and all).

But the chance to cut something like a Google bot off is easily possible so gotta be careful <s>...

John Vogel
July 31, 2007

# re: Shutting up Invalid ViewState from SPAM bots

I filter errors from being logged by a list of user agents as well as IPs. That may help with the different IPs, but only of course if the bot specifies a user agent.

Rick Strahl
August 01, 2007

# re: Shutting up Invalid ViewState from SPAM bots

@John, yeah that works for filtering robots, but it won't work for SPAM. From what I see SPAM bots always simultate regular browsers. Finding SPAMMING robot patterns are not really trackable as far as I can tell. Browser and IP address change in subsequent requests so there's not even a way to correlate.

It sure feels like a losing battle. The best you can hope for is that the SPAM doesn't actually have any effect on your application.

Amit
August 02, 2007

# re: Shutting up Invalid ViewState from SPAM bots

I completly agree with you Rick and thank for this wonderful post.

Owain Cleaver
August 02, 2007

# re: Shutting up Invalid ViewState from SPAM bots

I like Glen's idea of redirection - if they are found to follow 301s and download the entire response i say redirect them to the largest download you can find :-)

Rick Strahl
August 02, 2007

# re: Shutting up Invalid ViewState from SPAM bots

@Owain - The IP addresses aren't web servers most likely so it's not going to have any effect. But even if it did work I think that's a bad idea - all you're doing is pissing off those people and ensure they know that you care enough to make a ruckus which will only attract them all the more to your site.

John Vogel
August 03, 2007

# re: Shutting up Invalid ViewState from SPAM bots

From your code it looks like every single viewstate exception will be blocked on that page but if the problem is still occurring, would it be helpful to remove any recent errors from an IP once it has been identified as a spammer? Otherwise, make every first time user enter a CAPTCHA :D

You have great blog topics by the way, rock on.

Rick Strahl
August 03, 2007

# re: Shutting up Invalid ViewState from SPAM bots

@John - yes that's the point <s>. As mentioned above there appears to be no way to differentiate requests and 'filter' them because user agent and IP change on every request. Nor are these very frequent (2 or 3 at a time about twice an hour or so), but it's enough to make logging and notifications tedious. The Viewstate errors are generated because the Spammers are simply POSTing a set of data that includes some old random ViewState they captured on their original pass. It'll always fail because of that.

Typically I want to know about all errors in a system so I certainly don't want to globally block ViewState errors, but in this case it seems the attacks are isolated to this particular URL and it's unlikely (though possible) that legitimate Viewstate errors would occur.

Captcha's not option in the middle of a Web Store <s>... and it wouldn't solve this problem of the errors anyway.

Nicholas
August 07, 2007

# re: Shutting up Invalid ViewState from SPAM bots

I still get this crap on my lowly web server.. as well as the constant e-mails.. "Invalid Viewstate".

Usually it's like a spam e-mail's text, too. Don't these guys have lives? Are they trying to spam server admins/webmasters?

Milan Negovan
August 09, 2007

# re: Shutting up Invalid ViewState from SPAM bots

Rick, don't worry about pissing them off by locking them out. It's a "broken windows" paradox: if you show them there's someone on the other end purging the junk, they'll have little insentive to waste their time. Their "economics" is low-cost & high volume. You're a "bad customer" if you waste their time.

I've had an HttpModule similar to your for some time now. It looks at common view state "attacks" and black-lists offending IPs on the spot. It's all automated so don't worry if it grows to 2000 entries or whatever. I have 10,300 so far. Those are zombies and free proxies and crap anyway.

Also, those view state attacks always have patterns. They either feed you entire emails (what are they thinking?) or post links to farms, etc. They are easy enough to anaylze and code against.

One more thing: if they actually feed a bogus view state string (which was the case on my site), there's always something wacky with it, like an illegal character in the middle. ASP.NET won't botch up view state like that, neighter will Google or a news agent. So it's a pretty safe bet that if view state is corrupted, something is messing with it deliberately, so you can ban them.

I have an HttpModule here: http://www.aspnetresources.com/blog/fighting_view_state_spam.aspx which you are welcome to borrow. I also always recommend to generate a strong
machineKey
(see this post: http://aspnetresources.com/blog/view_state_lockdown.aspx)

Hope this helps!

Scott
June 03, 2008

# re: Shutting up Invalid ViewState from SPAM bots

I know this is old but how about this strategy. When you start getting alot of viewstate errors using old viewstate data what about ignoring those based on what viewstate data they are using? Would that be a valid strategy?

J. Bishop
August 28, 2009

# re: Shutting up Invalid ViewState from SPAM bots

Great column, i implemented and it works great. But IE8 is dropped 4k in some files causing view state errors and it's doing that in AXD files too.

So my AXD files stemming from AJAX (WebResource.axd,ScriptResource.axd) are kicking off view state errors. Can a global error handler handle AXDs as well as ASPX?

West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2024