Last week I started looking into a problem that causes my IIS Worker processes to have errors on shutdown. After some mind numbing debugging I finally seem to have traced the problem down to COM object invokation and a handle leak that results because of it. What I ran into here though is very odd as it appears to be not specific to my code but a general handle leak when instantiating EXE servers.

First off let me say C++ and COM are not my forte especially now after having been away from it for a long time. But I've boiled down the following problem to its bare minimum and I'm completely stumped as to why the code would leak handles because it's such fundamental COM code.

Ok so the scenario is as follows: I have a COM client (in my 'real' app it's the ISAPI extension) that is instantiating EXE COM servers and then shortly after releasing them all happening on a single thread. This scenario occurs for certain long running processes that go off and run outside of the normal thread pool. However, the problem is reproducible without any special thread related issues interfering.

In any case the operation of the servers works perfectly fine - servers load and are properly unloaded when inst->Release() is called. The problem is that there is a Windows Handle leak.

As I was debugging my app I noticed that the code that loads EXE COM instances was effectively leaking a Windows handle when I checked in ProcessExplorer. So I kept simplifying the code more and more until I ended up with nothing more than a call to CoCreateInstance() and a inst->Release() call in COM, which in effect leaked a handle. It leaks a handle when inst->Release() is called and it does this only on EXE servers, not on DLL servers.

I managed to boil the code down in a simple C++ Console application:

#include "stdafx.h"

int _tmain(int argc, _TCHAR* argv[])
{

    // COM Instance Handle Leak Bug
    IUnknown *pComObj = NULL;
    //IDispatch *pComObj = NULL;
    CLSID clsid;
    
    HRESULT hr = CLSIDFromProgID( L"VisualFoxPro.Application",&clsid);

    CoInitialize(NULL);
    char input[80];

    for(int x=1; x< 10; x++)
    {
        // *** Commenting the next two lines out results in no handle leaks
        // *** Running these two lines will leak one handle per request
        hr = CoCreateInstance(clsid, NULL, CLSCTX_SERVER, IID_IUnknown, (LPVOID *) &pComObj);
        //hr = CoCreateInstance(clsid, NULL, CLSCTX_SERVER, IID_IDispatch, (LPVOID *) &pComObj);

        DWORD count = pComObj->Release();
        
        printf("%d: %d\r\n",x,count);    
        scanf("%c",input);

    }
    printf("Completed.");
    scanf("%c",input);

    CoUninitialize();

    return 0;
}

When I run this code and check the handle count in process Explorer I see the handle count increase on every single call to  pComObj->Release().

It doesn't matter what EXE server I use here. I'm using a custom COM server of mine, or a more generic server above. Tried a tiny little ATL server, tried a few others like SnagIt, Word, Excel - the behavior is exactly the same - they all leak a handle.

Taking a closer look at the handles that are leaked on Release in Process Explorer (View | Lower Pane | Handles then Show unnamed handles) results in a <Unknown handle> created:

So as I said at the outside I'm not a C++ wizard, but this seems like a pretty fundamental problem given how basic this code is. Create an instance and release it - can't get any easier right? I've never noticed this before because I never debugged this all the way, but in testing I'm now seeing that the same behavior occurs on XP, Windows 2003 server and on my dev setup on Vista (as well as on Server 2008). I also tried the more round about DCOM mechanism using CoCreateInstanceEx() but the results there are the same.

I just can't make sense of what's happening here as the code is about as fundamental as it comes - make a call to CoCreateInstance() which ups the ref count, then call Release() to unload the server and all resources associated with it. The actual COM server is behaving properly, but its the damn handles that are being left behind.

What could possibly be wrong with this code? What else could I possibly do to release a handle that I don't even hold in my code? Is there some other way to release resources? It would appear that there's some issue in the DCOM proxy clean up that occurs when the server is released, but I'm not sure what that would be.

Are there any COM  wonks that can point at the error in my ways?

I've attached the C++ project and EXE if anybody's adventurous - if you check this out you may have to change the server name to an EXE server installed on your machine (Word.Application or whatever that exists - although Word is problematic because it doesn't actually unload unless visibility is set off or Close() is called).