How to Post Messages to an Sta Thread Running a Message Pump

How to post messages to an STA thread running a message pump?

Keep in mind that the message queue that Windows creates for an STA thread is already an implementation of a thread-safe queue. So just use it for your own purposes. Here's a base class that you can use, derive your own to include your COM object. Override the Initialize() method, it will be called as soon as the thread is ready to start executing code. Don't forget to call base.Initialize() in your override.

It you want to run code on that thread then use the BeginInvoke or Invoke methods, just like you would for the Control.Begin/Invoke or Dispatcher.Begin/Invoke methods. Call its Dispose() method to shut down the thread, it is optional. Beware that this is only safe to do when you are 100% sure that all COM objects are finalized. Since you don't usually have that guarantee, it is better that you don't.

using System;
using System.Threading;
using System.Windows.Forms;

class STAThread : IDisposable {
    public STAThread() {
        using (mre = new ManualResetEvent(false)) {
            thread = new Thread(() => {
                Application.Idle += Initialize;
                Application.Run();
            });
            thread.IsBackground = true;
            thread.SetApartmentState(ApartmentState.STA);
            thread.Start();
            mre.WaitOne();
        }
    }
    public void BeginInvoke(Delegate dlg, params Object[] args) {
        if (ctx == null) throw new ObjectDisposedException("STAThread");
        ctx.Post((_) => dlg.DynamicInvoke(args), null);
    }
    public object Invoke(Delegate dlg, params Object[] args) {
        if (ctx == null) throw new ObjectDisposedException("STAThread");
        object result = null;
        ctx.Send((_) => result = dlg.DynamicInvoke(args), null);
        return result;
    }
    protected virtual void Initialize(object sender, EventArgs e) {
        ctx = SynchronizationContext.Current;
        mre.Set();
        Application.Idle -= Initialize;
    }
    public void Dispose() {
        if (ctx != null) {
            ctx.Send((_) => Application.ExitThread(), null);
            ctx = null;
        }
    }
    private Thread thread;
    private SynchronizationContext ctx;
    private ManualResetEvent mre;
}

Which blocking operations cause an STA thread to pump COM messages?

BlockingCollection will indeed pump while blocking. I've learnt that while answering the following question, which has some interesting details about STA pumping:

StaTaskScheduler and STA thread message pumping

However, it will pump a very limited undisclosed set of COM-specific messages, same as the other APIs you listed. It won't pump general purpose Win32 messages (a special case is WM_TIMER, which won't be dispatched either). This might be a problem for some STA COM objects which expect a full-featured message loop.

If you like to experiment with this, create your own version of SynchronizationContext, override SynchronizationContext.Wait, call SetWaitNotificationRequired and install your custom synchronization context object on an STA thread. Then set a breakpoint inside Wait and see what APIs will make it get called.

To what extent the standard pumping behavior of WaitOne is actually limited? Below is a typical example causing a deadlock on the UI thread. I use WinForms here, but the same concern applies to WPF:

public partial class MainForm : Form
{
    public MainForm()
    {
        InitializeComponent();

        this.Load += (s, e) =>
        {
            Func<Task> doAsync = async () =>
            {
                await Task.Delay(2000);
            };

            var task = doAsync();
            var handle = ((IAsyncResult)task).AsyncWaitHandle;

            var startTick = Environment.TickCount;
            handle.WaitOne(4000);
            MessageBox.Show("Lapse: " + (Environment.TickCount - startTick));
        };
    }
}

The message box will show the time lapse of ~ 4000 ms, although the task takes only 2000 ms to complete.

That happens because the await continuation callback is scheduled via WindowsFormsSynchronizationContext.Post, which uses Control.BeginInvoke, which in turn uses PostMessage, posting a regular Windows message registered with RegisterWindowMessage. This message doesn't get pumped and handle.WaitOne times out.

If we used handle.WaitOne(Timeout.Infinite), we'd have a classic deadlock.

Now let's implement a version of WaitOne with explicit pumping (and call it WaitOneAndPump):

public static bool WaitOneAndPump(
    this WaitHandle handle, int millisecondsTimeout)
{
    var startTick = Environment.TickCount;
    var handles = new[] { handle.SafeWaitHandle.DangerousGetHandle() };

    while (true)
    {
        // wait for the handle or a message
        var timeout = (uint)(Timeout.Infinite == millisecondsTimeout ?
                Timeout.Infinite :
                Math.Max(0, millisecondsTimeout + 
                    startTick - Environment.TickCount));

        var result = MsgWaitForMultipleObjectsEx(
            1, handles,
            timeout,
            QS_ALLINPUT,
            MWMO_INPUTAVAILABLE);

        if (result == WAIT_OBJECT_0)
            return true; // handle signalled
        else if (result == WAIT_TIMEOUT)
            return false; // timed-out
        else if (result == WAIT_ABANDONED_0)
            throw new AbandonedMutexException(-1, handle);
        else if (result != WAIT_OBJECT_0 + 1)
            throw new InvalidOperationException();
        else
        {
            // a message is pending 
            if (timeout == 0)
                return false; // timed-out
            else
            {
                // do the pumping
                Application.DoEvents();
                // no more messages, raise Idle event
                Application.RaiseIdle(EventArgs.Empty);
            }
        }
    }
}

And change the original code like this:

var startTick = Environment.TickCount;
handle.WaitOneAndPump(4000);
MessageBox.Show("Lapse: " + (Environment.TickCount - startTick));

The time lapse now will be ~2000 ms, because the await continuation message gets pumped by Application.DoEvents(), the task completes and its handle is signaled.

That said, I'd never recommend using something like WaitOneAndPump for production code (besides for very few specific cases). It's a source of various problems like UI re-entrancy. Those problems are the reason Microsoft has limited the standard pumping behavior to only certain COM-specific messages, vital for COM marshaling.

StaTaskScheduler and STA thread message pumping

My understanding of your problem: you are using StaTaskScheduler only to organize the classic COM STA apartment for your legacy COM objects. You're not running a WinForms or WPF core message loop on the STA thread of StaTaskScheduler. That is, you're not using anything like Application.Run, Application.DoEvents or Dispatcher.PushFrame inside that thread. Correct me if this is a wrong assumption.

By itself, StaTaskScheduler doesn't install any synchronization context on the STA threads it creates. Thus, you're relying upon the CLR to pump messages for you. I've only found an implicit confirmation that the CLR pumps on STA threads, in Apartments and Pumping in the CLR by Chris Brumme:

I keep saying that managed blocking will perform “some pumping” when
called on an STA thread. Wouldn’t it be great to know exactly what
will get pumped? Unfortunately, pumping is a black art which is
beyond mortal comprehension. On Win2000 and up, we simply delegate to
OLE32’s CoWaitForMultipleHandles service.

This indicates the CLR uses CoWaitForMultipleHandles internally for STA threads. Further, the MSDN docs for COWAIT_DISPATCH_WINDOW_MESSAGES flag mention this:

... in STA is only a small set of special-cased messages dispatched.

I did some research on that, but could not get to pump the WM_TEST from your sample code with CoWaitForMultipleHandles, we discussed that in the comments to your question. My understanding is, the aforementioned small set of special-cased messages is really limited to some COM marshaller-specific messages, and doesn't include any regular general-purpose messages like your WM_TEST.

So, to answer your question:

... Should I implemented a custom synchronization context, which would
explicitly pump messages with CoWaitForMultipleHandles, and install it
on each STA thread started by StaTaskScheduler?

Yes, I believe that creating a custom synchronization context and overriding SynchronizationContext.Wait is indeed the right solution.

However, you should avoid using CoWaitForMultipleHandles, and use MsgWaitForMultipleObjectsEx instead. If MsgWaitForMultipleObjectsEx indicates there's a pending message in the queue, you should manually pump it with PeekMessage(PM_REMOVE) and DispatchMessage. Then you should continue waiting for the handles, all inside the same SynchronizationContext.Wait call.

Note there's a subtle but important difference between MsgWaitForMultipleObjectsEx and MsgWaitForMultipleObjects. The latter doesn't return and keeps blocking, if there's a message already seen in the queue (e.g., with PeekMessage(PM_NOREMOVE) or GetQueueStatus), but not removed. That's not good for pumping, because your COM objects might be using something like PeekMessage to inspect the message queue. That might later cause MsgWaitForMultipleObjects to block when not expected.

OTOH, MsgWaitForMultipleObjectsEx with MWMO_INPUTAVAILABLE flag doesn't have such shortcoming, and would return in this case.

A while ago I created a custom version of StaTaskScheduler (available here as ThreadAffinityTaskScheduler) in attempt to solve a different problem: maintaining a pool of threads with thread affinity for subsequent await continuations. The thread affinity is vital if you use STA COM objects across multiple awaits. The original StaTaskScheduler exhibits this behavior only when its pool is limited to 1 thread.

So I went ahead and did some more experimenting with your WM_TEST case. Originally, I installed an instance of the standard SynchronizationContext class on the STA thread. The WM_TEST message didn't get pumped, which was expected.

Then I overridden SynchronizationContext.Wait to just forward it to SynchronizationContext.WaitHelper. It did get called, but still didn't pump.

Finally, I implemented a full-featured message pump loop, here's the core part of it:

// the core loop
var msg = new NativeMethods.MSG();
while (true)
{
    // MsgWaitForMultipleObjectsEx with MWMO_INPUTAVAILABLE returns,
    // even if there's a message already seen but not removed in the message queue
    nativeResult = NativeMethods.MsgWaitForMultipleObjectsEx(
        count, waitHandles,
        (uint)remainingTimeout,
        QS_MASK,
        NativeMethods.MWMO_INPUTAVAILABLE);

    if (IsNativeWaitSuccessful(count, nativeResult, out managedResult) || WaitHandle.WaitTimeout == managedResult)
        return managedResult;

    // there is a message, pump and dispatch it
    if (NativeMethods.PeekMessage(out msg, IntPtr.Zero, 0, 0, NativeMethods.PM_REMOVE))
    {
        NativeMethods.TranslateMessage(ref msg);
        NativeMethods.DispatchMessage(ref msg);
    }
    if (hasTimedOut())
        return WaitHandle.WaitTimeout;
}

This does work, WM_TEST gets pumped. Below is an adapted version of your test:

public static async Task RunAsync()
{
    using (var staThread = new Noseratio.ThreadAffinity.ThreadWithAffinityContext(staThread: true, pumpMessages: true))
    {
        Console.WriteLine("Initial thread #" + Thread.CurrentThread.ManagedThreadId);
        await staThread.Run(async () =>
        {
            Console.WriteLine("On STA thread #" + Thread.CurrentThread.ManagedThreadId);
            // create a simple Win32 window
            IntPtr hwnd = CreateTestWindow();

            // Post some WM_TEST messages
            Console.WriteLine("Post some WM_TEST messages...");
            NativeMethods.PostMessage(hwnd, NativeMethods.WM_TEST, new IntPtr(1), IntPtr.Zero);
            NativeMethods.PostMessage(hwnd, NativeMethods.WM_TEST, new IntPtr(2), IntPtr.Zero);
            NativeMethods.PostMessage(hwnd, NativeMethods.WM_TEST, new IntPtr(3), IntPtr.Zero);
            Console.WriteLine("Press Enter to continue...");
            await ReadLineAsync();

            Console.WriteLine("After await, thread #" + Thread.CurrentThread.ManagedThreadId);
            Console.WriteLine("Pending messages in the queue: " + (NativeMethods.GetQueueStatus(0x1FF) >> 16 != 0));

            Console.WriteLine("Exiting STA thread #" + Thread.CurrentThread.ManagedThreadId);
        }, CancellationToken.None);
    }
    Console.WriteLine("Current thread #" + Thread.CurrentThread.ManagedThreadId);
}

The output:

Initial thread #9
On STA thread #10
Post some WM_TEST messages...
Press Enter to continue...
WM_TEST processed: 1
WM_TEST processed: 2
WM_TEST processed: 3

After await, thread #10
Pending messages in the queue: False
Exiting STA thread #10
Current thread #12
Press any key to exit

Note this implementation supports both the thread affinity (it stays on the thread #10 after await) and the message pumping. The full source code contains re-usable parts (ThreadAffinityTaskScheduler and ThreadWithAffinityContext) and is available here as self-contained console app. It hasn't been thoroughly tested, so use it at your own risk.

How can you pump a STA Thread in a non-WinForms assembly?

You may use AsyncContextThread from Nito.AsyncEx.Context nuget by Stephen Cleary. From its github description

AsyncContextThread provides properties that can be used to schedule tasks on that thread.

The nuget contains a custom implementation of SynchronizationContext, so the code could be easily rewritten e.g. as:

using System;
using System.Threading;
using Nito.AsyncEx;

class STAThread : IDisposable
{
    public STAThread()
    {
        ctx = new AsyncContextThread();
    }

    public void BeginInvoke(Delegate dlg, params Object[] args)
    {
        ctx.Context.SynchronizationContext
            .Post((_) => dlg.DynamicInvoke(args), null);
    }

    public object Invoke(Delegate dlg, params Object[] args)
    {
        object result = null;
        ctx.Context.SynchronizationContext
            .Send((_) => result = dlg.DynamicInvoke(args), null);
        return result;
    }
    
    public void Dispose()
    {
        ctx.JoinAsync().GetAwaiter().GetResult();
        ctx.Dispose();
    }

    private readonly AsyncContextThread ctx;
}

Btw, from this MSDN article, not all implementations of SynchronizationContext guarantee that delegates will be executed on specific thread, while WinForms and WPF SynchronizationContext guarantee that, default and ASP.NET do not.

Is STA Message Loop Required in This Case?

The STA contract requires pumping a message loop. But yes, it is possible to get away with not pumping. There are two major things that can go wrong:

• Any call that's made on an interface method from another apartment, including another STA thread or a thread in the MTA will not complete. This looks like deadlock in your program, the call simply never returns. Beware that you can control your own calls quite well but you don't know what the COM component is doing. It may well start a thread itself. You can see this in the debugger with Debug + Windows + Threads. Make sure you are running the debugger in unmanaged mode and that you can account for all the threads you see. Not particularly easy btw.

• Many apartment threaded COM components count on having a message loop taking care of their own needs. It could be something innocuous as a Timer, it won't tick when there's no loop. Or it may do marshaling internally. Use Spy++ and check if there are any hidden windows owned by your new STA thread, sure sign of trouble if you see one. The diagnosis is the component just misbehaving. Not raising events is a common mishap.

Nothing really to nail to a wall when you don't know enough about the internals of the server. Be sure to test the heck out of it.

UI Thread both running message pump AND executing code?

It takes time out of processing the message pump to handle the message, as this is synchronously called from the message pump (via the click event).

This is why expensive code can cause the UI to hang.

How to pump COM messages?

The short and long of it is that you have to pump ALL messages normally, you can't just single out COM messages by themselves (and besides, there is no documented messages that you can peek/pump by themselves, they are known only to COM's internals).

How to make WebBrower.Navigate2 synchronous?

You can't. But you don't have to wait for the OnDocumentComplete event, either. You can busy-loop inside of NavigateToEmpty() itself until the WebBrowser's ReadyState property is READYSTATE_COMPLETE, pumping the message queue when messages are waiting to be processed:

procedure TContoso.NavigateToEmpty(WebBrowser: IWebBrowser2);
begin
  WebBrowser.Navigate2('about:blank');
  while (WebBrowser.ReadyState <> READYSTATE_COMPLETE) and (not Application.Terminated) do
  begin
    // if MsgWaitForMultipleObjects(0, Pointer(nil)^, False, 5000, QS_ALLINPUT) = WAIT_OBJECT_0 then
    // if GetQueueStatus(QS_ALLINPUT) <> 0 then
      Application.ProcessMessages;
  end;
end;

How to pump COM messages?

You can't, not by themselves anyway. Pump everything, and be prepared to handle any reentry issues that result from that.

Does pumping COM messages cause COM events to callback?

Yes.

How to use CoWaitForMultipleHandles

Try something like this:

procedure TContoso.NavigateToEmpty(WebBrowser: IWebBrowser2);
var
  hEvent: THandle;
  dwIndex: DWORD;
  hr: HRESULT;
begin
  // when UseCOMWait() is true, TEvent.WaitFor() does not wait for, or
  // notify, when messages are pending in the queue, so use
  // CoWaitForMultipleHandles() directly instead.  But you have to still
  // use a waitable object, just don't signal it...
  hEvent := CreateEvent(nil, True, False, nil);
  if hEvent = 0 then RaiseLastOSError;
  try
    WebBrowser.Navigate2('about:blank');
    while (WebBrowser.ReadyState <> READYSTATE_COMPLETE) and (not Application.Terminated) do
    begin
      hr := CoWaitForMultipleHandles(COWAIT_INPUTAVAILABLE, 5000, 1, hEvent, dwIndex);
      case hr of
        S_OK: Application.ProcessMessages;
        RPC_S_CALLPENDING, RPC_E_TIMEOUT: begin end;
      else
        RaiseLastOSError(hr);
      end;
    end;
  finally
    CloseHandle(hEvent);
  end;
end;

---

Related Topics