Releasing a Unplugged Virtual Serial Port

Releasing a unplugged virtual Serial Port

Serial ports date from the stone age of computing. That's where you plugged in your ASR-33 teletype to start typing in your Fortran program. The electrical interface is very simple. So is the Windows API to use a serial port from your own code. Practically any runtime environment supports them.

USB has replaced serial port hardware completely. It has a much more advanced logical interface to the machine, supporting many different type of devices. And it supports Plug and Play, allowing the operating system to detect when a device is attached or removed as well as automatically installing the device driver, etcetera.

This flexibility comes at a price however, a USB device always needs a device driver to become usable. Device drivers are not created equal. Different drivers require different ways to talk to the device. Usually done through DeviceIoControl() or Read/WriteFile() but those are very opaque API functions. In the early days of USB, device manufacturers would supply a DLL that provided a rich API to hide the implementation details.

That did not work so well, manufacturers are not very good at writing good APIs and they sure don't like to support them. So a good solution would be to support a standard API, one that's available on any machine, supported by any runtime, documented and maintained by somebody else. Like the serial port API.

That did not work so well, manufacturers are not very good at writing device drivers that emulate serial ports. The biggest hang-up with the API is that it doesn't have any support for Plug and Play. The core support for it is missing, after all serial port hardware doesn't have the logical interface to support it. There is some support for detecting that a device is attached through the DTR hardware handshake line, but no support whatsoever for detecting that the port is no longer there.

Detaching the USB device is the problem. In an ideal world, the emulator built into the device driver would simply pretend that the serial port is still there until the last handle on the device is closed. That would be the logical implementation, given that there's no way to trigger a Plug and Play event. For some strange reason that seems to be difficult to implement. Most USB drivers take the crummy shortcut, they simply make the device disappear even while it is in use.

This plays havoc on any user mode code that uses the device. Which is typically written to assume it is a real serial port and real serial ports don't suddenly disappear. At least not without drawing a bright blue spark. What goes wrong is pretty unpredictable because it depends on how the driver responds to requests on a device that's no longer there. An uncatchable exception in a worker thread started by SerialPort was a common mishap. Sounds like your driver really gets it wrong, it generates an error return code on the MJ_CLOSE driver request. Which is kind of a logical thing to do for a driver, after all the device isn't there anymore, but quite unsolvable from your end. You have a handle and you can't close it. That's up a creek with no paddle.

Every major release of .NET had a small patch to the SerialPort classes to try to minimize the misery a bit. But there's a limited amount that Microsoft can do, catching all errors and pretending they didn't happen ultimately leads to class that provides no good diagnostic anymore, even with a good driver.

So practical approaches are:

always use the Remove Hardware Safely tray icon in Windows
use the latest version of .NET
contact the vendor and ask for a driver update
ditch vendors that supply lousy drivers
tell your users that, just because it is the only thing you can do with a USB device, that unplugging it doesn't solve any problems
make closing the port easy and accessible in your UI
glue the USB connector to the port so it can't be removed

The 5th bullet is also what gets programmers in trouble. Writing serial port code isn't easy, it is heavily asynchronous and the threadpool thread that runs the DataReceived event is difficult to deal with. When you can't diagnose the software problem you tend to blame the hardware. There's very little you can do with the hardware but unplug it. Bad Idea. Now you have two problems.

USB Serial Port Unplugged but still in the list of Ports

It is entirely up to the USB device driver that emulates the serial port. Unplugging a port while it is opened is in general a very bad idea. There are plenty of drivers that make the port disappear, even if your SerialPort object has a handle opened on the port. This tends to crash a worker thread that generates the DataReceived, PinChanged and ErrorReceived events. The exception is not catchable because it occurs on a worker thread, terminating your program. Some drivers even reject an attempt to close the port, making it impossible to end your program cleanly.

Sounds like you have a decent driver that keeps the emulated port alive as long as you don't call Close(). That's a Good Thing, not a problem. Don't count on this working on your user's machine, you cannot predict what kind of driver they'll get with their device. A buying recommendation is a good idea.

Long story short, serial ports are very primitive devices that date from the stone age of computing. There is no plug and play support for them so what happens is completely unpredictable. Only sane thing to do is never to unplug the cable while the device is in use. This is not hard to do :) More about the kind of trouble it causes in this answer.

How to capture a serial port that disappears because the usb cable gets unplugged

You can use WMI (Windows Management Instrumentation) to receive notification on USB events.
I did exactly that two years ago, monitoring for plugging and unplugging of a specific usb device.

Unfortunately, the code stays with my former employer, but I found one example at bytes.com:

using System;
using System.ComponentModel;
using System.Runtime.InteropServices;
using System.Management;
class UsbWatcher 
{
    public static void Main() 
    {
        WMIEvent wEvent = new WMIEvent();
        ManagementEventWatcher watcher = null;
        WqlEventQuery query;
        ManagementOperationObserver observer = new ManagementOperationObserver();

        ManagementScope scope = new ManagementScope("root\\CIMV2");
        scope.Options.EnablePrivileges = true; 
        try 
        {
            query = new WqlEventQuery();
            query.EventClassName = "__InstanceCreationEvent";
            query.WithinInterval = new TimeSpan(0,0,10);

            query.Condition = @"TargetInstance ISA 'Win32_USBControllerDevice' ";
            watcher = new ManagementEventWatcher(scope, query);

            watcher.EventArrived 
                += new EventArrivedEventHandler(wEvent.UsbEventArrived);
            watcher.Start();
        }
        catch (Exception e)
        {
            //handle exception
        }
}

I don't remember if I modified the query to receive events only for e specific device, or if I filtered out events from other devices in my event handler. For further information you may want to have a look at the MSDN WMI .NET Code Directory.

EDIT
I found some more info on the event handler, it looks roughly like this:

protected virtual void OnUsbConnected(object Sender, EventArrivedEventArgs Arguments)
{
    PropertyData TargetInstanceData = Arguments.NewEvent.Properties["TargetInstance"];

    if (TargetInstanceData != null)
    {
        ManagementBaseObject TargetInstanceObject = (ManagementBaseObject)TargetInstanceData.Value;
        if (TargetInstanceObject != null)
        {
            string dependent = TargetInstanceObject.Properties["Dependent"].Value.ToString();
            string deviceId = dependent.Substring(dependent.IndexOf("DeviceID=") + 10);

            // device id string taken from windows device manager
            if (deviceId = "USB\\\\VID_0403&PID_6001\\\\12345678\"")
            {
                // Device is connected
            }
        }
    }
}

You may want to add some exception handling, though.

Serial Port Cable Unplugged

Unlike USB, the serial port does not have any built-in way to detect a physical change in link status. A limited form of "device ready/not ready" signalling can be done using some of the pins (namely DTR, DSR, and sometimes DCD), but this doesn't seem like exactly what you're looking for (it's not built in to RS232 -- the device must support it, you mainly use it to talk to modems.)

So, in short: no, in the general case. If you know/can program the device you're trying to communicate with, and you know that it will hold a certain line high (for example), you could poll it looking for that line to go high. But if you plug in a device which isn't programmed to do something predictable like that, then there's really no way to tell. (Some devices may hold DSR high by default but it's in no way a sure bet.)

Serial port blocked by none process

I kind of solved the problem.

It turns out that if Windows detects constant data flow when the serial port enumarion starts (system boot or when you enable/load a new devices), Windows identifies the serial port COM as a serial mouse, creating a new devices linked to that serial port. So the port won't be available for any other process.

I found out the problem reading this refs (including a similar question):

RS232 COM port blocked after Windows restart
Force a COM port to close - Stack Overflow
How to use a standard serial mouse with the VAIO computer? | Sony
IN
My Computer thinks the GPS unit is a mouse

I'd like to give more details about what I did.

I made a C# ConsoleApp which constanly sends data to the port by using a USB-RS232 adapter connected to another PC which has a legacy serial port.
This is the app's code:

class SerialPortConnection : IDisposable
{
    SerialPort _serialPort;
    public void Start()
    {
        _serialPort = new SerialPort();
        _serialPort.BaudRate = 9600;
        _serialPort.Parity = Parity.None;
        _serialPort.StopBits = StopBits.One;
        _serialPort.Handshake = Handshake.None;
        _serialPort.PortName = "COM5";

        _serialPort.ReadTimeout = 500;
        _serialPort.WriteTimeout = 100;

        this._serialPort.Open();
        int i = 10 * 60 * 1000;//120 segundos

        while (i > 0)
        {
            this._serialPort.Write("ZABCDEFGHIJKLNOPQRSTUG");//22 bytes ->22 x 4(250ms) = 88 bytes/s -> con baudrate 9600
            
            
            System.Threading.Thread.Sleep(250);                                            
        
        }
        
    }
    public void Dispose()
    {
        this._serialPort.Close();
    }
}

I tried several amounts of bytes, at the end I realized that if the app sends exactly or more than 22 bytes each 250ms using 9600 baudrate and force an enumeration ( windows reboot or disabling/enabling or deleting the device), the port gets blocked and Windows creates a new device, a Serial Mouse which takes control of the COM port:

Screenshot Serial Mouse in Windows Device Manager

So that's why the port refuses any new connection.
To avoid it, you only need to disable the Serial Mouse (not uninstall, because in future reboots, windows may load it again).

You also can modify the Windows Registry to avoid the service to start:

\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\sermouse

Replacing the value 3 to 4 in the key Start. (3 is Default, 4 is Disabled).

if you don't have acces to Windows Registry, there's a simple (and temporary) solution.
Let's recap: the error comes from the identification (enumeration) of the port, when data is coming into the port, so, if you can avoid data flowing at the moment of the enumeration, then you got it.

Unplug the port before turning on the PC, and once Windows starts, you can plug it.
If you can manipulate your Serial device to stop working (actually, stop sending data) like turning it off, or pausing in some way the device, then you can boot Windows and then resume the device work.
If you have acces to Windows device Manager, you can apply the second point by disabling/enabling the COM port rather than rebooting Window.
You also can disable the COM port, unplug it, and enable it again (the SerialMouse will be removed) and plug it.

Getting these solutions took me more than a week. I hope it could be useful for someone else.

COM port disappears when unplugging USB

I found a solution that can handle plugging and unplugging a SerialPort.

First of all, it requires the use the SafeSerialPort, which allows you to dispose the serial port properly.

SafeSerialPort serialPort;

private void Connect()
{
    string portname = "COM8";
    serialPort = new SafeSerialPort(portname, 9600);
    serialPort.DataReceived += port_DataReceived;
    serialPort.Open(); 
}

Second, you need to use LibUsbDotNet to detect whether a USB device is connected or disconnected. This will allow you to determine whether to connect to the device or reset the COM port.

public UsbDevice MyUsbDevice;

//Find your vendor id etc by listing all available USB devices
public UsbDeviceFinder MyUsbFinder = new UsbDeviceFinder(0x2341, 0x0001);
public IDeviceNotifier UsbDeviceNotifier = DeviceNotifier.OpenDeviceNotifier();
private void OnDeviceNotifyEvent(object sender, DeviceNotifyEventArgs e)
{
    if (e.Object.ToString().Split('\n')[1].Contains("0x2341"))
    {
        if (e.EventType == EventType.DeviceArrival)
        {
            Connect();
        }
        else if(e.EventType == EventType.DeviceRemoveComplete)
        {
            ResetConnection();
        }
    }
}

Finally, disposing the SerialPort will makes sure it is registered by Windows in HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\SERIALCOMM, meaning that SerialPort.GetPortNames() can re-detect the port.

private void ResetConnection()
{
    try
    {
        //Send any data to cause an IOException
        serialPort.Write("Any value");
    }
    catch (IOException ex)
    {
        //Dispose the SafeSerialPort
        serialPort.Dispose();
        serialPort.Close();
    }
}

After this process, you can simply reconnect to the COM port when the USB device is connected without the need to restart the application.

Full code:

using LibUsbDotNet;
using LibUsbDotNet.DeviceNotify;
using LibUsbDotNet.Info;
using LibUsbDotNet.Main;    

SafeSerialPort serialPort;

            public SerialPortTest()
            {
                Connect();

                UsbDeviceNotifier.OnDeviceNotify += OnDeviceNotifyEvent;
            }

            private void Connect()
            {
                string portname = "COM8";
                serialPort = new SafeSerialPort(portname, 9600);
                serialPort.DataReceived += port_DataReceived;
                serialPort.Open(); 
            }

            private void ResetConnection()
            {
                try
                {
                    serialPort.Write("Any value");
                }
                catch (IOException ex)
                {
                    serialPort.Dispose();
                    serialPort.Close();
                }
            }

            void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
            {
                Console.WriteLine(serialPort.ReadExisting());
            }

            public UsbDevice MyUsbDevice;

            //Vendor ID etc can be found through enumerating the USB devices
            public UsbDeviceFinder MyUsbFinder = new UsbDeviceFinder(0x2341, 0x0001);
            public IDeviceNotifier UsbDeviceNotifier = DeviceNotifier.OpenDeviceNotifier();
            private void OnDeviceNotifyEvent(object sender, DeviceNotifyEventArgs e)
            {
                //if this is your usb device, in my case an Arduino
                if (e.Object.ToString().Split('\n')[1].Contains("0x2341"))
                {
                    if (e.EventType == EventType.DeviceArrival)
                    {
                        Connect();
                    }
                    else
                    {
                        ResetConnection();
                    }
                }
            }