How to Detect a Buffer Over Run on Serial Port in Linux Using C++

how to detect a buffer over run on serial port in linux using c++

According to the kernel sources, you should use the TIOCGICOUNT ioctl. The third ioctl argument should be a pointer to the following struct, defined in <linux/serial.h> :

/*
 * Serial input interrupt line counters -- external structure
 * Four lines can interrupt: CTS, DSR, RI, DCD
 */
struct serial_icounter_struct {
        int cts, dsr, rng, dcd;
        int rx, tx;
        int frame, overrun, parity, brk;
        int buf_overrun;
        int reserved[9];
};

I don't know if every driver detect all conditions however.

How to handle buffering serial data

To minimize the overhead of making many read() syscalls of small byte counts (e.g. the misguided solution of reading a byte at a time), use an intermediate buffer in your code.

The read() of the serial terminal should be in blocking mode to avoid a return code of zero bytes.

#define BLEN    1024
unsigned char rbuf[BLEN];
unsigned char *rp = &rbuf[BLEN];
int bufcnt = 0;

/* get a byte from intermediate buffer of serial terminal */
static unsigned char getbyte(void)
{
    if ((rp - rbuf) >= bufcnt) {
        /* buffer needs refill */
        bufcnt = read(fd, rbuf, BLEN);
        if (bufcnt <= 0) {
            /* report error, then abort */
        }
        rp = rbuf;
    }
    return *rp++;
}

For proper termios initialization code for the serial terminal, see this answer. You should increase the VMIN parameter to something closer to the BLEN value or at least the length of longest expected message, and a VTIME of 1.

Now you can conveniently access the received data a byte at a time with minimal performance penalty.

#define MLEN    1024  /* choose appropriate value for message protocol */
int main() 
{
    unsigned char mesg[MLEN];
    ...

    while (1) {
        while (getbyte() != 0x10)
            /* discard data until start found */ ;
    
        length = 0;
        while ((mesg[length] = getbyte()) != 0x11) {
            /* accumulate data until end found */ 
            length++;
        }

        /* process the message */
        ...

    
    }  /* loop for next message */
...
}

Note that your detection for a message frame is not robust.

If the data is binary and therefore can use the same values as these start and end bytes, then this parsing of the received data is prone to misaligned message frames.

See this answer for a description of a proper alogrithm.

buffer problem when making serial communication with C++

Things to fix

Check return value of fgets()

Simplify as desired.

if (fgets(var,sizeof(var),fp)) {
  printf("<%s>\n",var);
} else {
  printf("Nothing this time.\n");
}

---

GTG

How read data from serialport?

You opened the device with O_NDELAY, which places the descriptor into non-blocking mode. If there is nothing to be read then EWOULDBLOCK is returned from read. You could for example:

• remove O_NDELAY from there for this simple program
• use poll/select etc to wait for the descriptor to become readable.

How to properly received serial data using C?

Ok, my problem is: when I set my LED status do 3 (0x3), I could not read status from device!

...

I'm surre that the device is sending correct values (4 bytes, in this case, 0x23, 0x03, 0x3B, 0x0A ...

0x03 is also the ASCII control code named ETX.

The default value for the termios special character VINTR is 0x03, which is also known as ETX and Ctrl-C.

When the termios ISIG attribute of c_lflag is set, then receiving the VINTR character will cause the process to get a SIGINT signal, and that character is removed from the input buffer.

---

if (canonical == 1) {
    ...
    tty.c_lflag |= ICANON | ISIG; /* canonical input */
    ...
} else {
    ...
    tty.c_lflag &= ~(ICANON);
    tty.c_lflag &= ~(ECHO | ECHOE);        
    ...
}

Your program sets the ISIG attribute for "canonical" mode, and otherwise leaves that attribute in c_lflag untouched.

Apparently VINTR still has its default value of 0x03.

The net effect is that the ISIG attribute is always enabled, and therefore receiving 0x03 will always be unreadable.

Serial port programming in Linux

What is your test? Are you shortcutting pin 2 and 3 of your RS232 DB9 connector?

Otherwise read will always return -1 if there are no data to be read.

This is what your code is supposed to do using O_NDELAY flag opening the serial line.

To avoid to read if there are no data to be read, you could use select
In your case:

struct timeval tv;
fd_set rset;
int retV;
int timeout = 5000; // 5 seconds

tv.tv_usec = (timeout * 1000) % 1000000;
tv.tv_sec = timeout / 1000;

FD_ZERO ( &rset );
FD_SET (  fd, &rset );
retV = select ( fd+1, &rset, NULL, NULL, &tv );

if( retV == 0 )
{
   // timeout stuff
}
else if( retV < 0 )
{
    // Error stuff. Read errno to see what happened
}
else
{
   // read data
}

EDIT

Remove fcntl(fd, F_SETFL, FNDELAY);
If you want a normal blocking read, unset that flag.

In your code you are also assuming that read will return all sent chars but is not true. read will return chars that are available to be read. This means that if you send "LINUX" a 5 times read could be requested to read all 5 chars sent.

Last thing

printf("Bytes : %d and data: %s\n", bytes, buffer);

is Undefined Behavior because of buffer is not a NULL terminated string. You could solve it looping on received chars and print it with %c format, or NULL terminating your buffer with:

if (bytes > 0)
   buffer[bytes] = '\0';

or

char stringToSend[] = "LINUX";
size_t len = strlen(stringToSend) +1 ;

write(fd,"LINUX", len);
perror("write");

size_t receivedBytes = 0;
bytes = 0;
while (receivedBytes<len)
{
   bytes = read(fd, &buffer[receivedBytes], sizeof(buffer)-1);
   perror("read");

   if (bytes > 0)
       receivedBytes += bytes;
}

printf("Bytes : %d and data: %s\n", receivedBytes, buffer);

Check if there's incoming data in serial port linux (cbInQue for linux)

Yes, you will need your file descriptor and then use FIONREAD to see if anything is available.

Something like the following should work:

int available;
if( ioctl( fd, FIONREAD, &available ) < 0 ) {
    // Error handling here
}

What is the best way to read and write data to a buffer for my bar code reader?

Once more, the idea is very simple: I am sending a command to a device, and the device interupt me and replays. I read what was send, I work with the replay data, and according to that data, I am sending another command.

You are describing a common and simple arrangement of master & slave devices: the master device sends a command or request message, and the slave device has to reply with a response message.

You are using the proper POSIX method of modifying the flags. However the system calls fcntl(), tcgetattr() and tcsetattr() should have their return codes checked to ensure there were no errors.

You are configuring the serial port for non-canonical (aka raw) input & output. (So your receiving buffer should be unsigned char rather than signed char.) This mode offers a method of reading based on byte count and/or silence on the serial link. There is no reason to use async read involving a signal handler. (Your signal handler has the same bug as this question.) The master/slave device relationship dictates a request/response message exchange.

To properly read the complete response message, you need to define the VMIN and VTIME values. If the response message is 7 to 32 bytes in length, then

termAttr.c_cc[VMIN] = 32;
termAttr.c_cc[VTIME] = 5;

should do the trick. You would be expecting a response message of no more than 32 bytes, or when bytes stop arriving after a half second, then assume the message is complete.

#define CMDLEN  7
unsigned char buffer[32];
unsigned char cmd[CMDLEN];

while (connected == 1) {

    /* construct a new command */

    /* send request */
    nbytes = write(fd, cmd, CMDLEN);
    if (nbytes != CMDLEN) {
        /* problem! */
    }
    /* get response, wait if necessary */
    nbytes = read(fd, buffer, sizeof(buffer));
    if (nbytes < 7) {
        /* problem! */
    }

    /* process response of nbytes */
}

Addendum: response to questions in comments

First, can I leave the filling of the buffer in the signal handler with defined values for VMIN and VTIME?

You have not described any requirement for asynchronous reading of the serial port. (BTW this is not a "serial bus"; USB is a serial bus; EIA/RS-232 is a serial link.) If this is not a master/slave configuration, then please say so. Otherwise a signal handler for reading is an unnecessary complication, and unnecessary complexity is (usually) an indication of poor design. (Exception would be if this code is part of a system that is exclusively event driven.)

Be aware that the serial port is fully buffered for both reading and writing. Your program does not have to have a pending or active read() request when data arrives at the serial port. The kernel stores the serial data in an internal buffer as the data is received. Your program will not lose or miss any data even if it does not use async reading and a signal handler.

The last question is about fcntl(), tcgetattr() and tcsetattr() sistem call. Can you please give me some examples about what error flag I should check?

Such information is documented in the Linux man pages.

If you are using a Linux development system (and you should), then make sure that the man pages are installed. Otherwise you can google for the man page, e.g. "fcntl linux man page"

Check the function prototype.

The returned value is typically the syscall status. If an error is indicated (often by a negative value such as -1), then the global variable errno should be accessed to get more detail.

rc = tcgetattr(fd, &termAttr);
if (rc < 0) {
    printf("failed to get attr: %d, %s", rc, strerror(errno));
    close(fd);
    exit (-2);
}

P.S I am sending an example of the message that I am sending to the device, and an response, so that you can see for what I am talking about.

tx_buffer = 0xFC 0x05 0x11 0x27 0x56 rx_buffer = 0xFC 0x05 0x40 0x27 0x56 0xFC always same 0x05 number of bytes 0x11 command 0x40 response code 0x27 crc_low 0x56 crc_high

example of one command packet and response packet

That is a typical message format for a binary protocol.

As part of the read processing, each received message must be validated before any message processing is performed.

The 0xFC header byte must be checked to ensure that you have message frame synchronization. If the first byte is not equal to 0xFC, then the program has to go into a "hunt for synchonization" mode to find the start of a message (starting with the second byte in this buffer).

The message length should be checked for reasonableness and then used to locate the two CRC bytes, and the message contents can be validated. If the CRC check fails, then the "message" should be ignored and a "hunt for sync" should commence (starting with the second or third byte of this "message").

Addendum 2: response to question on "final code

So the question is now how to create a timer?

Userland can schedule a periodic timer using a POSIX timer and a signal, i.e. timer_create().

But you'll have to check the available clock resolutions to determine if it will let you do 1ms.

But IMO you are going down the wrong path.

You have ignored my advice on not using async I/O.

You are calling read() in a signal handler, which is a potential problem if it tries to sleep.

Your signal handler for async reading has no synchronization or buffer management with the main thread, so the program is at risk of losing data that was read.

You have ignored my advice on checking return codes of syscalls (except for write()s).

If I was writing this program, I would probably use a state machine and there would be only one write() and a read() using the serial port.

Addendum 3: response to 3rd set of questions in comments

Can you please give me a simple example of how can I create a timer, with the timer_create() function?

Apparently you didn't heed my previous advice on using man pages.

There's a code example in the timer_create() man page

About this: “Your signal handler for async reading has no synchronization or buffer management with the main thread, so the program is at risk of losing data that was read.” I think I can solve this If I call clearRXbuffer function after every read() function in the signal handler, so in this case the buffer will only contain the last message data which can be from 7 to 32 bytes. So if new message arrives it will be written in the beginning of the buffer. What do you thing, is this idea good, and am I going in right direction?If not, can you please give me some other buffer management idea.

That is not a reliable solution.

The signal handler executes asynchronously with respect to the main thread.

All shared variables and buffers are therefore critical regions.

Critical regions have to be protected with synchronizing constructs such as mutual exclusion locks (aka mutexes) or semaphores or conditional variables.

Leaving a critical region (like the receive buffer) unprotected might seem to work okay some of the time, but will be unreliable and cause (undetected) loss of data or "strange" program behavior.

You can try various "band-aids" like clearing the buffer, but in the end only proper synchronizing constructs (provided by the OS that execute atomically) will work reliably.

The system maintains a FIFO buffer for the data as it arrives from the serial port, so that no data is lost. Your signal handler counteracts that FIFO by using a single destination buffer that overwrites its buffer everytime the signal handler is executed. The only way this program won't lose any data is to somehow ensure that every write(), read() and CheckRXbuffer() are perfectly coordinated, and the remote device behaves perfectly and responds in a timely manner all the time.

But those are not conditions that a program can depend on when you have asynchronous reads and no synchronizing mechanisms in place!

Tutorials on multithreading usually cover these concepts.

Can I use one write()function, and then another one write() function, because I tried to do that, but it only is written once. Do you know why is that happening?

No.

... am I going in right direction?

Strange that you bother to ask for my opinion on a minor item, when you choose to completely ignore my advice on the major topic of (mis)using async I/O.

Addendum 4: response to 4th set of questions in comments

I want to tell you that in the datasheet of the device is mentioned that I need to use async I/O because ...

Wow, after 2 requests by me (original response & 1st addendum), you finally mention 2 days later why you think you need to use async I/O.

because when I try to power up the device, or reset the device, I should sent N number of commands, and after N number of commands the device response to me.(power up device sequence).

That does not require "async I/O".

Something like this also can happen when I am sending a status to the device, and the device for some reason won’t response to me, in amount period of time, so I should resend the command (async I/O).

That does not require "async I/O" either, nor does that describe async I/O.

because it’s written in the datasheet

Is this datasheet available in English online?

Are you confusing "asynchronous communication" with "asynchronous I/O"?

If this device only sends data in response to a command, and never sends data spontaneously on its own accord (i.e. client/server model), then that means that your program can expect or knows when it can expect messages from the device.

These would be messages that were "solicited" or requested from the device.

The device (from what you have described so far) never sends unsolicited messages.

If this device did send unsolicited messages, then yes, your program might need to have async I/O capability to process these message in a timely manner.

But again, the device (from what you have described so far) only sends solicited messages or no messages when it receives a command.

A device should not be dictating how the host program should be structured. It can only specify the protocol.

---

Related Topics