Binary Data Over Serial Terminal

Binary data over serial terminal

You can use an application like xmodem to transfer file over any terminal. Is the serial port you speak off a terminal, or is it also the kernel console.

If you're kernel is not noisy, then you can use your current connection to make xmodem like transfer. On the host side, you can use kermit, which is nice AND scriptable.

If you want to make your serial port raw, and you have file descriptor ttyfd opened,
here is one way to do it :

struct termios tty, orig_tty;

...

if(tcgetattr(ttyfd, &tty) < 0)
{
    // error checking
}
// backup tty, make it raw and apply changes
orig_tty = tty;
cfmakeraw(&tty);
if(tcsetattr(ttyfd, TCSAFLUSH, &tty) < 0)
{
    // error checking
}

...
//end of program or error path :
tcsetattr(ttyfd, TCSAFLUSH, &orig_tty)

Don't forget to restore the setting at the end of your program if you still want a good behaved terminal.

How to read a binary data over serial terminal in C program?

system("stty erase ^H);
system("stty -F /dev/ttyS0 -icrnl -ixon -ixoff -opost -isig -icanon -echo");    // enter into non-canonical (raw) mode

This would be insufficient code (and poor coding practice per POSIX conventions) to put the serial port into raw or non-canonical mode.

The easiest method in C and Linux is to use the function cfmakeraw() which is available in both the GNU libc and uClibc libraries. Use the man page to obtain the details on the termios structure members that are modified by cfmakeraw().

Beware that cfmakeraw() will setup the serial port in raw mode for a data length of 8 bits and no parity, for a total character frame of 10 bits (assuming one stop bit).

The preferred method is preserving a copy of the termios stucture (for restoration on program exit) and only modifying the required flag bits (rather than writing the complete structure members).

REVISION

Code that works on my ARM SoC is:

#include <stdio.h>
#include <stdlib.h>
#include <sys/un.h>
#include <unistd.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <sys/syslog.h>
#include <termios.h>

#define SERIALPORT_IS_CONSOLE

main()
{
    struct termios  tty;
    struct termios  savetty;
    speed_t     spd;
    unsigned int    sfd;
    unsigned char   buf[80];
    int     reqlen = 79;
    int     rc;
    int     rdlen;
    int     pau = 0;

#ifdef SERIALPORT_IS_CONSOLE
    sfd = STDIN_FILENO;
#else
    sfd = open("/dev/ttyS1", O_RDWR | O_NOCTTY);
#endif
    if (sfd < 0) {
        syslog(LOG_DEBUG, "failed to open: %d, %s", sfd, strerror(errno));
        exit (-1);
    }
    syslog(LOG_DEBUG, "opened sfd=%d for reading", sfd);

    rc = tcgetattr(sfd, &tty);
    if (rc < 0) {
        syslog(LOG_DEBUG, "failed to get attr: %d, %s", rc, strerror(errno));
        exit (-2);
    }
    savetty = tty;    /* preserve original settings for restoration */

    spd = B115200;
    cfsetospeed(&tty, (speed_t)spd);
    cfsetispeed(&tty, (speed_t)spd);

    cfmakeraw(&tty);

    tty.c_cc[VMIN] = 1;
    tty.c_cc[VTIME] = 10;

    tty.c_cflag &= ~CSTOPB;
    tty.c_cflag &= ~CRTSCTS;    /* no HW flow control? */
    tty.c_cflag |= CLOCAL | CREAD;
    rc = tcsetattr(sfd, TCSANOW, &tty);
    if (rc < 0) {
        syslog(LOG_DEBUG, "failed to set attr: %d, %s", rc, strerror(errno));
        exit (-3);
    }

    do {
        unsigned char   *p = buf;
    
        rdlen = read(sfd, buf, reqlen);
        if (rdlen > 0) {
            if (*p == '\r')
                pau = 1;
            syslog(LOG_DEBUG, "read: %d, 0x%x 0x%x 0x%x", \
                     rdlen, *p, *(p + 1), *(p + 2));
        } else {
            syslog(LOG_DEBUG, "failed to read: %d, %s", rdlen, strerror(errno));
        }
    } while (!pau);

    tcsetattr(sfd, TCSANOW, &savetty);
    close(sfd);
    exit (0);
}

The compiled program is loaded & executed on the target board.

From the host side of the serial comm link, the file seq.bin with the following contents is sent:

$ od -t x1 seq.bin
0000000 aa 02 fe
0000003

Then "ABC" is typed on the host (which is running the minicom terminal emulator program), followed by a carriage return.
The program terminates on the target, and the syslog is then examined:

# tail /var/log/messages                                                        
Sep xx xx:xx:42 atmel_soc user.info kernel: EXT3 FS on nvsram, internal journal 
Sep xx xx:xx:42 atmel_soc user.info kernel: EXT3-fs: mounted filesystem with or.
Sep xx xx:xx:42 atmel_soc user.info kernel: kjournald starting.  Commit intervas
Sep xx xx:xx:18 atmel_soc auth.info login[431]: root login on 'ttyS0'           
Sep xx xx:xx:04 atmel_soc user.debug syslog: opened sfd=0 for reading           
Sep xx xx:xx:14 atmel_soc user.debug syslog: read: 3, 0xaa 0x2 0xfe             
Sep xx xx:xx:50 atmel_soc user.debug syslog: read: 1, 0x41 0x2 0xfe              
Sep xx xx:xx:51 atmel_soc user.debug syslog: read: 1, 0x42 0x2 0xfe              
Sep xx xx:xx:51 atmel_soc user.debug syslog: read: 1, 0x43 0x2 0xfe              
Sep xx xx:xx:52 atmel_soc user.debug syslog: read: 1, 0xd 0x2 0xfe              
# 

The binary data has been received intact.

Note that since this is raw mode and the typed chars were entered relatively slowly, the read() returns "partial" data and the user program would be responsible for buffering/assembling the data into complete "messages". If the messages are of fixed length, then c_cc[VMIN]member could be set to the message length. But beware of message framing issues and complications when frame sync is lost!

How to send raw binary data over serial in C without non-native libraries in linux

It seems you have the serial port opening more or less in hand. I prefer to set the termios member components explicitly myself, but cfmakeraw() is perfectly fine too.

What you should consider, is having a separate function to send one or more of those structures at a time. For example,

int write_all(const int fd, const void *buf, const size_t len)
{
    const char *data = buf;
    size_t      written = 0;
    ssize_t     n;

    while (written < len) {
        n = write(fd, data + written, len - written);
        if (n > 0) {
            written += n;
        } else
        if (n != -1) {
            /* C library bug, should never occur */
            errno = EIO;
            return -1;
        } else {
            /* Error; n == -1, so errno is already set. */
            return -1;
        }
    }

    /* Success. */
    return 0;
}

The function will return 0 if all data was successfully written, and -1 with errno set if an error occurs.

To send a struct packetData pkt; just use write_all(fd, &pkt, sizeof pkt).

To send a full array struct packetData pkts[5]; use write_all(fd, pkts, sizeof pkts).
To send n packets starting at pkts[i], use write_all(fd, pkts + i, n * sizeof pkts[0]).

However, you do not want to use tcflush(). It does not do what you think it does; it actually just discards data.

Instead, to ensure that the data you have written has been transmitted, you need to use tcdrain(fd).

I recommend against adding tcdrain(fd) at the end of write_all() function, because it blocks, pauses the program, until the data has been transmitted. This means that you should only use tcdrain() before you do something that requires the other end has received the transmission; for example before trying to read the response.

However, if this is a query-response interface, and you do intend to also read from the serial device, you should set tty.c_cc[VMIN] and tty.c_cc[VTIME] to reflect how you intend to use the interface. I prefer asynchronous full-duplex operation, but that requires select()/poll() handling. For half-duplex, with these exact structures only, you can use tty.c_cc[VMIN] = sizeof (struct packetData) with say tty.c_cc[VTIME] = 30, which causes read() to try and wait until a full structure is available, but at most 30 deciseconds (3.0 seconds). Something like tty.c_cc[VMIN] = 1; tty.c_cc[VTIME] = 1; is more common; that causes read() to return a short count (even 0!) if there is no additional data received within a decisecond (0.1 seconds). Then, the receive function could be along the following lines:

int read_all(const int fd, void *buf, const size_t len)
{
    char *const ptr = buf;
    size_t      have = 0;
    ssize_t     n;

    /* This function is to be used with half-duplex query-response protocol,
       so make sure we have transmitted everything before trying to
       receive a response. Also assumes c_cc[VTIME] is properly set for
       both the first byte of the response, and interbyte response interval
       in deciseconds. */
    tcdrain(fd);

    while (have < len) {
        n = read(fd, ptr + have, len - have);
        if (n > 0) {
            have += n;
        } else
        if (n == 0) {
            /* Timeout or disconnect */
            errno = ETIMEDOUT;
            return -1;
        } else
        if (n != -1) {
            /* C library bug, should never occur */
            errno = EIO;
            return -1;
        } else {
            /* Read error; errno set by read(). */
            return -1;
        }
    }
    /* Success; no errors. */
    return 0;
}

If this returns -1 with errno == ETIMEDOUT, the other side took too long to answer. There may be remainder of the late response in the buffer, which you can discard with tcflush(TCIFLUSH) (or with tcflush(TCIOFLUSH), which also discards any written data not yet transmitted). Synchronization in this case is a bit difficult, because the above read_all() function doesn't return how many bytes it received (and therefore how many bytes to discard of a partial structure).

Sometimes the interface used always returns the number of bytes, but also sets errno (to 0 if no error occurred, and a nonzero error constant otherwise). That would be better for a query-response interface read and write functions, but many programmers find this use case "odd", even though it is perfectly okay by POSIX.1 standard (which is the relevant standard here).

Reading and writing binary data over serial port

file = fopen( "zname.jpg", "wb" );
while (1) { 
  n = read(readport, &buff, 1);
  if (n == -1) switch(errno) {
         case EAGAIN: /* sleep() */ 
            continue;
          ...
         default: goto quit;
         }
  if (n ==0) break;
  fputc(buff, file);
  }
quit:
   fclose (file);
...

Even better than sleep() and loop, would be to use select/poll. (You'd still have to check for EAGAIN)

How can I retrieve data through a serial terminal without corrupting its fundamental value in GNU ARM embedded?

Casting will not solve the issue and perhaps you have illustrated why casting is bad.

---

You need a function like something like this,

/* four 'nibbles' (nibble = 4 bits = one hex) is 16 bits. */
void int16ToString(char *buf[4], uint16_t val)
{
   static const char *lookup = "0123456789ABCDEF";
   buf[3] = lookup[val & 0xf]; /* write backwards or you need a copy of val */
   val = val >> 4;
   buf[2] = lookup[val & 0xf];
   val = val >> 4;
   buf[1] = lookup[val & 0xf];
   val = val >> 4;
   buf[0] = lookup[val & 0xf];
}

There are various functions in the C library, etc which do similar things. Obviously the routine needs a buffer of 4 bytes and it is not null terminated by the routine. It is just to demonstrate what needs to happen. sprintf, snprint, etc will be useful to look at as well as itoa, etc.

---

Why do you need a function like that? Because there are two sides to the conversation and they can (and probably do interpret things differently). Generally you can transfer the lower ascii characters without effect. For example 'XON' is a flow control character that can be embedded in a data stream.

If you are to transmit the binary data via the CLI you need to 'clean it' and then you need code on the other side of the CLI uart to undo that. You can just send hex and then convert back to binary. There are more efficient ways to do this such as zmodem and base64 if the data is large or the link (CLI uart) is not reliable. A simple itoa() with a hexdump conversion on another host will transfer the binary which is suitable for a debugging transfer.

How to transfer a binary file through serial?

There is an option in Teraterm under File->Send file... that can be used to send binary file over.

If you have a binary file just check the box "binary" in the option section of the sendFile Window before pressing "Open" with your specific file selected.

Some information about sendfile here: https://ttssh2.osdn.jp/manual/en/macro/command/sendfile.html

Bash script to read N bytes of binary data from serial device and save them to file

Thanks to the comment of @meuh, I was able to write a working script using dd:

dd ibs=1 count=$PLBYTE iflag=count_bytes if=/dev/ttyS0 of=/.../dump.bin

using dd operands count and iflag, (counting the received bytes and reading 1 byte/block) with $PLBYTE the number of expected bytes.

The script now works as expected.

Make sure to set stty in noncanonical mode (-icanon) otherwise data over 4096 bytes will be truncated and dd will not receive the expected amount of bytes.

Writing binary data over serial in windows

Alright so I figured it out, rather, a co-worker did. On the linux side, in the file /etc/inittab I had to comment out the line:

T0:23:respawn:/sbin/getty -L ttymxc0 115200 vt100

This was grabbing the serial port in a way that made it unusable for receiving bytes. I now see the expected output.

---

Related Topics