Behavior of Writing to Tcp Socket Before Reading All Data

Why is writing a closed TCP socket worse than reading one?

+1 To Greg Hewgill for leading my thought process in the correct direction to find the answer.

The real reason for SIGPIPE in both sockets and pipes is the filter idiom / pattern which applies to typical I/O in Unix systems.

Starting with pipes. Filter programs like grep typically write to STDOUT and read from STDIN, which may be redirected by the shell to a pipe. For example:

cat someVeryBigFile | grep foo | doSomeThingErrorProne

The shell when it forks and then exec's these programs probably uses the dup2 system call to redirect STDIN, STDOUT and STDERR to the appropriate pipes.

Since the filter program grep doesn't know and has no way of knowing that it's output has been redirected then the only way to tell it to stop writing to a broken pipe if doSomeThingErrorProne crashes is with a signal since return values of writes to STDOUT are rarely if ever checked.

The analog with sockets would be the inetd server taking the place of the shell.

As an example I assume you could turn grep into a network service which operates over TCP sockets. For example with inetd if you want to have a grep server on TCP port 8000 then add this to /etc/services:

grep     8000/tcp   # grep server

Then add this to /etc/inetd.conf:

grep  stream tcp nowait root /usr/bin/grep grep foo

Send SIGHUP to inetd and connect to port 8000 with telnet. This should cause inetd to fork, dup the socket onto STDIN, STDOUT and STDERR and then exec grep with foo as an argument. If you start typing lines into telnet grep will echo those lines which contain foo.

Now replace telnet with a program named ticker that for instance writes a stream of real time stock quotes to STDOUT and gets commands on STDIN. Someone telnets to port 8000 and types "start java" to get quotes for Sun Microsystems. Then they get up and go to lunch. telnet inexplicably crashes. If there was no SIGPIPE to send then ticker would keep sending quotes forever, never knowing that the process on the other end had crashed, and needlessly wasting system resources.

C socket - consecutive read/write, what happens?

Yes, this is possible. reads will block (by default) until data are available to read on the socket. There's no need to try to arrange for reads and writes in a particular order, your program will just wait until there is something to read before the read() call returns.

Your code will result in this:

Server blocks in read()
Client write()s
Client blocks in read()
Server receives data, read() returns
Server write()s three times
Client receives data, read(1) returns
Client's read(2) is called and returns as soon as data arrive
Client's read(3) is called and returns as soon as data arrive

On a really fast link it's possible that the server write()s and the client read()s happen at roughly the same time, maybe even interleaved, but the server write()s will always be in order and the client's read()s will always be in order.

Data ordering is preserved if the sockets are SOCK_STREAM, eg. TCP or UNIX sockets like you're asking about.

So read(2) will always return the data written in write(2) for a TCP socket.

If you used a UDP socket (SOCK_DGRAM's default) you might find that the messages are received out-of-order or not at all.

Can I send data via TCP before closing Socket?

Your code is working just fine - you are just blocking 3 times on the readLine method and then printing your responses (at this time socket closes on server side).

try this:

        String message1 = br.readLine();
        System.out.println(message1);
        String message2 = br.readLine();
        System.out.println(message2);
        String message3 = br.readLine();
        System.out.println(message3);

Another tip from me is to do the clean up in finally block.

try { ... }
catch { ... }
finally {
    s.close();
}

This way resources are cleaned up even if exception is thrown.

what happens when I write data to a blocking socket, faster than the other side reads?

For a blocking socket, the send() call will block until all the data has been copied into the networking stack's buffer for that connection. It does not have to be received by the other side. The size of this buffer is implementation dependent.

Data is cleared from the buffer when the remote side acknowledges it. This is an OS thing and is not dependent upon the remote application actually reading the data. The size of this buffer is also implementation dependent.

When the remote buffer is full, it tells your local stack to stop sending. When data is cleared from the remote buffer (by being read by the remote application) then the remote system will inform the local system to send more data.

In both cases, small systems (like embedded systems) may have buffers of a few KB or smaller and modern servers may have buffers of a few MB or larger.

Once space is available in the local buffer, more data from your send() call will be copied. Once all of that data has been copied, your call will return.

You won't get a "connection reset" error (from the OS -- libraries may do anything) unless the connection actually does get reset.

So... It really doesn't matter how quickly the remote application is reading data until you've sent as much data as both local & remote buffer sizes combined. After that, you'll only be able to send() as quickly as the remote side will recv().

In TCP socket program,client send some data, but server need read multiple times. why?

The reason you don't receive everything that is available in one read is because you only read 8 bytes at a time.

char* recv_buffer = (char*)malloc(1024+1);
int ret_read = recv(return_events[index].data.fd, recv_buffer, sizeof(recv_buffer), 0);
            // part_3-3-1:read return error

recv_buffer is a char* not an array, so sizeof recv_buffer equals the size of a pointer which in your case is 8.

Note that you should never rely on data arriving in packages. If your message protocol states that you should be getting 10 bytes never expect all 10 bytes to be available at once. You should always code in a way that can handle data being split up into multiple reads.

If the thread handles a single socket then a simple do { read... } while (total_bytes_received < expected_bytes); will suffice.

If the thread handles multiple connections, then you need to save the bytes you have read and then continue to manage other sockets that are ready before returning to your handling loop that will use select/epoll to wait for more data.

What is the correct way of reading from a TCP socket in C/C++?

Without knowing your full application it is hard to say what the best way to approach the problem is, but a common technique is to use a header which starts with a fixed length field, which denotes the length of the rest of your message.

Assume that your header consist only of a 4 byte integer which denotes the length of the rest of your message. Then simply do the following.

// This assumes buffer is at least x bytes long,
// and that the socket is blocking.
void ReadXBytes(int socket, unsigned int x, void* buffer)
{
    int bytesRead = 0;
    int result;
    while (bytesRead < x)
    {
        result = read(socket, buffer + bytesRead, x - bytesRead);
        if (result < 1 )
        {
            // Throw your error.
        }

        bytesRead += result;
    }
}

Then later in the code

unsigned int length = 0;
char* buffer = 0;
// we assume that sizeof(length) will return 4 here.
ReadXBytes(socketFileDescriptor, sizeof(length), (void*)(&length));
buffer = new char[length];
ReadXBytes(socketFileDescriptor, length, (void*)buffer);

// Then process the data as needed.

delete [] buffer;

This makes a few assumptions:

ints are the same size on the sender and receiver.
Endianess is the same on both the sender and receiver.
You have control of the protocol on both sides
When you send a message you can calculate the length up front.

Since it is common to want to explicitly know the size of the integer you are sending across the network define them in a header file and use them explicitly such as:

// These typedefs will vary across different platforms
// such as linux, win32, OS/X etc, but the idea
// is that a Int8 is always 8 bits, and a UInt32 is always
// 32 bits regardless of the platform you are on.
// These vary from compiler to compiler, so you have to 
// look them up in the compiler documentation.
typedef char Int8;
typedef short int Int16;
typedef int Int32;

typedef unsigned char UInt8;
typedef unsigned short int UInt16;
typedef unsigned int UInt32;

This would change the above to:

UInt32 length = 0;
char* buffer = 0;

ReadXBytes(socketFileDescriptor, sizeof(length), (void*)(&length));
buffer = new char[length];
ReadXBytes(socketFileDescriptor, length, (void*)buffer);

// process

delete [] buffer;

I hope this helps.