Linux C++ Socket Select Loop

linux C++ socket select loop

The select() function uses the specified file descriptor mask to determine which file descriptors to monitor for an event (read, write, etc.). When a file descriptor is available for an I/O activity (read, write) the select() function modifies the descriptors to indicate which of the files are ready for the given I/O action.

See this article on the select function and the macros/functions used with the file descriptors.

Old style Unix type programs often treated the file descriptor as a bit mask and just checked the bits. However the actual implementation of the file descriptor can vary by compiler so it is best to use the standard file descriptor macros/functions to set, clear, and test the various file descriptors.

So when using the select() function you need to use FD_ZERO() and FD_SET() so that you will set the specific file descriptors that you want for this particular call to the select() function. When select() returns, it will indicate which of the file descriptors designated are actually ready to be used for the I/O action (read, write, etc.).

So your code will actually be something like:

while(true)
{
    fd_set rfds;

    FD_ZERO(&rfds);
    FD_SET(socketHandle, &rfds);
    recVal = select(socketHandle + 1, &rfds, NULL, NULL, &tv);
    switch(recVal)
    {
        case(0):
        {
            //Timeout
            break;
        }
        case(-1):
        {
            //Error
             break;
        }
        default:
        {
            /*Packet Data Type*/ pkt;
            if(recvfrom(socketHandle, &pkt, sizeof(/*Packet Data Type*/), 0, (sockaddr*)&serverAddr, (socklen_t*)&sockLen) < 0)
            {
                //Failed to Recieve Data
                break;
            }
            else
            {
                //Recieved Data!!
            }
            break;
        }
    }

However what you really should do is to use the FD_ISSET() function to check which particular file descriptors are ready for use. In your case you have only the one but in a situation where there were multiple descriptors you would want to use the FD_ISSET() function as well.

How to use select() on sockets properly?

Read several times syscalls(2), select(2), select_tut(2), poll(2), errno(3) (BTW, you should prefer poll(2) to the obsolete select, which don't handle file descriptors bigger than __FD_SETSIZE, i.e. 1024 on my Linux/Debian/x86-64 system).

Then:

fd_set read_fds;
FD_ZERO(&read_fds);
int fdmax = listenfd;
FD_SET(listenfd, &read_fds);

should go inside the while(1) loop, before calling select. BTW, I recommend using poll instead of select

Using select system call in a loop and multiple operations

You need to reinitialise "timeout" after each call of select. From the select man page; "On Linux, select() modifies timeout to reflect the amount of time not slept". So in your case, after the first select call, your timeout values are all 0. Hence subsequent calls to select will timeout immediately.

Using select() then read() vs a blocking read() loop

You are correct that a thread does not consume CPU while blocked on I/O.

The select() syscall is a viable alternative, but its main use case is multiplexing I/O from / to multiple channels. In that case, it's not viable for one thread serve all the channels via blocking reads (or writes) because it would frequently be blocked waiting for data from one channel while a different one had data ready and waiting. On the other hand, you don't want to perform non-blocking reads in a loop, as that will waste CPU.

As for whether to prefer blocking on read() or blocking on select() (or poll()) for managing a single channel, that's a bit of an apples-to-oranges comparison, but if those are the choices then it's more straightforward to block on read(). That's what I would do. The apples-to-apples comparison would be more along the lines of multiple threads or child processes, each blocking on read()ing a different channel, vs. one thread of one process managing multiple channels with the help of select().

Understanding a Select loop for receiving UDP messages and opening TCP sockets

You need to call FD_ZERO() on each loop iteration before calling FD_SET() and select(). Also, don't add a TCP socket to the fd_set until after you attempt to connect it first. Keep a list of the TCP sockets you create, so you can re-add them to the fd_set on each loop iteration.

Try something like this:

udpsock = initudp(port); //setup udp socket

std::vector<SOCKET> tcpsocks;
SOCKET tcpsock;

while(1)
{
    FD_ZERO(&rdsocks);
    FD_SET(udpsock, &rdsocks);
    max = udpsock;

    for(size_t i = 0; i < tcpsocks.size(); ++i)
    {
        tcpsock = tcpsocks[i];
        FD_SET(tcpsock, &rdsocks);

        if( tcpsock > udpsock )
            max = tcpsock;
    }

    if( select(max+1, &rdsocks, NULL, NULL, NULL) == SOCKET_ERROR )
    {
        perror("Select error");
        WSACleanup();
        return 1;
    }

    for(size_t i = 0; i < tcpsocks.size(); ++i) // loop on TCP sockets
    {
        tcpsock = tcpsocks[i];            
        if( FD_ISSET(tcpsock, &rdsocks) )
        {
            printf("Socket %d is ready \n", tcpsock);
            // process tcp message
        }
    }

    if( FD_ISSET(udpsock, &rdsocks) )
    {
        int ret = 0;
        res = recvfrom(udpsock, buff, sizeof(buff), 0, (struct sockaddr*)&udpclient, lenaddr);
        // process udp message...
        if( setup tcp connection is requested )
        {
            tcpsock = ...;
            if( tcpsock != INVALID_SOCKET )
                tcpsocks.push_back(tcpsock);
        }
    }
}

Select() issue in C

The reason you are stuck in infinite loop is because you are not accepting new connections from the clients in your infinite loop & the file/socket descriptors that are in your structure array are not valid any more because those clients have terminated. You have chosen a very poor mechanism of using select.

You should do the like this:

while(1)
{
    FD_ZERO(&readfds);

    //add master socket to set
    FD_SET(master_socket, &readfds);
    max_sd = master_socket;

    //add child sockets to set
    for ( i = 0 ; i < max_clients ; i++) 
    {
        //socket descriptor
        sd = client_socket[i];

        //if valid socket descriptor then add to read list
        if(sd > 0)
            FD_SET( sd , &readfds);

        //highest file descriptor number, need it for the select function
        if(sd > max_sd)
            max_sd = sd;
    }

    //wait for an activity on one of the sockets , timeout is NULL , so wait indefinitely
    activity = select( max_sd + 1 , &readfds , NULL , NULL , NULL);

    if ((activity < 0) && (errno!=EINTR)) 
    {
        printf("select error");
    }

    //If something happened on the master socket , then its an incoming connection
    if (FD_ISSET(master_socket, &readfds)) 
    {
        if ((new_socket = accept(master_socket, (struct sockaddr *)&address, (socklen_t*)&addrlen))<0)
        {
            perror("accept");
            exit(EXIT_FAILURE);
        }

        //inform user of socket number - used in send and receive commands
        printf("New connection , socket fd is %d , ip is : %s , port : %d \n" , new_socket , inet_ntoa(address.sin_addr) , ntohs(address.sin_port));

        //send new connection greeting message
        if( send(new_socket, message, strlen(message), 0) != strlen(message) ) 
        {
            perror("send");
        }

        puts("Welcome message sent successfully");

        //add new socket to array of sockets
        for (i = 0; i < max_clients; i++) 
        {
            //if position is empty
            if( client_socket[i] == 0 )
            {
                client_socket[i] = new_socket;
                printf("Adding to list of sockets as %d\n" , i);

                break;
            }
        }
    }

    //else its some IO operation on some other socket :)
    for (i = 0; i < max_clients; i++) 
    {
        sd = client_socket[i];

        if (FD_ISSET( sd , &readfds)) 
        {
            //Check if it was for closing , and also read the incoming message
            if ((valread = read( sd , buffer, 1024)) == 0)
            {
                //Somebody disconnected , get his details and print
                getpeername(sd , (struct sockaddr*)&address , (socklen_t*)&addrlen);
                printf("Host disconnected , ip %s , port %d \n" , inet_ntoa(address.sin_addr) , ntohs(address.sin_port));

                //Close the socket and mark as 0 in list for reuse
                close( sd );
                client_socket[i] = 0;
            }

            //Echo back the message that came in
            else
            {
                //set the string terminating NULL byte on the end of the data read
                buffer[valread] = '\0';
                send(sd , buffer , strlen(buffer) , 0 );
            }
        }
    }
}

return 0;
}

Select() to read in sockets

A few comments that are too long to fit in the comments...

The first parameter to select really only needs to be sock+1.

By passing NULL for the timeout, select will block indefinitely (so you might as well have just called read).

When select does tell you that sock is ready for reading, there may only be one byte present, even if the other end wrote more then that. You will have to loop, reading the socket until you get the number of bytes you want. You will have to decide if you want the timeout only while waiting for the first byte, or if you want to timeout in the middle (I.e. is the select inside or outside the loop).

Normally, when using select, you only want to block in select, and not in read, so the socket should be put in non-blocking mode, and you should be prepared for the read to fail with EWOULDBLOCK.

If you do time out, do you want to close the connection? If you got half way through a message, you can't just throw away the half and keep going because you would now be expecting the next byte to be another start of message when it will now be a middle.

send and recive data using select() function

For starters two plus more issues.

This

ready = select(socket_dect,&readfds,NULL,NULL,&tv);

should be

ready = select(socket_dect + 1, &readfds, NULL, NULL, &tv);

From man select:

nfds is the highest-numbered file descriptor in any of the three sets, plus 1.

Instead of

if(ready == STDIN_FILENO)

use

if (FD_ISET(STDIN_FILENO, &readfds)

select() modifies the parameters passed, so you need to re-initialise them every time before calling select(), that is inside the loop.

Also you must test all system calls for their outcome. This includes connect().

Also^2 you may not assume recv() returns as many bytes as it was told to (len here). It may return having received few. To make sure you got as much as you wanted loop around recv() counting.

Confusion about select()

I'm busy with this for 2 days now and still don't understand it.

It's no wonder that you don't understand the code: The code in the example is nonsense.

Checking the ListenSocket should be done outside the for loop. And FD_ISSET must also be checked for the connections accepted using accept.

The correct code inside the while loop would look like this:

fd_set copy = master;

select(FD_SETSIZE, ©, NULL, NULL, NULL);

// If new connection
if (FD_ISSET(ListenSocket, ©)) 
{
    ...
}

for (int i = 0; i < FD_SETSIZE; i++)
{    
    // If an existing connection has data
    // or the connection has been closed
    if ((i != ListenSocket) && FD_ISSET(i, ©)) 
    {
        nBytes = recv(i, buffer, maxBytes, 0);
        // Connection dropped
        if(nBytes < 1)
        {
            close(i); // other OSs (Linux, MacOS ...)
            // closesocket(i); // Windows
            FD_CLR(i, &master);
        }
        // Data received
        else
        {
            ...
        }
    }
}

I know that if there is an incoming connection that can be accepted, the copy.fd_array[] will contain ListenSocket but when the while loop repeats it's still there.

What does fd_set copy contain after the select() call?

First of all: Before calling select(), copy.fd_array[] must contain all socket handles that you are interested in. This means it must contain ListenSocket and all handles returned by accept().

master.fd_array[] contains all these handles, so fd_set copy = master; will ensure that copy.fd_array[] also contains all these handles.

select() (with NULL as last argument) will wait until at least one socket becomes "available". This means that it will wait until at least one of the following conditions is true:

A connection accepted using accept() is closed by the other side
a connection accepted using accept() has data that can be received
there is a new connection that can be accepted using accept(ListenSocket...)

As soon as one condition is fulfilled, select() removes all other handles from copy.fd_array[]:

ListenSocket is removed from copy.fd_array[] if there is no incoming connection
A handle returned by accept() is removed from that array if the connection has neither been closed nor new data is available

If two events happen the same time, copy.fd_array[] will contain more than one handle.

You use FD_ISSET() to check if some handle is still in the array.

So how do we know if a client is disconnected?

When you detect FD_ISSET(i, ©) for a value i that has been returned by accept(), you must call recv() (under Linux read() would also work):

If recv() returns 0 (or negative in the case of errors), the other computer has dropped the connection. You must call close() (closesocket() on Windows) and remove the handle from copy.fd_array[] (this means: you must remove it from master.fd_array[] because of the line fd_set copy = master;).

If recv() returns a positive value, this is the number of bytes that have been received.

Linux C++ Socket Select Loop