Data Transfer Over Sockets[Tcp] How to Pack Multiple Integer in C/C++ and Transfer the Data with Send() Recv()

Using multipe recv() and send() calls in c++

TCP is a stream protocol. That means that the sender, the reciever or any element involved in the communication can split or re-assemble packets for any reason.

Because of that, the rules are:

on the sender part, you can either send the bytes with many send calls or assemble them in a buffer before sending them. The main difference is that as send is a true system call it adds some overhead for a context change. That is the reason why the C and C++ libraries have provision for buffered IO
on the receiver part, you cannot rely on the initial packet sizes to be unchanged but must read bytes in chunks and assemble them to re-build your structures if it matters.

recv() reading several times when send() sends only once in TCP C sockets. Can I synchronize the socket communication?

However, my problem is during the duration of the program, the recv() is being called multiple times when only one send() has been called

This is how TCP works, so you have to change code so it will handle this situation. It is slightly more complicated though, for example there is no guarantee that send() and fwrite() would handle the whole buffer. For fwrite() though that would be an error condition, for socket send is normal and expected and your code must handle that.

Or is my implementation completely wrong?

Not completely, you just need small change:

while ((read_size = recv(new_socket, buffer, TCP_BUFFER_SIZE, 0)) > 0)
{
    /* Write to binary output file */
    fwrite(buffer, read_size, 1, out_file);
}

Remember TCP socket is stream oriented - it guarantees that you will receive data without duplicate and in order, it does not guarantee that you will receive it the same packets as you send. There are actually no packets in TCP, just stream.

Note: you fread code may have similar issue:

while (fread(buffer, 1, TCP_BUFFER_SIZE, in_file) > 0)
{
    send_socket_data(socket_desc, buffer, TCP_BUFFER_SIZE);
}

may not read the whole buffer, for example if you reach the end of file (unless you file is always precisely size that devided by TCP_BUFFER_SIZE without reminder). So you need to keep size of read and use it to send data.

Multiple calls to send() are merged into one call to recv()

TCP is a streaming protocol. It is not aware at all of any kind of "message" boundaries. It does not add such information dependend on single calls to send().

Due to those facts any number of send()s on the sender side can lead to any number of recv()s (up to the number of bytes sent) on the receiver side.

To get around this behaviour define and implement an application level protocol to distinguish the different "messages" that had been sent.

One cannot rely on recv()/send() receiving/sending as much bytes as those two functions were told to receive/send. It is an essential necessity to check their return value to learn how much bytes those functions actually received/sent and loop around them until all data that was intended to be received/sent had been received/sent.

sending multiple send/recv in socket

TCP is a byte stream based protocol, it knows nothing about messages. You send 25 bytes in total, and each recv on the other side will read some of those bytes. You can get 20, you might get 1 and then 19 in the next read, you might get 5 then 4 then 11 then 5. The size parameter to recv is the maximum number to read.

You need to loop until you read the whole message yourself, and also understand you might receive more than one "send" in the same received message.

send and receive binary files properly using sockets c++

A rather general way to achieve this (in both C and C++) is something like this:

if (FILE *fp = fopen(filename, "rb"))
{
    size_t readBytes;
    char buffer[4096];
    while ((readBytes = fread(buffer, 1, sizeof(buffer), fp) > 0)
    {
        if (send(Connections[conindex], buffer, readBytes, 0) != readBytes)
        {
            handleErrors();
            break;
        }
    }
    close(Connections[conindex]);
}

And on the client side:

if (FILE *fp = fopen(filename, "wb"))
{
    size_t readBytes;
    char buffer[4096];
    while ((readBytes = recv(socket, buffer, sizeof(buffer), 0) > 0)
    {
        if (fwrite(buffer, 1, readBytes, fp) != readBytes)
        {
            handleErrors();
            break;
        }
    }
}

Alternatives to this rather FTP-esque connection style includes sending the size of the file first, so the client will know when to stop listening instead of waiting for the disconnect.

Please note that this code is untested and merely meant to illustrate the concept.

Efficient way to transfer data between 2 clients over TCP with intermediate server

If you want to do several transfers at once using sockets you have two options:

Blocking sockets and threads

This is the way you've written your server. The problem with threads is that they can lead to bugs that are rather hard to debug. Combine that with network bugs that in themselves can be rather hard to debug and you have a potential nightmarish debugging session on your hands.

Non-blocking sockets and select()

This way doesn't need threads, instead it uses select() to see which sockets have data waiting to be read. Combine that with some loops and you can transfer several files concurrently. Setting sockets to be non-blocking is easy, using correct can be slightly trickier but weighted against the potential for thread + network bugs this is the way I personally prefer to write network code.

Regarding your actual problem; I would suggest something like this:

Client A connects to server S. You need to bind the local side for the next step.
A also opens another socket for data transfer on the next port upwards.
S sends file list to A. How you build up the file list I leave to you.
A requests file F from S.
S checks which client has F.
S sends "send F to A on port X" to B. You can check which remote port is used, and then you know which port to send the file on.
B recieves and executes the command.

Sending multiple files through a TCP socket

TCP is a streaming protocol with no concept of message boundaries, so if you print msgp you will see it received more than you expected, probably folder name, number of files, and part of the binary file data. Since that data isn't UTF-8 encoded, you get a UnicodeDecodeError.

You have to define a protocol and buffer data from the socket until it satisfies the protocol (read to a newline character, for example). Also see socket.makefile which wraps a socket in a file-like object so you can treat it more like a file. Methods like .readline() and .read(n) exist, so you could define a protocol like:

Send Folder Name + newline
Send number of files + newline
Send filename #1 + newline
send file size + newline
send binary data of exactly “file size” bytes.
Repeat 3-5 for remaining files.

Example implementing the above protocol (no error handling if a client breaks the protocol). Prepare a folder or two to send, then start the server, in another terminal, run client.py <folder> to transmit <folder> to a Downloads folder.

server.py

import socket
import os

s = socket.socket()
s.bind(('', 8000))
s.listen()

while True:
    client, address = s.accept()
    print(f'{address} connected')

    # client socket and makefile wrapper will be closed when with exits.
    with client, client.makefile('rb') as clientfile:
        while True:
            folder = clientfile.readline()
            if not folder:  # When client closes connection folder == b''
                break
            folder = folder.strip().decode()
            no_files = int(clientfile.readline())
            print(f'Receiving folder: {folder} ({no_files} files)')
            # put in different directory in case server/client on same system
            folderpath = os.path.join('Downloads', folder)
            os.makedirs(folderpath, exist_ok=True)
            for i in range(no_files):
                filename = clientfile.readline().strip().decode()
                filesize = int(clientfile.readline())
                data = clientfile.read(filesize)
                print(f'Receiving file: {filename} ({filesize} bytes)')
                with open(os.path.join(folderpath, filename), 'wb') as f:
                    f.write(data)

client.py

import socket
import sys
import os

def send_string(sock, string):
    sock.sendall(string.encode() + b'\n')

def send_int(sock, integer):
    sock.sendall(str(integer).encode() + b'\n')

def transmit(sock, folder):
    print(f'Sending folder: {folder}')
    send_string(sock, folder)
    files = os.listdir(folder)
    send_int(sock, len(files))
    for file in files:
        path = os.path.join(folder, file)
        filesize = os.path.getsize(path)
        print(f'Sending file: {file} ({filesize} bytes)')
        send_string(sock, file)
        send_int(sock, filesize)
        with open(path, 'rb') as f:
            sock.sendall(f.read())

s = socket.socket()
s.connect(('localhost', 8000))
with s:
    transmit(s, sys.argv[1])

I prepared two folders then ran "client Folder1" and "client Folder2". Client terminal output:

C:\test>client Folder1
Sending folder: Folder1
Sending file: file1 (13 bytes)
Sending file: file2 (13 bytes)
Sending file: file3 (13 bytes)
Sending file: file4 (13 bytes)

C:\test>client Folder2
Sending folder: Folder2
Sending file: file5 (13 bytes)
Sending file: file6 (13 bytes)

Output (server.py):

C:\test>server
('127.0.0.1', 2303) connected
Receiving folder: Folder1 (4 files)
Receiving file: file1 (13 bytes)
Receiving file: file2 (13 bytes)
Receiving file: file3 (13 bytes)
Receiving file: file4 (13 bytes)
('127.0.0.1', 2413) connected
Receiving folder: Folder2 (2 files)
Receiving file: file5 (13 bytes)
Receiving file: file6 (13 bytes)

Other Examples:

Sending Multiple Files Python Using Socket
Example using makefile

transfer integer over a socket in C

First of all, sizeof(int) may differ on your sender and receiver machine. So I would recommend you to use something like int32_t from stdint.h.

Also, it is not guaranteed that read(..,..,sizeof(int)) will read exactly sizeof(int) bytes - it can read nothing, or it can read less bytes. So, the correct variant will be something more like this:

int send_int(int num, int fd)
{
    int32_t conv = htonl(num);
    char *data = (char*)&conv;
    int left = sizeof(conv);
    int rc;
    do {
        rc = write(fd, data, left);
        if (rc < 0) {
            if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) {
                // use select() or epoll() to wait for the socket to be writable again
            }
            else if (errno != EINTR) {
                return -1;
            }
        }
        else {
            data += rc;
            left -= rc;
        }
    }
    while (left > 0);
    return 0;
}

int receive_int(int *num, int fd)
{
    int32_t ret;
    char *data = (char*)&ret;
    int left = sizeof(ret);
    int rc;
    do {
        rc = read(fd, data, left);
        if (rc <= 0) { /* instead of ret */
            if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) {
                // use select() or epoll() to wait for the socket to be readable again
            }
            else if (errno != EINTR) {
                return -1;
            }
        }
        else {
            data += rc;
            left -= rc;
        }
    }
    while (left > 0);
    *num = ntohl(ret);
    return 0;
}

send multiple files parallely using tcp connection between client and server in c#

The TCPClient and TCPListener that both use sockets internally and simply provide a higher level of abstraction. In other words, you are still using sockets, just not directly.

In order to send multiple files in parallel, you will need the server to be able to accept more than one connection. There are several ways you can accomplish this but the general idea is the same:

start your listener
as soon as any client connects, don't block while you are handling that client, but instead attempt to connect the next client connection as soon as it is available
for each connected client, handle the connection to receive the file data and do whatever you need to do with it

On the client, you can spin up a number of TCPClient objects, one for each file you'd like to send and send the data to your listener.

You will have to figure out the logistics (the protocol) of your data transfers. As you get the incoming data transfer for each file you are receiving on the listener side, you have to decide what each file represents and who it is from. The details here are quite subjective and not really a good fit for StackOverflow Q&A.