New Line Issue with Netcat

Send text file, line by line, with netcat

Do you HAVE TO use netcat?

cat textfile > /dev/tcp/HOST/PORT

can also serve your purpose, at least with bash.

I'de like to send, and receive, one by one the lines, not all the file in a single shot.

Try

while read x; do echo "$x" | nc host port; done < textfile

ncat: piping multiline request breaks HTTP

The \n in the string is sent literally. Use echo -e to enable interpretation of backslash escapes. Also, the newline sequence for HTTP 1.1 is \r\n (CRLF). And the header section ends with an additional end-of-line.

Try:

echo -e 'GET / HTTP/1.1\r\nHost: 192.168.56.20\r\n\r\n' | ncat 192.168.56.20 80

Alternatively, the ncat has the option to convert new lines to CRLF:

-C, --crlf                 Use CRLF for EOL sequence

Hence, you can write:

echo -e 'GET / HTTP/1.1\nHost: 192.168.56.20\n\n' | ncat -C 192.168.56.20 80

and you should get the same result.

listening on netcat works but not grep'able (or several other utilities)

awk is buffering when its output is not going to a terminal. If you have GNU awk, you can use its fflush() function to flush after every print

gawk '{print; fflush()}' <&5 | grep foo

In this particular case though, you don't need awk and grep, either will do.

awk /foo/ <&5
grep foo <&5

See BashFAQ 9 for more on buffering and how to work around it.

socket can't communicate with netcat bash

From input()'s documentation:

The function then reads a line from input, converts it to a string
(stripping a trailing newline), and returns that.

But Bash is operating in canonical mode and won't process input till a new line arrives. This won't happen, leading to recv blocking forever.

add a + '\n' after the user = input("what to send?: ") to fix it.

My attempt at netcat in C++ won't get me replies from the server

The real netcat can read from socket and from stdin in parallel. Your program can read either from stdin or from the socket and both of these calls getline() and recv() are waiting of there are no data.

So your program probably:

getline() reads a line from stdin. The line contains something like "GET ... \n"
send the line to server but it is not complete http request because it is not ended by empty line so http server does not respond and waits for http headers.
Your program calls recv() and blocks forever.

You need to change the logic of your program. You can either use separate threads for reading from stdin and receiving data or you can use select() or poll() which provides waiting for an events on multiple file descriptors.

EDIT: additional clarification due to a question in comment

The HTTP request consists of

request line
request headers (0 headers are valid)
request body (can be empty)

Request headers and request body are diveded by an empty line. The empty line is required even if the body is empty because server needs to know that there are no more headers.

Moreover the HTTP standard requires <CR><LF> at the and of line. Minimum valid HTTP request (no headers, empty body) is a request line followed by an empty line: "GET / HTTP/1.1\r\n\r\n"

netcat closes connection after 8192 bytes

This is happening because netcat is receiving end-of-file from its standard input. That is, the command echo "something" causes the string something\n to be sent to the pipe connected to netcat's standard input; then the pipe is closed (because the echo command terminated). So, on the first read of the pipe, netcat will receive that string, but on its next read, it will receive EOF. This causes it to break its connection to the peer even though the peer may not be finished sending.

Essentially, after being started as above, netcat will keep sending its standard input to the socket, and the socket to its standard output until one of them is closed. Then it exits.

So you simply need to do something to ensure that netcat doesn't receive EOF on its standard input before it gets EOF on the socket. Something like this will likely do it:

(echo "something" ; sleep 1) | netcat localhost $myport

Now the output of echo "something" is sent to the pipe connected to netcat's input, but the pipe won't actually be closed until the sleep 1 also completes since both commands are started in a sub-shell which is connected to the write end of the pipe. (You might need to tinker with the number of seconds slept if the amount to be sent by the peer is large.)

Send String over netcat connection

Try this:

echo -n 'Line of text' | nc <ip> <port>

You can also use temp file syntax:

cat <(echo "Line of test") | nc <ip> <port>