Sending Data on Af_Packet Socket

Sending data on AF_PACKET socket

Don't. use write(), use sendto().

If you use PF_PACKET,SOCK_DGRAM, then it builds the link-layer headers for you, which is normally what you want. You still need to build whatever higher protocol you are using on top though.

You specify a sockaddr_ll in the destination parameter. You can specify a link-layer unicast address, multicast address or broadcast address.

AF_PACKET Socket Not Sending Empty UDP Packet With SOCK_RAW In C

I was able to figure out the issue. I wasn't converting the Ethernet protocol (ETH_P_IP) to network byte order via htons() since it's a big endian. With that said, I had to convert iphdr->total_len to network byte order as well. Otherwise, the IP header's total length would be incorrect according to tcpdump. I didn't do this in other programs I made and it worked okay. Therefore, I'd assume the kernel converted the IP header's total length to network byte order automatically.

Since I'm using an AF_PACKET socket to send, I have to do things the kernel would normally do.

Here's the program's final code for anyone wondering:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/ip.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <net/if.h>
#include <linux/if.h>
#include <linux/if_packet.h>
#include <linux/udp.h>
#include <net/ethernet.h>
#include <string.h>
#include <error.h>
#include <errno.h>
#include <inttypes.h>

#define REDIRECT_HEADER

#include "csum.h"

#define MAX_PCKT_LENGTH 65535

int main()
{
    int sockfd;
    struct sockaddr_ll dst;
    char pckt[MAX_PCKT_LENGTH];

    sockfd = socket(AF_PACKET, SOCK_RAW, IPPROTO_RAW);

    if (sockfd <= 0)
    {
        perror("socket");

        exit(1);
    }

    dst.sll_family = PF_PACKET;
    dst.sll_protocol = htons(ETH_P_IP);

    if ((dst.sll_ifindex = if_nametoindex("ens18")) == 0)
    {
        fprintf(stdout, "Interface 'ens18' not found.\n");

        exit(1);
    }

    // Do destination ethernet MAC (ae:21:14:4b:3a:6d).
    dst.sll_addr[0] = 0xAE;
    dst.sll_addr[1] = 0x21;
    dst.sll_addr[2] = 0x14;
    dst.sll_addr[3] = 0x4B;
    dst.sll_addr[4] = 0x3A;
    dst.sll_addr[5] = 0x6D;
    dst.sll_halen = ETH_ALEN;

    // I tried doing this with and without bind. Still not working.
    if (bind(sockfd, (struct sockaddr *)&dst, sizeof(dst)) < 0)
    {
        perror("bind");

        exit(1);
    }

    struct ethhdr *ethhdr = (struct ethhdr *) (pckt);
    struct iphdr *iphdr = (struct iphdr *) (pckt + sizeof(struct ethhdr));
    struct udphdr *udphdr = (struct udphdr *) (pckt + sizeof(struct ethhdr) + sizeof(struct iphdr));
    unsigned char *data = (unsigned char *) (pckt + sizeof(struct ethhdr) + sizeof(struct iphdr) + sizeof(struct udphdr));

    // Do source ethernet MAC (1a:c4:df:70:d8:a6).
    ethhdr->h_source[0] = 0x1A;
    ethhdr->h_source[1] = 0xC4;
    ethhdr->h_source[2] = 0xDF;
    ethhdr->h_source[3] = 0x70;
    ethhdr->h_source[4] = 0xD8;
    ethhdr->h_source[5] = 0xA6;

    for (int i = 0; i < 30; i++)
    {
        memcpy(data + i, "b", 1);
    }

    // Copy destination MAC to sockaddr_ll.
    memcpy(ethhdr->h_dest, dst.sll_addr, ETH_ALEN);

    // Protocol.
    ethhdr->h_proto = htons(ETH_P_IP);

    // Fill out ip header.
    iphdr->ihl = 5;
    iphdr->version = 4;
    iphdr->frag_off = 0;
    iphdr->id = htons(0);
    iphdr->protocol = IPPROTO_UDP;
    iphdr->tos = 0x0;
    iphdr->ttl = 64;
    iphdr->saddr = inet_addr("10.50.0.3");
    iphdr->daddr = inet_addr("10.50.0.4");
    iphdr->tot_len = htons(sizeof(struct iphdr) + sizeof(struct udphdr) + 30);
    iphdr->check = 0;
    iphdr->check = ip_fast_csum(iphdr, iphdr->ihl);

    // Fill out UDP header.
    udphdr->source = htons(27000);
    udphdr->dest = htons(27015);
    udphdr->len = htons(sizeof(struct udphdr) + 30);
    udphdr->check = 0;
    udphdr->check = csum_tcpudp_magic(iphdr->saddr, iphdr->daddr, sizeof(struct udphdr) + 30, IPPROTO_UDP, csum_partial(udphdr, sizeof(struct udphdr) + 30, 0));

    // Send packet
    uint16_t sent;
    int len = ntohs(iphdr->tot_len) + sizeof(struct ethhdr) + 30;

    if ((sent = sendto(sockfd, pckt, len, 0, (struct sockaddr *)&dst, sizeof(dst))) < 0)
    //if ((sent = write(sockfd, pckt, len)) < 0)
    {
        perror("sendto");
    }

    fprintf(stdout, "Sent %d of data. %d is IPHdr len. %d is len.\n", sent, iphdr->tot_len, len);

    close(sockfd);

    exit(0);
}

Is there a simple way to send a packet over a raw socket using Ruby's Socket class?

This will do it:

require 'socket'

interface = 'lo'         # loopback interface
interface_index = 0x8933 # SIOCGIFINDEX
geonet_frame = "\x00\x1f\xc6\x51\x07\x07\x07\x07\x07\x07\x07\x07\x07\x07\xc6\x51\x07\x07\x07\x07\x07\x07\xef\x06\x07\x35\x97\x00\x24\x8c\x7a\xdf\x6f\x08\x00\x45\x00\x00\x3d\xf3\x7f\x40\x00\x40\x11\x30\xc6\x0a\x01\x01\x68\x0a\x01\x01\x01\x99\x80\x00\x35\x00\x29\x16\xa5\x01\x76\x01\x00\x00\xff\x00\x00\x01\x00\x00\x00"

socket = Socket.new(Socket::AF_PACKET, Socket::SOCK_RAW, Socket::IPPROTO_RAW)
ifreq = [interface.dup].pack 'a32'
socket.ioctl(interface_index, ifreq)
socket.bind([Socket::AF_PACKET].pack('s') + [Socket::IPPROTO_RAW].pack('n') + ifreq[16..20]+ ("\x00" * 12))

socket.send(geonet_frame, 0)

I used AF_PACKET because it matched the python example, but you can replace it with PF_PACKET if your system doesn't support it.

Replacing the geonet_frame variable as Cocoabean suggested also works:

geonet_frame = [ 0x00, 0x1f, 0xc6, 0x51, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0xc6, 0x51, 0x07, 0x07, 0x07, 0x07,0x07, 0x07, 0xef, 0x06, 0x07, 0x35, 0x97, 0x00, 0x24, 0x8c, 0x7a, 0xdf, 0x6f, 0x08, 0x00, 0x45, 0x00, 0x00, 0x3d, 0xf3, 0x7f, 0x40, 0x00, 0x40, 0x11, 0x30, 0xc6, 0x0a, 0x01, 0x01, 0x68, 0x0a, 0x01, 0x01, 0x01, 0x99, 0x80, 0x00, 0x35, 0x00, 0x29, 0x16, 0xa5, 0x01, 0x76, 0x01, 0x00, 0x00, 0xff, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00].pack("C*")

Tested with ruby 1.9.3p484 on CentOS 6.6. Hope it helps.

Cheers.

Retransmitting large packets with raw sockets

Your network card probably has segmentation offload enabled, which means the hardware can re-assemble TCP segments before they reach the OS or your code.

You can check whether that is the case by running ethtool -k.
While transparently capturing TCP traffic and re-transmitting it at such a low level is often more trouble than it is worth(one are often better off doing this at the application layer, terminate the TCP connection and set up a new TCP connection towards your host B), you cannot capture and re-send packets if your network card has messed with the packets. You need to:

• Turn off generic-segmentation-offload
• Turn off generic-receive-offload
• Turn off tcp-segmentation-offload
• Turn off udp-fragmentation-offload if you are also dealing with UDP
• Turn off rx-vlan-offload/tx-vlan-offload if your packets are VLAN encapsulated
• Possibly turn off rx-checksumming and tx-checksumming. It either works if both
  are enabled, or it's broken wrt. RAW sockets if enabled, depending on your
  kernel version and type of network card.

These can be turned on/off with the ethtool -K command, the exact syntax is described in the ethtool manpage.

C Program to receive and send the same packets out over another interface

I finally got it to work ! I was able to send packets received on eth0 to eth1 unchanged.

Corrections:

1. "write" instead of "sendto" , although it isn't advisable.
2. A fresh socket for sending only.
3. It seems the SO_BINDTODEVICE using setsockopt doesn't work properly so I had to use a bind function to bind the socket to the interface.
4. struct sockaddr_in to struct sockaddr_ll

I put this together in a hurry so there may be some redundant lines in there. I don't know if daddr is even neccessry but it works so there it is.
Here's the full code :

// All #includes 

int main()
{
    int saddr_size , data_size, daddr_size, bytes_sent;
    struct sockaddr_ll saddr, daddr;
    unsigned char *buffer=malloc(65535);

    int sock_raw = socket( AF_PACKET , SOCK_RAW , htons(ETH_P_ALL)) ; //For receiving
    int sock = socket( PF_PACKET , SOCK_RAW , IPPROTO_RAW) ;            //For sending

    memset(&saddr, 0, sizeof(struct sockaddr_ll));
    saddr.sll_family = AF_PACKET;
    saddr.sll_protocol = htons(ETH_P_ALL);
    saddr.sll_ifindex = if_nametoindex("eth0");
    if (bind(sock_raw, (struct sockaddr*) &saddr, sizeof(saddr)) < 0) {
        perror("bind failed\n");
        close(sock_raw);
    }

    memset(&daddr, 0, sizeof(struct sockaddr_ll));
    daddr.sll_family = AF_PACKET;
    daddr.sll_protocol = htons(ETH_P_ALL);
    daddr.sll_ifindex = if_nametoindex("eth1");
    if (bind(sock, (struct sockaddr*) &daddr, sizeof(daddr)) < 0) {
      perror("bind failed\n");
      close(sock);
    }
    struct ifreq ifr;
    memset(&ifr, 0, sizeof(ifr));
    snprintf(ifr.ifr_name, sizeof(ifr.ifr_name), "eth1");
    if (setsockopt(sock, SOL_SOCKET, SO_BINDTODEVICE, (void *)&ifr, sizeof(ifr)) < 0) {
        perror("bind to eth1");
        }

    while(1)
    {
        saddr_size = sizeof (struct sockaddr);
        daddr_size = sizeof (struct sockaddr);
        //Receive a packet
        data_size = recvfrom(sock_raw , buffer , 65536 , 0 ,(struct sockaddr *) &saddr , (socklen_t*)&saddr_size);

        if(data_size <0 )
        {
            printf("Recvfrom error , failed to get packets\n");
            return 1;
        }
        else{
        printf("Received %d bytes\n",data_size);

        //Huge code to process the packet (optional)

        //Send the same packet out
        bytes_sent=write(sock,buffer,data_size);
        printf("Sent %d bytes\n",bytes_sent);
         if (bytes_sent < 0) {
            perror("sendto");
            exit(1);
         }

        }
    }
    close(sock_raw);
    return 0;
}

Thanks to all those who responded.

Packet Sockets not receiving data for custom protocol ID

ETH_P_ALL vs any other protocol

The protocol ETH_P_ALL has the special role of capturing outgoing packets.

Receiver socket with any protocol that is not equal to ETH_P_ALL receives packets of that protocol that come from the device driver.

Socket with protocol ETH_P_ALL receives all packets before sending outgoing packets to the device driver and all incoming packets that are received from the device driver.

Loopback device vs Ethernet device

Packets sent to the loopback device go out from that device and then the same packets are received from the device as incoming.
So, when CUSTOM_PROTO is used with loopback the socket captures packets with custom protocol as incoming.

Note that if ETH_P_ALL is used with the loopback device each packet is received twice. Once it is captured as outgoing and the second time as incoming.

In case of eth0 the packet is transmitted from the device. So, such packets go to the device driver and then they can be seen on the other side of the physical Ethernet port. For example, with VirtualBox "Host-only" network adapter those packets can be captured by some sniffer in the host system.

However, packets transmitted to the physical port (or its emulation) are not redirected back to that port. So, they are not received as incoming from the device. That is why such packets can be captured only by ETH_P_ALL in outgoing direction and they cannot be seen by CUSTOM_PROTO in incoming direction.

Technically it should possible to prepare special setup to do external packet loopback (packets from the device port should be sent back to that port). In that case the behavior should be similar to the loopback device.

Kernel implementation

See the kernel file net/core/dev.c. There are two different lists:

struct list_head ptype_base[PTYPE_HASH_SIZE] __read_mostly;
struct list_head ptype_all __read_mostly;   /* Taps */

The list ptype_all is for socket handlers with protocol ETH_P_ALL. The list ptype_base is for handlers with normal protocols.

There is a hook for outgoing packets in xmit_one() called from dev_hard_start_xmit():

    if (!list_empty(&ptype_all))
        dev_queue_xmit_nit(skb, dev);

For outgoing packets the function dev_queue_xmit_nit() is called for ETH_P_ALL processing each item of ptype_all. Finally the sockets of type AF_SOCKET with protocol ETH_P_ALL capture that outgoing packet.

---

So, the observed behavior is not related to any custom protocol. The same behavior can be observed with ETH_P_IP. In that case the receiver is able to capture all incoming IP packets, however it cannot capture IP packets from sender.c that sends from "eth0" to MAC address of "eth0" device.

It can be also seen by tcpdump. The packets sent by the sender are not captured if tcpdump is called with an option to capture only incoming packets (different versions of tcpdump use different command line argument to enable such filtering).

---

The initial task where on the same machines it is needed to distinguish packets by protocol IDs can be solved using ETH_P_ALL. The receiver should capture all packets and check the protocol, for example:

while (1) {
    int len = recvfrom(sockfd, recvbuf, 1514, 0, NULL, NULL);

    if (ntohs(*(uint16_t*)(recvbuf + ETH_ALEN + ETH_ALEN)) == CUSTOM_PROTO) {
        printf("I received: \n");
        break;
    }
}

---

Useful reference "kernel_flow" with a nice diagram http://www.linuxfoundation.org/images/1/1c/Network_data_flow_through_kernel.png

It is based on the 2.6.20 kernel, however in the modern kernels ETH_P_ALL is treated in the same way.

AF_PACKET raw IP packets with SOCK_DGRAM require fragmenting?

After development, I can confirm that if you are using an AF_PACKET socket with SOCK_DGRAM for IP packets, you do have to manually handle the fragmenting. I tried to write a 4000 byte UDP datagram, and received a "message size too long" error.

Type of socket address from recvfrom() with AF_PACKET / PF_PACKET

I am actually looking for the interface the packet was received on.

You should be using recvmsg, which gives access to metadata such as the packet's destination address (useful for multihomed systems) and I think also the interface.

The sender address you're looking at now isn't going to tell you the interface. It's good for sending a reply packet with sendto though.

---

Related Topics