Python Ctypes and Sysctl

python ctypes and sysctl

You are not providing the correct values to the sysctl function. Detailed information on the arguments of sysctl() can be found here.

Here are your errors:

• You have forgotten the nlen argument (second argument)
• The oldlenp argument is a pointer to the size, not directly the size

Here is the correct function (with minor improvement):

def posix_sysctl_long(name):
    _mem = c_uint64(0)
    _def = sysctl_names[name]
    _arr = c_int * len(_def)
    _name = _arr()
    for i, v in enumerate(_def):
        _name[i] = c_int(v)
    _sz = c_size_t(sizeof(_mem))
    result = libc.sysctl(_name, len(_def), byref(_mem), byref(_sz), None, c_size_t(0))
    if result != 0:
        raise Exception('sysctl returned with error %s' % result)
    return _mem.value

In python how do I get the value of kern.module_path?

I think your best bet is the sysctl library: https://pypi.python.org/pypi/sysctl/0.1b2

Alternatively, you could use ctypes to interface directly with the C library. There's already a stackoverflow answer with an example of how to read sysctl values here: python ctypes and sysctl

(Obviously, if you wanted to set values, you would have to modify that ctypes example a little).

Garbage uint64 values from ctypes

In if_data64, the ifi_baudrate field should be c_uint64 instead of c_uint32. Fixing this should align ifi_bytes and ifo_bytes to the correct offset. It's also missing the last field, ifi_lastchange. This is an 8 byte timeval32 struct. All together, this explains why sdl needed an extra 12 byte offset.

class timeval32(Structure):
    _fields_ = [('tv_sec', c_int32), 
                ('tv_usec', c_int32)]

class if_data64(Structure):
    _pack_ = 4
    _fields_ = [('ifi_type', c_ubyte),
                ('ifi_typelen', c_ubyte),
                ('ifi_physical', c_ubyte),
                ('ifi_addrlen', c_ubyte),
                ('ifi_hdrlen', c_ubyte),
                ('ifi_recvquota', c_ubyte),
                ('ifi_xmitquota', c_ubyte),
                ('ifi_unused1', c_ubyte),
                ('ifi_mtu', c_uint32),
                ('ifi_metric', c_uint32),
                ('ifi_baudrate', c_uint64),    # was c_uint32
                ('ifi_ipackets', c_uint64),
                ('ifi_ierrors', c_uint64),
                ('ifi_opackets', c_uint64),
                ('ifi_oerrors', c_uint64),
                ('ifi_collisions', c_uint64),
                ('ifi_ibytes', c_uint64),
                ('ifi_obytes', c_uint64),
                ('ifi_imcasts', c_uint64),
                ('ifi_omcasts', c_uint64),
                ('ifi_iqdrops', c_uint64),
                ('ifi_noproto', c_uint64),
                ('ifi_recvtiming', c_uint32),
                ('ifi_xmittiming', c_uint32),
                ('ifi_lastchange', timeval32)] # was missing

Instead of using cast with pointers, I'd write the loop using the from_buffer method (Python 2.6+) and offsets.

buf_offset = 0
while buf_offset < sysctl_buf_len.value:
   msg_offset = buf_offset
   ifmsg = if_msghdr2.from_buffer(sysctl_buf, msg_offset)
   buf_offset += ifmsg.ifm_msglen
   if ifmsg.ifm_type != RTM_IFINFO2 or ifmsg.ifm_flags & IFF_LOOPBACK:
      continue
   sdl_offset = msg_offset + c_sizeof(if_msghdr2)
   sdl = sockaddr_dl.from_buffer(sysctl_buf, sdl_offset)
   if sdl.sdl_family != AF_LINK:
      continue
   print sdl.sdl_data[:sdl.sdl_nlen]
   print ifmsg.ifm_data.ifi_ibytes, ifmsg.ifm_data.ifi_obytes

Why is multiprocessing copying my data if I don't touch it?

The problem was that by default Linux checks for the worst case memory usage, which can indeed exceed memory capacity. This is true even if the python language doesn't exposure the variables. You need to turn off "overcommit" system wide, to achieve the expected COW behavior.

sysctl `vm.overcommit_memory=2'

See https://www.kernel.org/doc/Documentation/vm/overcommit-accounting

Measuring elapsed time in python

You can use perf_counter function of time module in Python Standard Library:

from datetime import timedelta
from time import perf_counter

startTime = perf_counter()
CallYourFunc()
finishedTime = perf_counter()
duration = timedelta(seconds=(finishedTime - startTime))

---

Related Topics