What Is a Glibc Free/Malloc/Realloc Invalid Next Size/Invalid Pointer Error and How to Fix It

What is a glibc free/malloc/realloc invalid next size/invalid pointer error and how to fix it?

Answer for Example Question

unwind gave the accepted answer to the example question:

Your code is wrong.

You are allocating space for a single pointer (malloc(sizeof(char*))), but no characters. You are overwriting your allocated space with all the strings, causing undefined behavior (in this particular case, corrupting malloc()'s book-keeping data).

You don't need to allocate space for the pointer (res); it's a local variable. You must allocate space for all the characters you wish to store at the address held by the pointer.

Since you're going to be traversing a list to find strings to concatenate, you can't know the total size upfront. You're going to have to do two passes over the list: one to sum the strlen() of each string, then allocate that plus space for the separator and terminator, then another pass when you actually do the concatenation.

Generic Answer

What you are seeing is the result of a corruption in the internal structures of the glibc allocator. When you are allocating or freeing dynamic memory, the allocator has to manage the memory it reserved from the OS and, depending on the action requested by you, find a new chunk to hand out, sort a freed chunk into the list of those that it can hand out later again, or give the memory back to the operating system. These error messages show that the data structures it uses to manage this functionality are corrupted.

These errors all mean that some of your code has modified memory that it was not given to use, invoking undefined behaviour. This is most likely the result of overwriting some memory quite a bit earlier in your program, and it is totally possible that the error does not lie in the frobnicate() function.

Yes, this means that the error can be anywhere in your program or 3rd party libraries you use.

This is probably not a good question for Stack Overflow. Unless you have a good simple reproduction of your problem, this community may be unable to help you very much. The cause of the error can be anywhere in your code (and is very often not in the function where the error is spotted), and it may be in code that we cannot see. Stack Overflow is not a collaborative debugging site. Even when someone can find the flaw in your code, it is unlikely that your specific question will ever help any future visitor.

Common causes

• Use after free. You have freed/deleted some memory and writing into it afterwards, overwriting the structures glibc needs for bookkeeping.
• Off-by-N error. You are writing N bytes after an allocated chunk into unallocated memory that glibc uses internally for its bookkeeping.
• Uninitialized pointers. You are not initializing a pointer. By coincidence it points to some memory reserved by glibc but not allocated by your program and you write to it.
• Allocating the wrong amount of space. This can be because you wrote long *data = malloc(number * 4) instead of long *data = malloc(number * sizeof(long)); or (better) long *data = malloc(number * sizeof(*data));. There are many other ways to get the size calculation wrong. Another common one is to forget to account for the null terminator character at the end of a string: char *copy = malloc(strlen(str)); instead of char *copy = malloc(strlen(str)+1);.

What you need to do now is to roll up your sleeves and debug that problem

There is no simple answer what to look for, or what to fix. No single syntactical construct that you were using wrong. The cause of this bug can come in literally thousands of varieties.

Tools

• valgrind A tool created mostly for the purpose of finding exactly this kinds of errors. If it can't find anything make sure you are using the latest version, and you are also trying out the included exp-sgcheck tool. If you are running multithreaded code, the cause might also be related to a race condition so you might want to try the included race condition checkers drd and helgrind for more insight. At the point of writing this, valgrind supports the following platforms:
  
  • X86/Linux,
  • AMD64/Linux,
  • ARM/Linux,
  • PPC32/Linux,
  • PPC64/Linux,
  • S390X/Linux,
  • MIPS32/Linux,
  • MIPS64/Linux,
  • ARM/Android (2.3.x and later),
  • X86/Android (4.0 and later),
  • X86/Darwin and
  • AMD64/Darwin (Mac OS X 10.7, with limited support for 10.8).
• purify A similar tool to valgrind, but commercial and aimed at a different set of platforms.
• AddressSanitizer A similar tool, but integrated into the compiler toolchain (gcc and clang).
• efence A drop in allocator replacement that will try to crash your program earlier, so that you can find out with a normal debugger where the write to invalid memory happened.
• dmalloc a library with a similar purpose as efence.

Needing more assistance

If you can't solve your problem using one these tools, you should try to create an MCVE (How to create a Minimal, Complete, and Verifiable Example?) or, equivalently, an SSCCE (Short, Self Contained, Correct (Compilable), Example).

Remember to work on a copy of your code because creating an MCVE requires you to ruthlessly remove code that does not help reproduce the problem. Using a VCS (version control system) to assist is a good idea; you can record intermediate stages in reducing the problem to a minimum. It might be a new throw-away repository just for reducing your problem to a manageable size.

With a good modular design to your code, it should be relatively easy to create the MCVE. Maybe you also already have a unit test that is better suited to be fed into one of the above tools. You also might just want to create one that can later serve as a regression test for this bug.

glibc detected: realloc(): invalid next size

This code

 ptr=malloc(ptrsize*sizeof(char *));
 ptr[ptrsize]=(char *)malloc(wordsize*sizeof(char));

And the similar lines repeated later in the loop body are incorrect.

An array like

Type* ptr = malloc(N * sizeof(Type));

has valid indexes from 0 to N - 1. ptr[N] is always going to be past the end of the array. Writing to this memory and/or reallocing it is likely to end up corrupting the heap data structures.

free char*: invalid next size (fast)

Your code is wrong.

You are allocating space for a single pointer (malloc(sizeof(char*))), but no characters. You are overwriting your allocated space with all the strings, causing undefined behavior (in tihs particular case, corrupting malloc()'s book-keeping data).

You don't need to allocate space for the pointer (res), it's a local variable. You must allocate space for all the characters you wish to store at the address held by the pointer.

Since you're going to be traversing a list to find strings to concatenate, you can't know the total size upfront. You're going to have to do two passes over the list: one to sum the strlen() of each string, then allocate that plus space for the separator and terminator, then another pass when you actually do the concatenenation.

realloc invalid next size

With each new matrix you "allocate", you should be expanding your global pointer lists. you don't. You just reallocate them to the same size the were before:

This:

void newMatrix()
{
    nMatrix++;
    values = realloc(values, sizeof(int*));
    columns = realloc(columns, sizeof(int*));
    rowPointer = realloc(rowPointer, sizeof(int*));
    dimensions = realloc(dimensions, sizeof(Dims));
    lineCount = realloc(lineCount, sizeof(int));
    highestRow = realloc(highestRow, sizeof(int));
}

Should be this:

void newMatrix()
{
    nMatrix++;
    values = realloc(values, (nMatrix+1)*sizeof(int*));
    columns = realloc(columns, (nMatrix+1)*sizeof(int*));
    rowPointer = realloc(rowPointer, (nMatrix+1)*sizeof(int*));
    dimensions = realloc(dimensions, (nMatrix+1)*sizeof(Dims));
    lineCount = realloc(lineCount, (nMatrix+1)*sizeof(int));
    highestRow = realloc(highestRow, (nMatrix+1)*sizeof(int));
}

Note: the (nMatrix+1) value is used because you start with nMatrix as (-1), and upon first increment it is (0), the next is (1), etc... I.e. it always indexes the last row inserted, but your vector magnitudes needs to be +1 to that for hopefully obvious reasons.

I would strongly suggest you consider what happens when realloc() fails as well, as it will return NULL and in the process leak whatever memory was pointed to by the pointer you passed in.

There may be other issues, but that was the first one that jumped out at me.

realloc(): invalid next size

The data passed in to write_data() isn't necessarily nul-terminated; that's why it tells you the number of bytes.

This means you can't use strcat(). Using it is running off the end of the array and corrupting the data structures used by malloc / realloc, hence the error.

Your write_data() should use memcpy() instead, like so:

int write_data( char *ptr, size_t size, size_t nmemb, void *stream )
{
    size_t ThisSize = (size * nmemb); //Stores the size of the data to be stored
    size_t DataLen = RunningSize; //length of the data so far

    RunningSize = (RunningSize + ThisSize ); //update running size (used as new size)

    Data = realloc( Data, RunningSize ); //get new mem location (on the 12th iteration, this fails)
    memcpy((char *)Data + DataLen, ptr, ThisSize); //add data in ptr to Data

    return ThisSize; //the function must return the size of the data it received so cURL knows things went ok.
}

You will also need to initialise RunningSize to 0, not 1. You can initialise Data to NULL - passing NULL to realloc() is allowed (and makes it behave just like malloc()).

realloc says Invalid next size

Here

path = (char **) malloc(sizeof(char *));

you are allocating a pointer to a pointer to char, that is space for one pointer to a C-"string".

Then here:

for (int i = 0; i < 5; i++)
{
   path[i] = strdup("TEST");
   path = (char **) realloc(path, sizeof(char *));
}

you not only use one pointer to a C-"string" but reallocation for more does nothing more then just reallocate the same size as before, but infact you access as if you added memory for one more pointer in each iteration.

Doing so you overwrite memory not allocated (or alrerady in use by something else) and mess up the programs memory management.

A somewhat more straight forward approach to this would be:

  ...

  char ** path = NULL;

  for (size_t i = 0; i < 5; ++i)
  {
    path = realloc(path, (i+1) * sizeof(*path));
    path[i] = strdup("TEST");
  }

  ...

realloc: invalid next size

I believe you're accessing memory outside of your malloced/realloced region.

Not setting count = 0 inside your while look should fix this.

---

To try and elaborate...

When starting the formula:

count = 0
length = 0
CUR_MAX = 4095

These will increment until they reach 4095. Once we reach 4095, we will have:

count = 0
length = 4096
CUR_MAX = 8190

We will then increment until count is 8190. Shorly before then, we have:

count = 8100
length = 12196
CUR_MAX = 8190

Your array is only 8190 in length, but you're dereferencing buffer[12196], which is an invalid index.

---

Also, you probably want to handle the else case for when temp == NULL. This probably can be handled by an assert or a big old failure. And don't free(temp) inside your success side. That's freeing the memory you just attempted to allocate. And is causing your new segmentation fault.

Problems with realloc - invalid next size

This is the most likely problem:

lineToken = malloc(sizeof(1));

Here you allocate the size of an integer literal, but lineToken needs at least sizeof(*lineToken) (or sizeof(char *)) bytes. The size of an integer and the size of a pointer may not be the same, especially on a 64-bit platform (where int is four bytes and pointers are eight bytes).

So when you do

lineToken[0] = strtok(string, " ")

you write beyond what's allocated (buffer overflow) and overwrite data put there by the allocator.

There's also a problem with the realloc call which will allocate i + 1 bytes. So even when you're on a 32-bit platform (where the size of int just happens to be the same as the size of a pointer) you will reallocate the pointer from being four bytes to two bytes in the first iteration of the loop.

---

You also have other problems, like these loops:

for(i = 1; i; i++)

Remember that in C all non-zero values are true, and when will i ever be zero in that loop?

---

Related Topics