Create a Big Array in C++

Creating large arrays in C

Usually you need to create such an array dynamically on the heap.

int *integer_array = (int*)malloc(2000000 * sizeof(int));
float *float_array = (float*)malloc(2000000 * sizeof(float));

The array might be too large for stack allocation, e.g. if used not globally, but inside a function.

int main () {
    int a[200000000]; /* => SEGV */
    a[0]=0;
}

The easiest fix, move the array outside:

int a[200000000];
int main () {
    a[0]=0;
}

You can also declare it static:

int main () {
    static int a[200000000];
    a[0]=0;
}

Note that the stack size is system dependent. One can change it with ulimit.

How to declare and use huge arrays of 1 billion integers in C?

Michael is right, you can't fit that much on the stack. However, you can make it global (or static) if you don't want to malloc it.

#include <stdlib.h>
#include <time.h>
#define N 1000000000
static int list[N];

int main(int argc, char **argv)
{
  size_t i;
  srand(time(NULL));
  for(i=0; i<N; i++)
     list[i] = rand()%1000;
  return 0;
}

Allocating a large array on the stack in C

You're statically allocating an array on the stack, this means that the compiler will write code to reserve that space, and when your main() is called, it will try to move the stack pointer way out of the available mapped stack area for your program. Touching the stack would then cause a segmentation fault, which is what you see.

You could increase the stack size, but it's not that simple nor portable, and in general allocating such a large array on the stack is bad practice and should be avoided. To handle such a big array, you should dynamically allocate it, using for example malloc().

Here's a working example:

#include <stdio.h>
#include <stdlib.h>

int main(void)
{
    short int *big;

    big = malloc(6000000 * sizeof(short int));
    if (big == NULL) {
        fputs("Failed to allocate memory!\n", stderr);
        return 1;
    } 

    // Do whatever...

    free(big);
    return 0;
}

Also, remember that you cannot use sizeof() in this case since big is a dynamically allocated array (sizeof(big) would yield the size of the pointer, not the real size of the array). This is because sizeof() is a compile-time operator and can only help you if the size is known at compile time. In this case, it is not, since the space is allocated at runtime.

If you want to know the size of that array, you can simply calculate it with a multiplication:

short int *big;
const size_t big_size = 6000000ULL * sizeof(short int);

printf("Size: %zu\n", big_size);

big = malloc(big_size);
// ...

C creating a struct array beyond certain size causes a crash

I think your program crashes because you statically allocate too much memory on the stack.
Try using the malloc or calloc function. It dynamically allocates memory on the heap instead e.g. :

struct object *l = malloc(variable*sizeof(struct object));

Don't forget to free it afterwards using the free function.

free(l);

Create too large array in C++, how to solve?

A matrix of size 60,000 x 60,000 has 3,600,000,000 elements.

You're using type float so it becomes:

60,000 x 60,000 * 4 bytes = 14,400,000,000 bytes ~= 13.4 GB

Do you even have that much memory in your machine?

---

Note that the issue of stack vs heap doesn't even matter unless you have enough memory to begin with.

---

Here's a list of possible problems:

• You don't have enough memory.
• If the matrix is declared globally, you'll exceed the maximum size of the binary.
• If the matrix is declared as a local array, then you will blow your stack.
• If you're compiling for 32-bit, you have far exceeded the 2GB/4GB addressing limit.

Filling a large array instantly - C

If you are on an LP64 system then the problem likely comes from only allocating elements of size sizeof(int) bytes (4) but then expecting the array to contain enough 8-byte elements.

If you are using a 32-bit compiler where int and long int are both 32-bit, then a likely cause of the problem would come from:

malloc(dNum2 * sizeof(int));

If the multiplication overflows SIZE_MAX (which is 4 gigabytes on the 32-bit system) then it will actually wrap around to zero and allocate a small amount; and then your loop will run off the end of the allocated space.

To detect this situation, and also fix the type mismatch bug by using sizeof in the recommended way, and detect allocation failure:

if ( dnum1 >= SIZE_MAX / sizeof *firstSet 
  || dnum2 >= SIZE_MAX / sizeof *secondSet )
{
    fprintf(stderr, "Allocation too big.\n");
    exit(EXIT_FAILURE);
}

long int* firstSet = malloc(dNum1 * sizeof *firstSet);
long int* secondSet = malloc(dNum2 * sizeof *secondSet);

if ( !firstSet || !secondSet )
{
    fprintf(stderr, "Not enough system memory available.\n");
    exit(EXIT_FAILURE);
}

Finally, even if this all succeeds then another possible cause of a crash is due to lazy allocation: many operating systems will return a valid pointer but not actually allocate memory yet. Then when you go to use the memory they will allocate some virtual memory. If there is not enough virtual memory then the OS kills the process.

---

Related Topics