What's the Rationale For Null Terminated Strings

What's the rationale for null terminated strings?

From the horse's mouth

None of BCPL, B, or C supports
character data strongly in the
language; each treats strings much
like vectors of integers and
supplements general rules by a few
conventions. In both BCPL and B a
string literal denotes the address of
a static area initialized with the
characters of the string, packed into
cells. In BCPL, the first packed byte
contains the number of characters in
the string; in B, there is no count
and strings are terminated by a
special character, which B spelled
*e. This change was made partially
to avoid the limitation on the length
of a string caused by holding the
count in an 8- or 9-bit slot, and
partly because maintaining the count
seemed, in our experience, less
convenient than using a terminator.

_{Dennis M Ritchie, Development of the C Language}

Why do strings in C need to be null terminated?

From Joel's excellent article on the topic:

Remember the way strings work in C: they consist of a bunch of bytes followed by a null character, which has the value 0. This has two obvious implications:

There is no way to know where the string ends (that is, the string length) without moving through it, looking for the null character at the end.
Your string can't have any zeros in it. So you can't store an arbitrary binary blob like a JPEG picture in a C string.
Why do C strings work this way? It's because the PDP-7 microprocessor, on which UNIX and the C programming language were invented, had an ASCIZ string type. ASCIZ meant "ASCII with a Z (zero) at the end."

Is this the only way to store strings? No, in fact, it's one of the worst ways to store strings. For non-trivial programs, APIs, operating systems, class libraries, you should avoid ASCIZ strings like the plague.

What are null-terminated strings?

What are null-terminating strings?

In C, a "null-terminated string" is a tautology. A string is, by definition, a contiguous null-terminated sequence of characters (an array, or a part of an array). Other languages may address strings differently. I am only discussing C strings.

How are they different from a non-null-terminated strings?

There are no non-null-terminated strings in C. A non-null-terminated array of characters is just an array of characters.

What is this null that terminates the string? Is it different from NULL?
The "null character" is a character with the integer value of zero. (Characters are, in essence, small integers). It is sometimes, especially in the context of ASCII, referred to as NUL (single L). This is distinct from NULL (double L), which is a null pointer. The null character can be written as '\0' or just 0 in the source code. The two forms are interchangeable in C (but not in C++). The former is usually preferred because it shows the intent better.

Should I null-terminate my strings myself, or the compiler will do it for me?

If you are writing a string literal, you don't need to explicitly insert a null character in the end. The compiler will do it.

char* str1 = "a string";   // ok, \0 is inserted automatically
char* str2 = "a string\0"; // extra \0 is not needed

The compiler will not insert a null character when declaring an array with an explicit size and initialising it with a string literal with more characters than the array can hold.

char str3[5] = "hello"; // not enough space in the array for the null terminator
char str4[]  = "hello"; // ok, there is \0 in the end, the total size is 6

The compiler will not insert a null character when declaring an array and not initialising it with a string literal.

char str5[] = { 'h', 'e', 'l', 'l', 'o' };       // no null terminator
char str6[] = { 'h', 'e', 'l', 'l', 'o', '\0' }; // null terminator

If you are building a string at run-time out of some data that comes from IO or from a different part of the program, you need to make sure a null terminator is inserted. Standard library functions such as fread and POSIX functions such as read never null-terminate their arguments. strncpy will add a null-terminator if there is enough space for it, so use it with care. Confusingly, strncat will always add a null-terminator.

Why are null-terminated strings needed?

Many functions from the standard C library, and many functions from third-party libraries, operate on strings (and all strings need to be null-terminated). If you pass a non-null-terminated character array to a function that expects a string, the results are likely to be undefined. So if you want to interoperate with the world around you, you need null-terminated strings. If you never use any standard-library or third-party functions that expect string arguments, you may do what you want.

How do I set up my code/data to handle null-terminated strings?

If you plan to store strings of length up to N, allocate N+1 characters for your data. The character needed for the null terminator is not included the length of the string, but it is included in the size of the array required to store it.

What is a null-terminated string?

A null-terminated string is a contiguous sequence of characters, the last one of which has the binary bit pattern all zeros. I'm not sure what you mean by a "usual string", but if you mean std::string, then a std::string is not required (until C++11) to be contiguous, and is not required to have a terminator. Also, a std::string's string data is always allocated and managed by the std::string object that contains it; for a null-terminated string, there is no such container, and you typically refer to and manage such strings using bare pointers.

All of this should really be covered in any decent C++ text book - I recommend getting hold of Accelerated C++, one of the best of them.

Why do we need a null terminator in C++ strings?

The other characters in the array char john[100] = "John" would be filled with zeros, which are all null-terminators. In general, when you initialize an array and don't provide enough elements to fill it up, the remaining elements are default-initialized:

int foo[3] {5};           // this is {5, 0, 0}
int bar[3] {};            // this is {0, 0, 0}

char john[5] = "John";    // this is {'J', 'o', 'h', 'n', 0}
char peter[5] = "Peter";  // ERROR, initializer string too long
                          // (one null-terminator is mandatory)

Also see cppreference on Array initialization. To find the length of such a string, we just loop through the characters until we find 0 and exit.

The motivation behind null-terminating strings in C++ is to ensure compatibility with C-libraries, which use null-terminated strings. Also see What's the rationale for null terminated strings?

Containers like std::string don't require the string to be null-terminated and can even store a string containing null-characters. This is because they store the size of the string separately. However, the characters of a std::string are often null-terminated anyways so that std::string::c_str() doesn't require a modification of the underlying array.

C++-only libraries will rarely -if ever- pass C-strings between functions.

Why null-terminated strings? Or: null-terminated vs. characters + length storage

The usual solution is to do both - keep the length and maintain the null terminator. It's not much extra work and means that you are always ready to pass the string to any function.

Null-terminated strings are often a drain on performance, for the obvious reason that the time taken to discover the length depends on the length. On the plus side, they are the standard way of representing strings in C, so you have little choice but to support them if you want to use most C libraries.

Why is integer value of the (n+1)th term in a string is 0?

C-strings are null terminated. That's why when you print it like an integer it gives 0.

If you do:

if(b[5] == '\0')
    printf("null detected\n");

then "null detected" will be printed.

The reason is that strings are passed as pointers and their size is not known/passed. So all functions use the null character to detect the end, as commented by @Downloadpizza.

You should not access the memory past the null terminator, since this is like accessing an array out of bounds.

How does this work if c strings are null terminated?

Other answers in this thread have mentioned this, but have not provided resources to help you better understand the issue. So, that is what I will do here.

You have created your first buffer overflow error. This is because C does not have memory safety, meaning that there no restrictions on the boundaries of memory unless explicitly implemented by the user.

In your case, the function scanf("%s",name); gets the data from the user and puts it in the buffer. This function does not check memory boundaries. This means that if the input size is larger than the allocated buffer you are writing outside your allocated memory. In many cases, it is harmless. However, it can be the source of really hard to find bugs and some vulnerabilities. The Wikipedia article about buffer overflow is excellent and explains everything in a lot of detail, which should help you get a better understanding.

The function that you used to read the length of the string strlen(name), also does not do a boundary check. If you look at the man page of strlen, you can see that it does not mention anything about memory boundaries. Instead, it states that it calculates the length between the start of the string and the next terminating null byte.

The strlen() function calculates the length of the string pointed
to by s, excluding the terminating null byte ('\0').

So, why does your application work?

You have a function that does not care about memory boundaries when writing, and you have a function that does not care about memory boundaries when reading.

How can you prevent it from happening?

Luckily for you, you are not the first with this problem. If you are interested in understanding how you can prevent this from happening, I recommend you check out this post How to prevent scanf causing a buffer overflow in C? . If you are a beginner, I wouldn't recommend you implement these functions, as some are quite complicated when you are just getting started. However, you can look through the post and see what people are posting and find out more about things mentioned in that post.

Why does a C++ string end in a null terminator?

Traditional strings in C and C++ use a null terminator to indicate the end of the string. Since string pointers simply pointed to an array of characters, without any length or other meta data, the null terminator was the only way to determine the length of the string.

As far as why it was done this way, that's more difficult question to answer. There are many ways to store string data, that's just one of them.

What is the different between a null terminated string and a string that is not terminated by null in x86 assembly language

There's nothing specific to asm here; it's the same issue in C. It's all about how you store strings in memory and keep track of where they end.

what is the different between a null terminated string and a string that is not terminated by null?

A null-terminated string has a 0 byte after it, so you can find the end with strlen. (e.g. with a slow repne scasb). This makes is usable as an implicit-length string, like C uses.

NASM Assembly - what is the ", 0" after this variable for? explains the NASM syntax for creating one in static storage with db. db usage in nasm, try to store and print string shows what happens when you forget the 0 terminator.

Are they interchangeable?

If you know the length of a null-terminated string, you can pass pointer+length to a function that wants an explicit-length string. That function will never look at the 0 byte, because you will pass a length that doesn't include the 0 byte. It's not part of the string data proper.

But if you have a string without a terminator, you can't pass it to a function or system-call that wants a null-terminated string. (If the memory is writeable, you could store a 0 after the string to make it into a null-terminated string.)

---

In Linux, many system calls take strings as C-style implicit-length null-terminated strings. (i.e. just a char* without passing a length).

For example, open(2) takes a string for the path: int open(const char *pathname, int flags); You must pass a null-terminated string to the system call. It's impossible to create a file with a name that includes a '\0' in Linux (same as most other Unix systems), because all the system calls for dealing with files use null-terminated strings.

OTOH, write(2) takes a memory buffer which isn't necessarily a string. It has the signature ssize_t write(int fd, const void *buf, size_t count);. It doesn't care if there's a 0 at buf+count because it only looks at the bytes from buf to buf+count-1.

You can pass a string to write(). It doesn't care. It's basically just a memcpy into the kernel's pagecache (or into a pipe buffer or whatever for non-regular files). But like I said, you can't pass an arbitrary non-terminated buffer as the path arg to open().

Or they are not equivalent of each other?

Implicit-length and explicit-length are the two major ways of keeping track of string data/constants in memory and passing them around. They solve the same problem, but in opposite ways.

Long implicit-length strings are a bad choice if you sometimes need to find their length before walking through them. Looping through a string is a lot slower than just reading an integer. Finding the length of an implicit-length string is O(n), but an explicit-length string is of course O(1) time to find the length. (It's already known!). At least the length in bytes is known, but the length in Unicode characters might not be known, if it's in a variable-length encoding like UTF-8 or UTF-16.

---

Related Topics