When to Use Dynamic Vs. Static Libraries

When to use dynamic vs. static libraries

Static libraries increase the size of the code in your binary. They're always loaded and whatever version of the code you compiled with is the version of the code that will run.

Dynamic libraries are stored and versioned separately. It's possible for a version of the dynamic library to be loaded that wasn't the original one that shipped with your code if the update is considered binary compatible with the original version.

Additionally dynamic libraries aren't necessarily loaded -- they're usually loaded when first called -- and can be shared among components that use the same library (multiple data loads, one code load).

Dynamic libraries were considered to be the better approach most of the time, but originally they had a major flaw (google DLL hell), which has all but been eliminated by more recent Windows OSes (Windows XP in particular).

What is the difference between a static library and a dynamic one

This concept might be a little bit too broad to explain, but i will try to give you a basic idea from which you can study further.

Firstly, you need to know what a library is. Basically, a library is a collection of functions. You may have noticed that we are using functions which are not defined in our code, or in that particular file. To have access to them, we include a header file, that contains declarations of those functions. After compile, there is a process called linking, that links those function declarations with their definitions, which are in another file. The result of this is the actual executable file.

Now, the linking as I described it is a static linking. This means that every executable file contains in it every library (collection of functions) that it needs. This is a waste of space, as there are many programs that may need the same functions. In this case, in memory there would be more copies of the same function. Dynamic linking prevents this, by linking at the run-time, not at the compile time. This means that all the functions are in a special memory space and every program can access them, without having multiple copies of them. This reduces the amount of memory required.

As I mentioned at the beginning of my answer, this is a very simplified summary to give you a basic understanding. I strongly suggest you study more on this topic.

Differences between static libraries and dynamic libraries ignoring how they are used by the linker/loader

It's a pity the terms static library and dynamic library are
both of the form ADJECTIVE library, because it perpetually leads programmers
to think that they denote variants of the essentially the same kind of thing.
This is almost as misleading as the thought that a badminton court and a supreme court
are essentially the same kind of thing. In fact it's far more misleading,
since nobody actually suffers from thinking that a badminton court and a supreme
court are essentially the same kind of thing.

Can someone throw some light on the differences between the contents of static and shared library files?

Let's use examples. To push back against the badminton court / supreme court fog
I'm going to use more accurate technical terms. Instead of static library I'll say ar archive, and instead of dynamic library I'll say
dynamic shared object, or DSO for short.

What an ar archive is

I'll make an ar archive starting with these three files:

foo.c

#include <stdio.h>

void foo(void)
{
    puts("foo");
}

bar.c

#include <stdio.h>

void bar(void)
{
    puts("bar");
}

limerick.txt

There once was a young lady named bright
Whose speed was much faster than light
She set out one day
In a relative way
And returned on the previous night.

I'll compile those two C source into Position Independent object files:

$ gcc -c -Wall -fPIC foo.c
$ gcc -c -Wall -fPIC bar.c

There's no need for object files destined for an ar archive to be compiled with
-fPIC. I just want these ones compiled that way.

Then I'll create an ar archive called libsundry.a containing the object files foo.o and bar.o,
plus limerick.txt:

$ ar rcs libsundry.a foo.o bar.o limerick.txt

An ar archive is created, of course, with ar,
the GNU general-purpose archiver. So it is not created by the linker. No linkage
happens. Here's how ar reports the contents of the archive:

$ ar -t libsundry.a 
foo.o
bar.o
limerick.txt

Here's what the limerick in the archive looks like:

$ rm limerick.txt
$ ar x libsundry.a limerick.txt; cat limerick.txt
There once was a young lady named bright
Whose speed was much faster than light
She set out one day
In a relative way
And returned on the previous night.

Q. What's the point of putting two object files and an ASCII limerick into the same ar archive?

A. To show that I can. To show that an ar archive is just a bag of files.

Let's see what file makes of libsundry.a.

$ file libsundry.a 
libsundry.a: current ar archive

Now I'll write a couple of programs that use libsundry.a in their linkage.

fooprog.c

extern void foo(void);

int main(void)
{
    foo();
    return 0;
}

Compile, link and run that one:

$ gcc -c -Wall fooprog.c
$ gcc -o fooprog fooprog.o -L. -lsundry
$ ./fooprog 
foo

That's hunky dory. The linker apparently wasn't bothered by the presence of
an ASCII limerick in libsundry.a.

The reason for that is the linker didn't even try to link limerick.txt
into the program. Let's do the linkage again, this time with a diagnostic option
that will show us exactly what input files are linked:

$ gcc -o fooprog fooprog.o -L. -lsundry -Wl,-trace
/usr/bin/ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o
fooprog.o
(./libsundry.a)foo.o
-lgcc_s (/usr/lib/gcc/x86_64-linux-gnu/5/libgcc_s.so)
/lib/x86_64-linux-gnu/libc.so.6
(/usr/lib/x86_64-linux-gnu/libc_nonshared.a)elf-init.oS
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
-lgcc_s (/usr/lib/gcc/x86_64-linux-gnu/5/libgcc_s.so)
/usr/lib/gcc/x86_64-linux-gnu/5/crtend.o
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o

Lots of default libraries and object files there, but the only object
files we have created that the linker consumed are:

fooprog.o
(./libsundry.a)foo.o

All that the linker did with ./libsundry.a was take foo.o out of
the bag and link it in the program. After linking fooprog.o into the program,
it needed to find a definition for foo.
It looked in the bag. It found the definition in foo.o, so it took foo.o from
the bag and linked it in the program. In linking fooprog,

gcc -o fooprog fooprog.o -L. -lsundry

is exactly the same linkage as:

$ gcc -o fooprog fooprog.o foo.o

What does file say about fooprog?

$ file fooprog
fooprog: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), \
dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, \
for GNU/Linux 2.6.32, BuildID[sha1]=32525dce7adf18604b2eb5af7065091c9111c16e,
not stripped

Here's my second program:

foobarprog.c

extern void foo(void);
extern void bar(void);

int main(void)
{
    foo();
    bar();
    return 0;
}

Compile, link and run:

$ gcc -c -Wall foobarprog.c
$ gcc -o foobarprog foobarprog.o -L. -lsundry
$ ./foobarprog 
foo
bar

And here's the linkage again with -trace:

$ gcc -o foobarprog foobarprog.o -L. -lsundry -Wl,-trace
/usr/bin/ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o
foobarprog.o
(./libsundry.a)foo.o
(./libsundry.a)bar.o
-lgcc_s (/usr/lib/gcc/x86_64-linux-gnu/5/libgcc_s.so)
/lib/x86_64-linux-gnu/libc.so.6
(/usr/lib/x86_64-linux-gnu/libc_nonshared.a)elf-init.oS
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
-lgcc_s (/usr/lib/gcc/x86_64-linux-gnu/5/libgcc_s.so)
/usr/lib/gcc/x86_64-linux-gnu/5/crtend.o
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o

So this time, our object files that the linker consumed were:

foobarprog.o
(./libsundry.a)foo.o
(./libsundry.a)bar.o

After linking foobarprog.o into the program, it needed to find definitions for foo and bar.
It looked in the bag. It found definitions respectively in foo.o and bar.o, so it took them from
the bag and linked them in the program. In linking foobarprog,

gcc -o foobarprog foobarprog.o -L. -lsundry

is exactly the same linkage as:

$ gcc -o foobarprog foobarprog.o foo.o bar.o

Summing all that up. An ar archive is just a bag of files. You can use
an ar archive to offer to the linker a bunch of object files from which to
pick the ones that it needs to continue the linkage. It will take those object files
out of the bag and link them into the output file. It has absolutely no other
use for the bag. The bag contributes nothing at all to the linkage.

The bag just makes your life a little simpler by sparing you the need to know
exactly what object files you need for a particular linkage. You only need
to know: Well, they're in that bag.

What a DSO is

Let's make one.

foobar.c

extern void foo(void);
extern void bar(void);

void foobar(void)
{
    foo();
    bar();
}

We'll compile this new source file:

$ gcc -c -Wall -fPIC foobar.c

and then make a DSO using foobar.o and re-using libsundry.a

$ gcc -shared -o libfoobar.so foobar.o -L. -lsundry -Wl,-trace
/usr/bin/ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/5/crtbeginS.o
foobar.o
(./libsundry.a)foo.o
(./libsundry.a)bar.o
-lgcc_s (/usr/lib/gcc/x86_64-linux-gnu/5/libgcc_s.so)
/lib/x86_64-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
-lgcc_s (/usr/lib/gcc/x86_64-linux-gnu/5/libgcc_s.so)
/usr/lib/gcc/x86_64-linux-gnu/5/crtendS.o
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o

That has made the DSO libfoobar.so. Notice: A DSO is made by the linker. It
is linked just like a program is linked. The linkage of libfoopar.so looks very much
like the linkage of foobarprog, but the addition of the option
-shared instructs the linker to produce a DSO rather than a program. Here we see that our object
files consumed by the linkage were:

foobar.o
(./libsundry.a)foo.o
(./libsundry.a)bar.o

ar does not understand a DSO at all:

$ ar -t libfoobar.so 
ar: libfoobar.so: File format not recognised

But file does:

$ file libfoobar.so 
libfoobar.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), \
dynamically linked, BuildID[sha1]=16747713db620e5ef14753334fea52e71fb3c5c8, \
not stripped

Now if we relink foobarprog using libfoobar.so instead of libsundry.a:

$ gcc -o foobarprog foobarprog.o -L. -lfoobar -Wl,-trace,--rpath=$(pwd)
/usr/bin/ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o
foobarprog.o
-lfoobar (./libfoobar.so)
-lgcc_s (/usr/lib/gcc/x86_64-linux-gnu/5/libgcc_s.so)
/lib/x86_64-linux-gnu/libc.so.6
(/usr/lib/x86_64-linux-gnu/libc_nonshared.a)elf-init.oS
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
-lgcc_s (/usr/lib/gcc/x86_64-linux-gnu/5/libgcc_s.so)
/usr/lib/gcc/x86_64-linux-gnu/5/crtend.o
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o

we see

foobarprog.o
-lfoobar (./libfoobar.so)

that ./libfoobar.so itself was linked. Not some object files "inside it". There
aren't any object files inside it. And how this has
contributed to the linkage can be seen in the dynamic dependencies of the program:

$ ldd foobarprog
    linux-vdso.so.1 =>  (0x00007ffca47fb000)
    libfoobar.so => /home/imk/develop/so/scrap/libfoobar.so (0x00007fb050eeb000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb050afd000)
    /lib64/ld-linux-x86-64.so.2 (0x000055d8119f0000)

The program has come out with runtime dependency on libfoobar.so. That's what linking a DSO does.
We can see this runtime dependency is satisfied. So the program will run:

$ ./foobarprog
foo
bar

just the same as before.

The fact that a DSO and a program - unlike an ar archive - are both products
of the linker suggests that a DSO and a program are variants of the essentially the same kind of thing.
The file outputs suggested that too. A DSO and a program are both ELF binaries
that the OS loader can map into a process address space. Not just a bag of files.
An ar archive is not an ELF binary of any kind.

The difference between a program-type ELF file and non-program-type ELF lies in the different values
that the linker writes into the ELF Header structure and Program Headers
structure of the ELF file format. These differences instruct the OS loader to
initiate a new process when it loads a program-type ELF file, and to augment
the process that it has under construction when it loads a non-program ELF file. Thus
a non-program DSO gets mapped into the process of its parent program. The fact that a program
initiates a new process requires that a program shall have single default entry point
to which the OS will pass control: that entry point is the mandatory main function
in a C or C++ program. A non-program DSO, on the other hand, doesn't need a single mandatory entry point. It can be entered through any of the global functions it exports by function calls from the
parent program.

But from the point of view of file structure and content, a DSO and a program
are very similar things. They are files that can be components of a process.
A program must be the initial component. A DSO can be a secondary component.

It is still common for the further distinction to be made: A DSO must consist entirely
of relocatable code (because there's no knowing at linktime where the loader may need to
place it in a process address space), whereas a program consists of absolute code,
always loaded at the same address. But in fact its quite possible to link a relocatable
program:

$ gcc -pie -o foobarprog foobarprog.o -L. -lfoobar -Wl,--rpath=$(pwd)

That's what -pie (Position Independent Executable) does here. And then:

$ file foobarprog
foobarprog: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), ....

file will say that foobarprog is a DSO, which it is, although it is
also still a program:

$ ./foobarprog
foo
bar

And PIE executables are catching on. In Debian 9 and derivative distros (Ubuntu 17.04...) the GCC toolchain
builds PIE programs by default.

If you hanker for detailed knowledge of the ar and ELF file
formats, here are details of the ar format
and here are details of the ELF format.

why not have a single type of library file accompanied by compiler flags which
indicate how the library should be linked (static vs dynamic)?

The choice between dynamic and static linkage is already fully controllable by
commandline linkage options, so there's no need abandon either ar archives or DSOs or to invent a another kind
of library to achieve this. If the linker couldn't use ar archives the way it does,
that would be a considerable inconvenience. And of course if the linker couldn't link
DSOs we'd back to the operating systems stone-age.

Static Vs Dynamic libraries

Unfortunately, the words "static" and "dynamic" are way too overused, especially in C and C++. So, I prefer the following terminology:

Link-time linking, a.k.a "static linking": All symbols are resolved at link time from static libraries. The result is a monolithic, statically linked executable with no load-time dependencies.
Load-time linking: This is the standard practice on modern platforms, unresolved symbols are looked up in shared libraries (Unix) or the unfortunately named dynamic link libraries (DLLS) on Windows and only references are recorded at link time, the actual resolution of the symbols and code loading happens at load time.
This results in a "dynamically linked" executable which must be loaded with a loader (e.g. ld.so on Linux). Loading is part of the OS and usually transparent to the user, though it's open to inspection (e.g. with ldd on Linux). All shared libraries must be available at load time, or the program will not launch.
Run-time linking, a.k.a. "dynamic linking": There are no unresolved symbols; rather, the runtime dynamically decides to look up symbols in a shared/dynamic library using dlopen() or LoadLibrary(). Failure to find symbols is a handlable runtime condition that is not an error. This technique is used commonly for plug-in architecture, and on Windows for code injection.

Note however that there is a fundamental technical difference between Linux's shared objects and Windows's DLLs, they're not just the same thing with a different name. Both can however be used both for load-time and run-time linking.

Dynamic and Static Libraries in C++

Is writing them any different than a normal C++ program, minus the main() function?

No.

How does the compiled program get to be a library? It's obviously not an executable, so how do I turn, say 'test.cpp' into 'test.dll'?

Pass the -dynamiclib flag when you're compiling. (The name of the result is still by default a.out. On Mac OS X you should name your dynamic libraries as lib***.dylib, and on Linux, lib***.so (shared objects))

Once I get it to its format, how do I include it in another program?

First, make a header file so the the other program can #include to know what functions can be used in your dylib.

Second, link to your dylib. If your dylib is named as libblah.dylib, you pass the -lblah flag to gcc.

Is there a standard place to put them, so that whatever compilers/linkers need them can find them easily?

/usr/lib or /usr/local/lib.

What is the difference (technically and practically) between a dynamic and static library?

Basically, for a static lib, the whole library is embedded into the file it "links" to.

How would I use third party libraries in my code (I'm staring at .dylib and .a files for the MySql C++ Connector)

See the 3rd answer.

What is static library and what is dynamic library, what is the difference and what is better to use and why?

A static library is meant to be combined with your code into a single executable file by a linker.

A dynamic library is meant to be loaded by the operating system after the main executable has been loaded, and the linking of the symbol addresses will be done by the OS at that time. This may be done automatically based on dependency information in the executable, or it may be done explicitly by the program. This is called "dynamic linking" because the library may change at any point before the OS has loaded it.

dynamic library vs static library at runtime

It depends on your OS, and the options you use to compile and
link. Under Unix, by default, all common symbols in two or more
shared libraries will resolve to the names in the first library
loaded; for most Unices, this applies to names in the main as
well (but the GNU linker used in Linux requires a special option
for this). Under Windows, it's a bit trickier; each statically
linked library will have its own copy of the state, and there's
no easy work-around, other than creating a third DLL to wrap the
static library, and only accessing it through symbols in that
DLL.

Static linking vs dynamic linking

Dynamic linking can reduce total resource consumption (if more than one process shares the same library (including the version in "the same", of course)). I believe this is the argument that drives its presence in most environments. Here "resources" include disk space, RAM, and cache space. Of course, if your dynamic linker is insufficiently flexible there is a risk of DLL hell.
Dynamic linking means that bug fixes and upgrades to libraries propagate to improve your product without requiring you to ship anything.
Plugins always call for dynamic linking.
Static linking, means that you can know the code will run in very limited environments (early in the boot process, or in rescue mode).
Static linking can make binaries easier to distribute to diverse user environments (at the cost of sending a larger and more resource-hungry program).
Static linking may allow slightly faster startup times, but this depends to some degree on both the size and complexity of your program and on the details of the OS's loading strategy.

Some edits to include the very relevant suggestions in the comments and in other answers. I'd like to note that the way you break on this depends a lot on what environment you plan to run in. Minimal embedded systems may not have enough resources to support dynamic linking. Slightly larger small systems may well support dynamic linking because their memory is small enough to make the RAM savings from dynamic linking very attractive. Full-blown consumer PCs have, as Mark notes, enormous resources, and you can probably let the convenience issues drive your thinking on this matter.

To address the performance and efficiency issues: it depends.

Classically, dynamic libraries require some kind of glue layer which often means double dispatch or an extra layer of indirection in function addressing and can cost a little speed (but is the function calling time actually a big part of your running time???).

However, if you are running multiple processes which all call the same library a lot, you can end up saving cache lines (and thus winning on running performance) when using dynamic linking relative to using static linking. (Unless modern OS's are smart enough to notice identical segments in statically linked binaries. Seems hard, does anyone know?)

Another issue: loading time. You pay loading costs at some point. When you pay this cost depends on how the OS works as well as what linking you use. Maybe you'd rather put off paying it until you know you need it.

Note that static-vs-dynamic linking is traditionally not an optimization issue, because they both involve separate compilation down to object files. However, this is not required: a compiler can in principle, "compile" "static libraries" to a digested AST form initially, and "link" them by adding those ASTs to the ones generated for the main code, thus empowering global optimization. None of the systems I use do this, so I can't comment on how well it works.

The way to answer performance questions is always by testing (and use a test environment as much like the deployment environment as possible).

When to Use Dynamic Vs. Static Libraries