Difference Between Shared Objects (.So), Static Libraries (.A), and Dll's (.So)

Difference between shared objects (.so), static libraries (.a), and DLL's (.so)?

I've always thought that DLLs and shared objects are just different terms for the same thing - Windows calls them DLLs, while on UNIX systems they're shared objects, with the general term - dynamically linked library - covering both (even the function to open a .so on UNIX is called dlopen() after 'dynamic library').

They are indeed only linked at application startup, however your notion of verification against the header file is incorrect. The header file defines prototypes which are required in order to compile the code which uses the library, but at link time the linker looks inside the library itself to make sure the functions it needs are actually there. The linker has to find the function bodies somewhere at link time or it'll raise an error. It ALSO does that at runtime, because as you rightly point out the library itself might have changed since the program was compiled. This is why ABI stability is so important in platform libraries, as the ABI changing is what breaks existing programs compiled against older versions.

Static libraries are just bundles of object files straight out of the compiler, just like the ones that you are building yourself as part of your project's compilation, so they get pulled in and fed to the linker in exactly the same way, and unused bits are dropped in exactly the same way.

Difference between static and shared libraries?

Shared libraries are .so (or in Windows .dll, or in OS X .dylib) files. All the code relating to the library is in this file, and it is referenced by programs using it at run-time. A program using a shared library only makes reference to the code that it uses in the shared library.

Static libraries are .a (or in Windows .lib) files. All the code relating to the library is in this file, and it is directly linked into the program at compile time. A program using a static library takes copies of the code that it uses from the static library and makes it part of the program. [Windows also has .lib files which are used to reference .dll files, but they act the same way as the first one].

There are advantages and disadvantages in each method:

• Shared libraries reduce the amount of code that is duplicated in each program that makes use of the library, keeping the binaries small. It also allows you to replace the shared object with one that is functionally equivalent, but may have added performance benefits without needing to recompile the program that makes use of it. Shared libraries will, however have a small additional cost for the execution of the functions as well as a run-time loading cost as all the symbols in the library need to be connected to the things they use. Additionally, shared libraries can be loaded into an application at run-time, which is the general mechanism for implementing binary plug-in systems.

• Static libraries increase the overall size of the binary, but it means that you don't need to carry along a copy of the library that is being used. As the code is connected at compile time there are not any additional run-time loading costs. The code is simply there.

Personally, I prefer shared libraries, but use static libraries when needing to ensure that the binary does not have many external dependencies that may be difficult to meet, such as specific versions of the C++ standard library or specific versions of the Boost C++ library.

Architecturally what is the difference between a shared object (SO) and a dynamic link library (DLL)?

A Dll is pretty much the same mechanism as used by .so or .dylib (MacOS) files, so it is very hard to explain exactly what the differences are.

The core difference is in what is visible by default from each type of file. .so files export the language (gcc) level linkage - which means that (by default) all C & c++ symbols that are "extern" are available for linking when .so's are pulled in.
It also means that, as resolving .so files is essentially a link step, the loader doesn't care which .so file a symbol comes from. It just searches the specified .so files in some order following the usual link step rules that .a files adhere to.

Dll files on the other hand are an Operating system feature, completely separate to the link step of the language. MSVC uses .lib files for linking both static, and dynamic libraries (each dll file generates a paired .lib file that is used for linking) so the resulting program is fully "linked" (from a language centric point of view) once its built.

During the link stage however, symbols were resolved in the lib's that represents the Dlls, allowing the linker to build the import table in the PE file containing an explicit list of dlls and the entry points referenced in each dll. At load time, Windows does not have to perform a "link" to resolving symbols from shared libraries: That step was already done - the windows loader just loads up the dll's and hooks up the functions directly.

File format differences between a static library (.a) and a shared library (.so)?

A static library, e.g. libfoo.a is not an executable of any kind.
It is simply an indexed archive in unix ar format
of other files which happen to be ELF
object files.

A static library is created like any archive:

ar crs libfoo.a objfile0.o objfile1.0...objfileN.o

outputs the new archive (c) libfoo.a, with those object files inserted (r)
and index added (s).

You'll hear of linking libfoo.a in a program. This doesn't mean that
libfoo.a itself is linked into or with the program. It means that libfoo.a
is passed to the linker as an archive from which it can extract and link into
the program just those object files within the archive that the program needs.
So the format of a static libary (ar format) is just an object-file
bundling format for linker input: it could equally well have been some other bundling
format without any effect on the linker's mission, which is to digest a set of
object files and shared libraries and generate a program, or shared library,
from them. ar format was history's choice.

On the other hand a shared library, e.g. libfoo.so, is an ELF file
and not any sort of archive.

Don't be tempted to suspect that a static library is a sort of ELF file by
the fact that all the well-known ELF-parsers - objdump, readelf, nm -
will parse a static libary. These tools all know that a static library is
an archive of ELF object files, so they just parse all the object files
in the library as if you had listed them on the commandline.

The use of the -D option with nm just instructs the tool to select
only the symbols that are in the dynamic symbol table(s), if any,
of the ELF file(s) that it parses - the symbols visible to the runtime linker
- regardless of whether or not they are parsed from within an archive. It's
the same as objdump -T and readelf --dyn-syms. It is not
necessary to use these options to parse the symbols from a shared library. If
you don't do so, then by default you'll just see the full symbol table.
If you run nm -D on a static library you'll be told no symbols, for
each object file in the archive - likewise if you ran nm -D for each of
those object files individually. The reason for that is that an object file
hasn't got a dynamic symbol table: only a shared library or progam has one.

Object file, shared library and program are all variants of the ELF format.
If you're interested in ELF variants, those are the variants of interest.

The ELF format itself is a long and thorny technical read and is required
background for precisely distinguishing the variants. Intro: An ELF file
contains a ELF header structure one of whose fields contains a type-identifier
of the file as an object file, shared library, or program. When the file is a
program or shared library, it also contains an optional Program header table
structure whose fields provide the runtime linker/loader with the parameters
it needs to load the file in a process. In terms of ELF structure,
the differences between a program and a shared library are slight: it's
the detailed content that makes the difference to the behaviour that they
elicit from the loader.

For the long and thorny technical read, try Excutable and Linkable Format (ELF)

what is the difference between .so and .a files?

But it appears that even .a is shared library

Nope, it's a static library.

and can be used just like a .so lib

If you mean linking to it, then yes. But you can't dlopen() an .a file which you could do with an .so file.

You can always ask our old friend Uncle G to answer your questions.

Why are shared and static libraries different things?

So I see lots of answers talking about why you would want to use shared libraries instead of static libraries, but I think your question is why they are even distinct things nowadays, i.e. why isn't it possible to use a shared library as a static library and pull what you need out of it at build time?

Here are some reasons. Some of these are historical - keep in mind that something as fundamental as binary formats changes very slowly in computer systems.

Compiled Differently

Code can be compiled either to be dependent on the address it sits at (position-dependent) or independent (position-independent). This affects things like loads of global constants, function calls, etc. Position-dependent code needs fixups if it isn't loaded at the address it expects, i.e. the loader has to go over the code and actually change offsets.

For executables, this isn't a problem. An executable is the first thing that is loaded into the address space, so it will always be loaded at the same address. You generally don't need any fixups. But a shared library is used by different executables, by different processes. Multiple libraries can conflict: if they expect to be at overlapping address ranges, one will have to budge. When it does, and it is position-dependent, it needs to be fixed by the loader. But now you have process-specific changes in the library code, which means the code can't be shared (at runtime) with other processes anymore. You lose one of the big benefits of shared libraries.

If the shared library uses position-independent code (PIC), it doesn't need fixups. So PIC is good for shared libraries. On the other hand, PIC is slower on some architectures (notably x86, but not x64), so compiling executables as PIC is a waste of resources.

Executables were therefore usually compiled as position-dependent code, while shared libraries were compiled as position-independent code. If you used shared libraries as sources for code directly pulled into executables, you get PIC. If you want PDC, you need a separate code repository, and that's a static library.

Of course, on most modern architectures, PIC isn't less efficient than PDC, and security techniques like address space randomization make it useful to compile executables as PIC too, so this is more of a historical reason than a current one.

Contain Different Things

But there's another, more current reason for separating static and shared libraries, and that's link-time optimization.

Basically, the more information an optimizer has about a program, the better it can reason about it. Classical optimizer worked on a per-module basis: compile a .c file, optimize it, generate object code. The linker took all the object files and merged them together. This means that the optimizer can only reason about one module at a time. It cannot look into the called functions that are outside the module in order to reason about them, or even simply inline them.

In modern toolchains, however, the compiler often works differently. Instead of compiling and optimizing a module and then producing object code, it takes a module, produces an intermediate form, possibly optimizes it a bit, and then puts the intermediate form into the object file. The linker, instead of just merging object files and resolving references, actually merges the intermediate representation and then invokes the optimizer and code generator on the merged form. With much more information available, the optimizer can do a vastly better job.

This intermediate representation is more detailed, more faithful to the original code than machine code. You want this for your compilation process. You don't want to ship it to the customer, because it is much bigger, and if you use a closed-source model also because it is much easier to reverse-engineer. Moreover, there's no point in shipping it, because the loader doesn't understand it, and you don't want to re-optimize and recompile your program at startup time anyway (JIT languages aside).

Thus, a shared library contains real object code. A static library, on the other hand, is a good container for intermediate code, because it is consumed by the linker. This is a key difference between static and shared libraries.

Linkage Model

Finally, we have another semi-historical reason: linkage.

Linkage defines how a symbol (a variable or function name) is visible outside a code unit. The C language defines two linkages: internal (not visible outside the compilation unit, i.e. static) and external (visible to the whole program, i.e. extern). You generally have a lot of externally visible symbols.

Shared libraries, however, have their symbols resolved at load time, and this should be fast. Fewer symbols means lookup in the symbol table is faster. Of course this was more relevant when computers were slower, but it still can have a noticeable effect. It also affects the size of the libraries.

Therefore, object file specifications used by the operating systems (ELF for *nix, PE/COFF for Windows) defined separate visibilities for shared libraries. Instead of making everything that's external in C visible, you have the option to specify the visible functions explicitly. (In Windows, only things annotated as __declspec(dllexport), or listed in a .def file are exported from a DLL. In Linux, everything extern is exported by default, but you can use __attribute__((visibility("hidden"))) to not do that, or you can specify the -fvisibility=hidden command line switch or the visibility pragma to override the default.)

The end result is that a shared library throws away all symbol information except for the exported functions.

A static library has no need to throw away any symbol information. What's more, you don't want to do that, because carefully specifying which functions are exported and which aren't is some work, and you don't want to have to do that work unless necessary. If you're using static libraries, it isn't necessary.

So a shippable shared library should minimize its exported symbols in order to be fast and small. This makes it less useful as a code repository for static linking, where you may want a greater selection of functions to link in, especially once the interface functions get inlined (see link-time optimization above).

When to use dynamic vs. static libraries

Static libraries increase the size of the code in your binary. They're always loaded and whatever version of the code you compiled with is the version of the code that will run.

Dynamic libraries are stored and versioned separately. It's possible for a version of the dynamic library to be loaded that wasn't the original one that shipped with your code if the update is considered binary compatible with the original version.

Additionally dynamic libraries aren't necessarily loaded -- they're usually loaded when first called -- and can be shared among components that use the same library (multiple data loads, one code load).

Dynamic libraries were considered to be the better approach most of the time, but originally they had a major flaw (google DLL hell), which has all but been eliminated by more recent Windows OSes (Windows XP in particular).

is using shared libraries is not always better than using a static libraries?

Yes but... if your app is the only one on the system using that library you should use static library whatsoever.

The shared libraries save memory because they are shared; it is possible to organize them so that for each shared library on a big server, only one copy exists in ram even though 10 different exes for 100 different users are using it. This is not possible static libraries; if you have 10 different exes using the same library then you might have 10 copies in RAM.

---

Related Topics