Is There a Linker Flag to Force It to Load All Shared Libraries at Start Time

Is there a linker flag to force it to load all shared libraries at start time?

Setting environment variable LD_BIND_NOW = 1 will do that.

Thanks to @skwllsp for the answer.

Automatically executed functions when loading shared libraries

You can define an on-load function for a linux library using the .init mechanism. This is the same as specifying the load-time entry point for a binary (e.g. using something other than main as the entry point for a program).

When linking using ld directly you use the:

-init <function name>

or if you're using cc/gcc to link, you use:

-Wl,-init,<function name>

This is at it's most simple level.

Edit
For destructors/finalizers, you use the .fini mechanism. This operates in the same manner as the init option, and you use:

-fini <function name>

when invoking ld. Availability is limited to the -init option on the Mac OSX platform.

You should also be able to use the __attribute__((constructor)) syntax for gcc:

static void con() __attribute__((constructor));

void con() {
    printf("I'm a constructor\n");
}

Which is probably a more portable way rather than screwing with the linker options. All constructors should be invoked at load-time, but don't depend on the order of their initialization, that leads to insanity and unreproducible bugs that cost time and effort to debug.

Edit 2 The use of the __attribute__((constructor))/__attribute__((destructor)) semantic is the most preferable mechanism for the C/C++ programming language.

For the D programming language you should really use the static module constructor/destructor:

static this() {
    printf("static this for mymodule\n");
}
static ~this() {
    printf("static ~this for mymodule\n");
}

Or the static class constructor:

class Foo {
    static this() {
        printf("static this for Foo\n");
    }
}

This is strongly hinted at in the writing win32 DLLS and in the language specification relating to static constructors/destructors.

Edit 3 You will need to link in a .o that exports constructor/destructor routines, that will allow the use of the static initializers. As all it should do is call Runtime.initialize(), this actually invokes all the static constructors/destructors in the D code.

Stub d code for the initializer (in a file called myshared.d):

import core.runtime;

extern (C) {
    void attach();
    void detach();
}

export void attach() {
    Runtime.initialize();
}

export void detach() {
    Runtime.terminate();
}

Create the .o for this stub:

 dmd -m32 -c myshared.d

Check the names of the attach/detach functions:

nm myshared.o

Shows (among other output):

0000001c S _D8myshared6attachFZv
00000034 S _D8myshared6detachFZv

sample .c code for invoking this (called export.c in this case), we reference the names of the exported routines from the my shared.o file:

extern void D8myshared6attachFZv(void);
extern void D8myshared6detachFZv(void);

void __attach(void) __attribute__((constructor));
void __detach(void) __attribute__((destructor));

void __attach(void)
{
    D8myshared6attachFZv();
}

void __detach(void)
{
    D8myshared6detachFZv();
}

Note that the extern void references need to use the mangled name of the exported function. These must match or the code will not link.

compile the C code using:

gcc -m32 -c export.c

link the .c.o and the .d.o files together using:

cc -o libmyshared.dylib -m32 -shared myshared.o export.o -lphobos2

Assuming that the phobos2 library is in your standard linker search path. The smatterings of -m32 options for the compiler and linker are because the version of the D compiler that I built locally only supported 32bit.

This produces a .dylib that can be linked to. It seems to work based on the limited testing I performed. It looks like support for shared objects/dynamic libraries is very limited, so there is a good chance that there will be another hurdle to overcome.

How can I force ld to link against static library

Inside the Netica_API_607/lib directory, I have both libnetica.a and libnetica.so

This should work:

gcc -shared ... -L/home1/.../Netica_API_607/lib -Wl,-Bstatic -lnetica -Wl,-Bdynamic ...

Alternatively, you should be able to do this:

gcc -shared ... /home1/.../Netica_API_607/lib/libnetica.a ...

I thought that the -dn and -dy switches would

These switches are "global"; that is: -dn sets a boolean inside the linker to "never use shared libraries", and the -dy flips that boolean back to default, regardless of where on the command line these switches occur.

Is there any way to tell the linker to ignore DT_SONAME when loading a shared object?

In other words, can when running an executable can you force the linker to ignore the DT_SONAME of external shared libraries and instead use file names only?

No.

First, the linker is not involved in the running of the executable, the loader (also known as the dynamic linker) is. The loader is a completely separate program, and usually shares no code, and is totally different from the linker.

Second, after the linker is done, the filename that was used to link the executable is not recorded anywhere (only the DT_SONAME is recorded). So even if the loader wanted to use the original filename, there is no way for it to find out what it was.

That said, you could change the string in the .dynstr section of the executable that records the DT_SONAME. If the string is (say) foobar.so.1, and you binary edit it to be barfoo.so instead, then the loader will look for barfoo.so (using its usual shared library search rules) instead of foobar.so.1. The only limitation here is that the new name is not longer than the old one (or else you'll corrupt some other string).

It's also possible to replace the .dynstr section with a new one with longer strings (and replace foobar.so.1 with /full/path/to/libsomething.so), but this transformation is much harder to perform correctly.

how to force linker to use shared library instead of static library?

You could use -l option in its form -l:filename if your linker supports it (older versions of ld didn't)

gcc -o app app.o -L. -l:libtest.so

Other option is to use the filename directly without -l and -L

gcc -o app app.o /path/to/library/libtest.so

gcc, how to force the final executable link a unused shared library?

You have an XY-problem, where X is: libfoo has unresolved symbols, but the linker doesn't warn about it

So use the -z defs option linkage-time, and when you get the linker error about the unresolved symbol add -lfoo to the linkage command.

That's still not enough, you will have to use a -L and a -Wl,-rpath option too. Here is a complete Makefile:

# Makefile

# LIBDIR should be the final place of the shared libraries
# such as /usr/local/lib or ~/libexec/myproject

LIBDIR  := ${PWD}
TARGETS := shared_main libbar.so libfoo.so

all: ${TARGETS}

clean:
    rm -f ${TARGETS} 2>/dev/null || true

shared_main: shared_main.c
    gcc -g -Wall -o shared_main shared_main.c -ldl

libbar.so: bar.c
    gcc -g -Wall -fPIC -shared -o libbar.so bar.c

libfoo.so: foo.c libbar.so
    gcc -g -Wall -fPIC -shared -z defs -o libfoo.so foo.c \
    -L${LIBDIR} -Wl,-rpath,${LIBDIR} -lbar

Edit: nonetheless, here is a hackish solution for the original question: use option -Wl,--no-as-needed

shared_main:
    gcc -g -Wall -o shared_main shared_main.c \
    -Wl,--no-as-needed -Wl,-rpath,${PWD} libbar.so -ldl

Link only needed symbols when compiling an executable with a Shared Library

Details of dynamic linking and the kinds of objects involved vary across environments and toolchains. On Linux, where you say you are, and on Solaris, and several other UNIX-y platforms, you are looking at ELF objects and semantics.

So far I tried using -Wl,--as-needed,
-Wl,--unresolved-symbols=ignore-in-shared-libs,

These both have their full effect at (static) link time. The first tells the linker that the libraries following it on the command line should be linked in only if they resolve at least one as-yet undefined symbol. The latter tells the linker to not worry about resolving symbols in shared libraries included in the link. That has nothing to do with the behavior of the dynamic linker when you run the program.

and opening the shared object with dlopen to get the function I want with dlsym.

dlopen instructs the dynamic linker to link in a shared object at runtime that was not specified in the binary as a required shared library. Its behavior at that point can be modulated by the flags passed to dlopen, but the options available are not more than can be specified at link time. There is little reason to use dlopen when you actually know at link time what libraries you need.

Are you forced to resolve every undefined symbol of a dynamic library
when linking it against an executable ?

Focusing on ELF and the GNU toolchain, no. -Wl,--unresolved-symbols=ignore-in-shared-libs serves precisely the purpose of avoiding that. But as you've discovered, that comes with caveats.

In the first place, in every shared object, every symbol referring to data needs to be resolved at runtime by the dynamic linker, no matter how you linked the various shared objects, including the main program. This is primarily an operational consideration -- the dynamic linker has no way to defer resolving symbols referring to objects because it has no good way to trap attempts to access them.

On the other hand, it is possible to defer resolution of symbols referring to functions until their first use. In fact, this is the GNU linker's default, but you can reaffirm this by passing -Wl,-z,lazy to gcc when linking. Note well, however, that this sets a property of the object being linked, so you should ensure that every shared object is built with that link option (but ordinarily they are because, again, that's the default).

Additionally, you should be aware that the dynamic linker's behavior can be influenced by environment variables. In particular, lazy binding will be disabled if the dynamic linker finds LD_BIND_NOW set to a nonempty string in the runtime environment.

A simple workaround would be to add all the dependencies of my
libraries when compiling the executable. But they're so full of
dependencies that this sometimes means adding 10+ libraries to the
command line, and this would be for something like a hundred
executable.

And what's the big deal with that, really? Surely you have a well-factored Makefile (or several) to help you, so it shouldn't be a big deal to ensure that all the libraries are linked. Right?

But you should also consider refactoring your libraries, especially if "interdependent" means there are loops in the dependency graph. Dynamic linking is different from static linking, as you've discovered, and the differences are sometimes more subtle than those you're presently struggling with. Although it is not a hard rule, I urge you to avoid creating situations where the shared objects used by one process contain among them multiple definitions of the same external symbol, especially if that symbol is actually used.

---

Update

The above discussion focuses on linking shared libraries to an executable, but there is another important consideration: how the libraries themselves are linked. Each ELF object, whether executable or shared library, carries its own list of needed shared libraries. The dynamic linker will recursively include all of these in the list of shared libraries to be loaded (immediately) at program startup, notwithstanding its behavior with respect to lazy binding of symbols referring to functions.

Therefore, if you want an executable not to require a given shared library X, then not only that executable itself but also every shared library it does rely upon must avoid expressing a dependency on X. If some of the shared libs require X when used in conjunction with other programs, then that puts the onus on you to link in all the needed libraries when building those programs (otherwise, you can arrange to link only direct dependencies). You can tell the GNU linker to build shared libraries this way by passing it the --allow-shlib-undefined flag.

Here is a complete proof of concept:

main.c

int mul(int, int);

int main(void) {
    return mul(2, 3);
}

mul.c

int add(int, int);

int mul(int x, int y) {
    return x * y;
}

int mul2(int x, int y) {
    return add(x, y) * add(x, -y);
}

Makefile

CC = gcc
LD = gcc
CFLAGS = -g -O2 -fPIC -DPIC
LDFLAGS = -Wl,--unresolved-symbols=ignore-in-shared-libs
SHLIB_LDFLAGS = -shared -Wl,--allow-shlib-undefined

all: main

main: main.o libmul.so
    $(LD) $(CFLAGS) $(LDFLAGS) -o $@ $^

libmul.so: mul.o
    $(LD) $(CFLAGS) $(SHLIB_LDFLAGS) -o $@ $^

clean:
    rm -f main main.o libmul.so mul.o

Demo

$ make
gcc -g -O2 -fPIC -DPIC   -c -o main.o main.c
gcc -g -O2 -fPIC -DPIC   -c -o mul.o mul.c
gcc -g -O2 -fPIC -DPIC -shared -Wl,--allow-shlib-undefined -o libmul.so mul.o
gcc -g -O2 -fPIC -DPIC -Wl,--unresolved-symbols=ignore-in-shared-libs -o main main.o libmul.so
$ LD_LIBRARY_PATH=$(pwd) ./main
$ echo $?
6
$

Note that the -zlazy linker option discussed in comments is omitted, as it's the default.

Static link of shared library function in gcc

Refer to:

http://www.linuxquestions.org/questions/linux-newbie-8/forcing-static-linking-of-shared-libraries-696714/

You need the static version of the library to link it.

A shared library is actually an executable in a special format
with entry points specified (and some sticky addressing issues
included). It does not have all the information needed to
link statically.

You can't statically link a shared library (or dynamically link a static one).

The flag -static will force the linker to use static libraries (.a) instead of shared (.so) ones. But static libraries aren't always installed by default, so you may have to install the static library yourself.

Another possible approach is to use statifier or Ermine. Both tools take as input a dynamically linked executable and as output create a self-contained executable with all shared libraries embedded.

---

Related Topics