How to export symbols from a shared library
This is how it works on linux:
1) No, you needn't do anything. You can, however, restrict exporting variables with gcc -fvisibility
command line argument and explicitly flag exported entries with the visibility attribute.
2) The executable will have a table of all functions it imports (these are all functions with default visibility). The loader/linker will pick an address to load the libraries to and fill this table just before running, the calls to those functions are indirect calls. (Note that this holds for shared objects as well)
3) Static linking is performed on link-time (which is after you compile). The actual addresses are substituted in the assembly, and they are direct calls.
Note: There is the thing called PIC (position independent code). AFAIK, this deals with references to data/functions in the same shared object, so the linker needn't overwrite half of the code of the library when loading the library, in the way that the code doesn't make any absolute references to its own data. You might try to experiment with it.
Keep all exported symbols when creating a shared library from a static library
What you observe results when some of the global symbol definitions in some of
the object files archived in libxxx.a
were compiled with the function attribute
or variable attribute visibility("hidden")
This attribute has the effect that when the object file containing the
the global symbol definition is linked into a shared library:
- The linkage of the symbol is changed from global to local in the static symbol table (
.symtab
) of the output shared library,
so that when that shared library is linked with anything else, the linker cannot see the definition of the symbol. - The symbol definition is not added to the dynamic symbol table (
.dynsym
) of the output shared library (which by default it would be)
so that when the shared library is loaded into a process, the loader is likewise unable to find a definition of the symbol.
In short, the global symbol definition in the object file is hidden for the purposes of dynamic linkage.
Check this out with:
$ readelf -s libxxx.a | grep HIDDEN
and I expect you to get hits for the unexported global symbols. If you don't,
you need read no further because I have no other explanation of what you see
and wouldn't count on any workaround I suggested not to shoot you in the foot.
Here is an illustration:
a.c
#include <stdio.h>
void aa(void)
{
puts(__func__);
}
b.c
#include <stdio.h>
void __attribute__((visibility("hidden"))) bb(void)
{
puts(__func__);
}
de.c
#include <stdio.h>
void __attribute__((visibility("default"))) dd(void)
{
puts(__func__);
}
void ee(void)
{
puts(__func__);
}
We'll compile a.c
and b.c
like so:
$ gcc -Wall -c a.c b.c
And we can see that symbols aa
and ab
are defined and global in their respective object files:
$ nm --defined-only a.o b.o
a.o:
0000000000000000 T aa
0000000000000000 r __func__.2361
b.o:
0000000000000000 T bb
0000000000000000 r __func__.2361
But we can also observe this difference:
$ readelf -s a.o
Symbol table '.symtab' contains 13 entries:
Num: Value Size Type Bind Vis Ndx Name
...
10: 0000000000000000 19 FUNC GLOBAL DEFAULT 1 aa
...
as compared with:
$ readelf -s b.o
Symbol table '.symtab' contains 13 entries:
Num: Value Size Type Bind Vis Ndx Name
...
10: 0000000000000000 19 FUNC GLOBAL HIDDEN 1 bb
...
aa
is a GLOBAL
symbol with DEFAULT
visibility and bb
is a GLOBAL
symbol with HIDDEN
visibility.
We'll compile de.c
differently:
$ gcc -Wall -fvisibility=hidden -c de.c
Here, we're instructing the compiler that any symbol shall be given hidden
visibility unless a countervailing visibility
attribute is specified for
it in the source code. And accordingly we see:
$ readelf -s de.o
Symbol table '.symtab' contains 15 entries:
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
...
11: 0000000000000000 19 FUNC GLOBAL DEFAULT 1 dd
...
14: 0000000000000013 19 FUNC GLOBAL HIDDEN 1 ee
Archiving these object files in a static library changes them in no way:
$ ar rcs libabde.a a.o b.o de.o
And then if we link all of them into a shared library:
$ gcc -o libabde.so -shared -Wl,--whole-archive libabde.a -Wl,--no-whole-archive
we find that:
$ readelf -s libabde.so | egrep '(aa|bb|dd|ee|Symbol table)'
Symbol table '.dynsym' contains 8 entries:
6: 0000000000001105 19 FUNC GLOBAL DEFAULT 12 aa
7: 000000000000112b 19 FUNC GLOBAL DEFAULT 12 dd
Symbol table '.symtab' contains 59 entries:
45: 0000000000001118 19 FUNC LOCAL DEFAULT 12 bb
51: 000000000000113e 19 FUNC LOCAL DEFAULT 12 ee
54: 0000000000001105 19 FUNC GLOBAL DEFAULT 12 aa
56: 000000000000112b 19 FUNC GLOBAL DEFAULT 12 dd
bb
and ee
, which were GLOBAL
with HIDDEN
visibility in the object files,
are LOCAL
in the static symbol of libabde.so
and are absent altogether
from its dynamic symbol table.
In this light, you may wish to re-evaluate your mission:
The symbols that have been given hidden visibility in the object files in libxxx.a
have
been hidden because the person who compiled them had a reason for
wishing to conceal them from dynamic linkage. Do you have a countervailing need
to export them for dynamic linkage? Or do you maybe just want to export them because
you've noticed that they're not exported and don't know why not?
If you nonetheless want to unhide the hidden symbols, and cannot change the source code
of the object files archived in libxxx.a
, your least worst resort is to:
- Extract each object file from
libxxx.a
- Doctor it to replace
HIDDEN
withDEFAULT
visibility on its global definitions - Put it into a new archive
libyyy.a
- Then use
libyyy.a
instead oflibxxx.a
.
The binutils
tool for doctoring object files is objcopy
.
But objcopy
has no operations to directly manipulate the dynamic visibility of
a symbol and you'd have to settle for a circuitous kludge that "achieves the effect
of" unhiding the hidden symbols:
- With
objcopy --redefine-sym
, rename each hidden global symbolS
as, say,__hidden__S
. - With
objcopy --add-symbol
, add a new global symbolS
that has the same value as__hidden_S
but getsDEFAULT
visibility by default.
ending up with two symbols with the same definition: the original hidden one
and a new unhidden alias for it.
Preferable to that would a means of simply and solely changing the visibility of a symbol in
an ELF object file, and a means is to hand in the LIEF library (Library to Instrument Executable Formats) -
Swiss Army Chainsaw for object and executable file alterations1.
Here is a Python script that calls on pylief
, the LIEF Python module, to unhide the
hidden globals in an ELF object file:
unhide.py
#!/usr/bin/python
# unhide.py - Replace hidden with default visibility on global symbols defined
# in an ELF object file
import argparse, sys, lief
from lief.ELF import SYMBOL_BINDINGS, SYMBOL_VISIBILITY, SYMBOL_TYPES
def warn(msg):
sys.stderr.write("WARNING: " + msg + "\n")
def unhide(objfile_in, objfile_out = None, namedsyms=None):
if not objfile_out:
objfile_out = objfile_in
binary = lief.parse(objfile_in)
allsyms = { sym.name for sym in binary.symbols }
selectedsyms = set([])
nasyms = { sym.name for sym in binary.symbols if \
sym.type == SYMBOL_TYPES.NOTYPE or \
sym.binding != SYMBOL_BINDINGS.GLOBAL or \
sym.visibility != SYMBOL_VISIBILITY.HIDDEN }
if namedsyms:
namedsyms = set(namedsyms)
nosyms = namedsyms - allsyms
for nosym in nosyms:
warn("No symbol " + nosym + " in " + objfile_in + ": ignored")
for sym in namedsyms & nasyms:
warn("Input symbol " + sym + \
" is not a hidden global symbol defined in " + objfile_in + \
": ignored")
selectedsyms = namedsyms - nosyms
else:
selectedsyms = allsyms
selectedsyms -= nasyms
unhidden = 0;
for sym in binary.symbols:
if sym.name in selectedsyms:
sym.visibility = SYMBOL_VISIBILITY.DEFAULT
unhidden += 1
print("Unhidden: " + sym.name)
print("{} symbols were unhidden".format(unhidden))
binary.write(objfile_out)
def get_args():
parser = argparse.ArgumentParser(
description="Replace hidden with default visibility on " + \
"global symbols defined in an ELF object file.")
parser.add_argument("ELFIN",help="ELF object file to read")
parser.add_argument("-s","--symbol",metavar="SYMBOL",action="append",
help="Unhide SYMBOL. " + \
"If unspecified, unhide all hidden global symbols defined in ELFIN")
parser.add_argument("--symfile",
help="File of whitespace-delimited symbols to unhide")
parser.add_argument("-o","--out",metavar="ELFOUT",
help="ELF object file to write. If unspecified, rewrite ELFIN")
return parser.parse_args()
def main():
args = get_args()
objfile_in = args.ELFIN
objfile_out = args.out
symlist = args.symbol
if not symlist:
symlist = []
symfile = args.symfile
if symfile:
with open(symfile,"r") as fh:
symlist += [word for line in fh for word in line.split()]
unhide(objfile_in,objfile_out,symlist)
main()
Usage:
$ ./unhide.py -h
usage: unhide.py [-h] [-s SYMBOL] [--symfile SYMFILE] [-o ELFOUT] ELFIN
Replace hidden with default visibility on global symbols defined in an ELF
object file.
positional arguments:
ELFIN ELF object file to read
optional arguments:
-h, --help show this help message and exit
-s SYMBOL, --symbol SYMBOL
Unhide SYMBOL. If unspecified, unhide all hidden
global symbols defined in ELFIN
--symfile SYMFILE File of whitespace-delimited symbols to unhide
-o ELFOUT, --out ELFOUT
ELF object file to write. If unspecified, rewrite
ELFIN
And here is a shell script:
unhide.sh
#!/bin/bash
OLD_ARCHIVE=$1
NEW_ARCHIVE=$2
OBJS=$(ar t $OLD_ARCHIVE)
for obj in $OBJS; do
rm -f $obj
ar xv $OLD_ARCHIVE $obj
./unhide.py $obj
done
rm -f $NEW_ARCHIVE
ar rcs $NEW_ARCHIVE $OBJS
echo "$NEW_ARCHIVE made"
that takes:
$1
= Name of an existing static library$2
= Name for a new static library
and creates $2
containing the object files from $1
, each modified
with unhide.py
to unhide all of its hidden global definitions.
Back with our illustration, we can run:
$ ./unhide.sh libabde.a libnew.a
x - a.o
0 symbols were unhidden
x - b.o
Unhidden: bb
1 symbols were unhidden
x - de.o
Unhidden: ee
1 symbols were unhidden
libnew.a made
and confirm that worked with:
$ readelf -s libnew.a | grep HIDDEN; echo Done
Done
$ readelf -s libnew.a | egrep '(aa|bb|dd|ee)'
10: 0000000000000000 19 FUNC GLOBAL DEFAULT 1 aa
10: 0000000000000000 19 FUNC GLOBAL DEFAULT 1 bb
11: 0000000000000000 19 FUNC GLOBAL DEFAULT 1 dd
14: 0000000000000013 19 FUNC GLOBAL DEFAULT 1 ee
Finally if we relink the shared library with the new archive
$ gcc -o libabde.so -shared -Wl,--whole-archive libnew.a -Wl,--no-whole-archive
all of the global symbols from the archive are exported:
$ readelf --dyn-syms libabde.so | egrep '(aa|bb|dd|ee)'
6: 0000000000001105 19 FUNC GLOBAL DEFAULT 12 aa
7: 000000000000112b 19 FUNC GLOBAL DEFAULT 12 dd
8: 0000000000001118 19 FUNC GLOBAL DEFAULT 12 bb
9: 000000000000113e 19 FUNC GLOBAL DEFAULT 12 ee
[1]
Download C/C++/Python libraries
Debian/Ubuntu provides C/C++ dev package lief-dev
.
Symbols from a static library are not exported while linking to a shared library
You need to pass the --whole-archive
option to the linker.
In CMake, you can do it as follows.
CMakeLists.txt
cmake_minimum_required(VERSION 3.0.2)
add_library(unit1 STATIC unit1.c)
target_compile_options(unit1 PRIVATE -fPIC)
add_library(unit2 STATIC unit2.c)
target_compile_options(unit2 PRIVATE -fPIC)
add_library(merged SHARED)
set_target_properties(merged PROPERTIES LINKER_LANGUAGE C)
target_link_libraries(merged
"-Wl,--whole-archive libunit1.a libunit2.a -Wl,--no-whole-archive"
unit1 unit2
)
Note: The target_link_libraries
command can be used to specify linker flags as well, not only library names. The quotes are important, otherwise CMake might rearrange the flags and remove duplicates.
Exported symbols
$ nm libmerged.so | grep " T "
000000000000065d T GoodbyeWorld
000000000000064a T HelloWorld
0000000000000670 T _fini
0000000000000520 T _init
Another option, to avoid the problem, would be to create OBJECT
instead of STATIC
libraries for unit1
and unit2
.
CMakeLists.txt
cmake_minimum_required(VERSION 3.0.2)
add_library(unit1 OBJECT unit1.c)
target_compile_options(unit1 PRIVATE -fPIC)
add_library(unit2 OBJECT unit2.c)
target_compile_options(unit2 PRIVATE -fPIC)
add_library(merged SHARED $<TARGET_OBJECTS:unit1> $<TARGET_OBJECTS:unit2>)
set_target_properties(merged PROPERTIES LINKER_LANGUAGE C)
Some C symbols in a C shared library are not exported even with explicit visibility
Forgot to add allocator.c
to the build system...
link a static library to a shared library and hide exported symbols
You want to use a linker version script, which exports the symbol(s) you want (bar
here) and hides everything else.
Example here.
choose exported symbols when linking an elf application and shared library
Found out after testing a little:
for ELF applications, you can use
-rdynamic
or-Wl,--export-dynamic
to export all symbols, or you can use-Wl,--dynamic-list <sym-file>
to export only some symbols when linking your application throughgcc
.for ELF libraries, you can't use
-rdynamic
,-Wl,--export-dynamic
or-Wl,--dynamic-list <symfile>
, you must use-Wl,--version-script=<verfile>
when linking your library throughgcc
.
The version-script and the sym-file are almost the same, except that for sym-file you do not code a version and a scope. Documentation: gnu ld
How can I control what symbols are exported from a shared library in Bazel?
First of all, shared libraries are generally declared with cc_binary
in conjunction with its linkshared
attribute rather than cc_library
. This is counterintuitive but reflects the intention that cc_binary
creates a transitive link while cc_library
declares an intermediate library. (More adventurous folks may try out cc_shared_library
Second of all, it's possible to use version scripts (and other linker scripts) by using them in linkopts
and declaring them as a dependency in deps
.
So, all together:
cc_binary(
name = 'mylib.so',
linkshared = True,
srcs = ['mylib.cc'],
linkopts = ['-Wl,-version-script=$(location version-script.lds)'],
deps = [
'version-script.lds',
]
)
How to find which shared library exported which imported symbol in my binary?
So, for example how can I find the shared library which exports the function named foo or printf or anything in an efficient way?
You can run your program with env LD_DEBUG=bindings ./a.out
. This will produce a lot of output, which you can grep
for foo
and printf
.
Note that the answer to "which external symbol in my binary is dependent on which shared library" is "whichever library defines this symbol first".
So if today your binary depends on lifoo.so
for foo
and on libc.so.6
for printf
, nothing stops you from running with a different libfoo.so
tomorrow, and that different version of libfoo.so
may define different symbols. If the new version of libfoo.so
defines printf
, that would cause the answer to your question for symbol printf
to change from libc.so.6
to libfoo.so
.
Related Topics
Installing Gnuplot 5.0 on Ubuntu
Linux Synchronization with Fifo Waiting Queue
Getting Current Path in Variable and Using It
How to Toggle Cr/Lf in Gnu Screen
Can 'Connect' Call on Socket Return Successfully Without Server Calling 'Accept'
How to Remove Only the First Occurrence of a Line in a File Using Sed
Using Output of Awk to Run Command
Who Can Access a File with Octal Permissions "000" on Linux/Unix
Using Named Pipes with Bash - Problem with Data Loss
Microsecond Accurate (Or Better) Process Timing in Linux
Linux Command Output as a Parameter of Another Command
Hadoop: Require Root's Password After Enter "Start-All.Sh"
Postgresql on Elastic Beanstalk (Amazon Linux 2)
Force Gnu Linker to Generate 32 Bit Elf Executables
What Scheduling Algorithms Does Linux Kernel Use
How to Search an Image for Subimages Using Linux Console
Unix File System: How Are File Names Translated to Disk Sectors