Swift package manager unable to compile ncurses installed through Homebrew
Problem
The main problem are conflicts of the header files, since ncurses is also supplied in /Applications/Xcode.app/.../MacOSX10.14.sdk/usr/include.
A common pure C solution in such situations is to just specify the custom include and lib directories with -I repective -L and it would work, see my answer regarding C ncurses here: https://stackoverflow.com/a/56623033/2331445
This approach does not seem to work with the Swift package manager. But that doesn't mean it's not possible with a little effort.
Possible Solution
We need to make sure that the ncurses header files provided by the macOS SDK are ignored. We can do this by specifying the -Xcc -D__NCURSES_H parameter for the swift build command.
This works because in the header file, there is this typical:
#ifndef __NCURSES_H
#define __NCURSES_H
...
#endif
The problem, of course, is that our custom installation of ncurses using Brew is also affected. But we can work around it:
- copy the new ncurses header files into our Sources/Cncurses directory
- replace __NCURSES_H through something different, e.g. __CNCURSES_H (note leading 'C')
- then make sure that all further nested includes are first searched in our local include directory by replace the angle brackets of the includes with quotes, so e.g. instead of
#include <ncursesw/unctrl.h>
the form '#include "ncursesw/unctrl.h"' is used
This can actually be done with the following command line commands:
cd Sources/Cncurses
cp -r /usr/local/Cellar/ncurses/6.1/include include
find . -name '*.h' -exec sed -i '' 's/__NCURSES_H/__CNCURSES_H/g' {} \;
find . -name '*.h' -exec sed -i '' -E -e "s/<(.*(`find . -name '*.h' -exec basename {} \; | paste -sd "|" -`))>/\"\1\"/g" {} \;
The last statement may require some explanation. With the help of an echo command, you can look at the generated sed expression, i.e. if you execute
echo "s/<(.*(`find . -name '*.h' -exec basename {} \; | paste -sd "|" -`))>/\"\1\"/g"
you get the following output:
s/<(.*(termcap.h|form.h|term.h|panel.h|ncurses.h|termcap.h|cursesp.h|cursesf.h|etip.h|form.h|cursesw.h|nc_tparm.h|unctrl.h|cursesapp.h|term.h|cursslk.h|panel.h|ncurses.h|tic.h|eti.h|ncurses_dll.h|term_entry.h|menu.h|cursesm.h|curses.h|curses.h|cncurses.h))>/"\1"/g
As you can see, it searches and replaces only local available include files.
Test
For a test we need a simple ncurses example program. It should be built and we should make sure that the correct version of the library is used.
module.modulemap
My header file is called cncurses.h. The module.modulemap looks like this:
module cncurses [system]
{
umbrella header "cncurses.h"
link "ncurses"
export *
}
cncurses.h
cncurses.h is a one-liner, it imports our copied and customized ncurses.h file from our local include folder:
#include "include/ncurses.h"
main.swift
In the NcursesExample folder we have main.swift where we have a simple cncurses swift app:
import cncurses
initscr()
curs_set(0)
move(5, 10)
addstr("NCURSES")
move(10, 10)
addstr("Hello World!")
refresh()
select(0, nil, nil, nil, nil)
Package.swift
Please note here the pkgConfig: "ncurses"
in the systemLibrary targets:
// swift-tools-version:5.0
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "NcursesExample",
dependencies: [
],
targets: [
.systemLibrary(name: "cncurses", pkgConfig: "ncurses"),
.target(name: "NcursesExample", dependencies: ["cncurses"]),
.testTarget(
name: "NcursesExampleTests",
dependencies: ["NcursesExample"]),
]
)
Build
For pkg-config to do its job properly, we must first call the following:
export PKG_CONFIG_PATH="/usr/local/opt/ncurses/lib/pkgconfig"
Finally we initiate the build with:
swift build -Xcc -D__NCURSES_H
So first we should test if the correct ncurses lib was used. We can do that with:
otool -L .build/x86_64-apple-macosx/debug/NcursesExample
Among other lines, the output contains this:
/usr/local/opt/ncurses/lib/libncursesw.6.dylib (compatibility version 6.0.0, current version 6.0.0)
which looks promising.
Finally calling the binary:
Xcode Project
If you want to generate a Xcode project, use the following command:
swift package generate-xcodeproj
Then load the project in Xcode and
- select the project node
- in Build settings enter Preprocessor in the search field in the upper right
- under Apple Clang - Preprocessing / Preprocess Macros add __NCURSES_H=1 for Debug and Release
Clang on macOS fails linking lmenu from ncurses
You can do the following:
brew install ncurses
Since macOS already contains a ncurses version brew installs its alternative version in /usr/local/opt/ncurses
.
So that the compiler and linker can access it, your build command should now look like this:
gcc -I/usr/local/opt/ncurses/include -L/usr/local/opt/ncurses/lib libmenutest.c -o test -lmenu -lncurses
When you finally call your program, the following is output:
This worked :^)
CMake
For the folks using CMake your CMakeLists.txt could look like this:
cmake_minimum_required(VERSION 3.14)
project(libmenutest C)
set(CMAKE_C_STANDARD 99)
include_directories(/usr/local/opt/ncurses/include)
link_directories(/usr/local/opt/ncurses/lib)
add_executable(libmenutest libmenutest.c)
target_link_libraries(libmenutest menu ncurses)
Linux: Compile using a specific ncurses version when both V6 and V5 are installed?
Debian has several packages for ncurses (I find the bug-reporting links easier to navigate than the package-tracking). At the moment, buster is "oldstable". That has development packages for ABI 5 and 6:
- libncurses-dev
- libncursesw-dev
- libncurses5-dev
- libncursesw5-dev
Each of those has a package-page which includes a list of files for each architecture, e.g., this, and for example
/usr/lib/x86_64-linux-gnu/pkgconfig/form.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/formw.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/menu.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/menuw.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/ncurses++.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/ncurses++w.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/ncurses.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/ncursesw.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/panel.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/panelw.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/tic.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/tinfo.pc
Generally, the filename for the ".pc" files is useful as a parameter to pkg-config
, e.g.,
pkg-config --cflags --libs libncurses-dev
which prints the -I
, -D
, -L
and -l options you'd use in a makefile or script.
You don't have to search through the webpages for that;
dpkg -l | grep 'ncurses'
lists the package with "ncurses" in their name, and for the example shown
dpkg -L libncurses-dev | grep -F .pc
shows the pkg-config files.
In a command-line, I could do this:
gcc -c $(pkg-config --cflags libncurses-dev) foo.c
for just the compiler, or to compile/link:
gcc -o foo $(pkg-config --cflags --libs libncurses-dev) foo.c
The package also has the "*-config" scripts which give similar information, without using pkg-config
, e.g.,
/usr/bin/ncurses5-config
/usr/bin/ncurses6-config
/usr/bin/ncursesw5-config
/usr/bin/ncursesw6-config
In the Debian development packages, it appears that there are ".pc" files only for ABI 6. But the "-config" files are provided for both ABI 5 and 6. So you should be able to compile with ABI 5, e.g,.
gcc -c $(ncurses5-config --cflags libncurses-dev) foo.c
Use ansible package module to work with apt and homebrew
If you want a single set of tasks that flexible enough for for multiple Linux
package managers and macOS brew, the choice is either more logic or more
duplication.
These three patterns should help. They still have repetition and
boilerplate code, but that's the territory we're in with Ansible plays
for cross-platform.
- Declare
become: yes
(root) globally for Linux only - Address packages that need platform-specific treatment as-needed with
when
- This might be
--head
forbrew
, or setting up a PPA forapt
, etc
- This might be
- Map package name discrepancies with variables
- For example:
brew install ncurses
,apt install libncurses5-dev
, anddnf install ncurses-devel
are all the same library.
- For example:
1) Declare become: yes
(root) globally for Linux only
For Linux hosts, switching to root for installation is the intended behavior.
For macOS a la Homebrew, installing as root is not good. So, we need become: no
(false
) when using brew
, and become: yes
(true
) otherwise (for
Linux).
In your example the become
directive is nested inside each task ("step"). To
prevent duplication, invoke become
at a higher lexical scope, before the tasks
start. The subsequent tasks will then inherit the state of become
, which is
set based on a conditional expression.
Unfortunately a variable for become
at the root playbook scope will be
undefined and throw an error before the first task is run:
# playbook.yml
- name: Demo
hosts: localhost
connection: local
# This works
become: True
# This doesn't - the variable is undefined
become: "{{ False if ansible_pkg_mgr == 'brew' else True }}"
# Nor does this - also undefined
become: "{{ False if ansible_os_family == 'Darwin' else True }}"
tasks:
# ...
To fix this, we can store the tasks in another file and import them, or
wrap the tasks in a block. Either of these patterns will provide
a chance to declare become
with our custom variable value in time for the
tasks to pick it up:
# playbook.yml
---
- name: Demo
hosts: localhost
connection: local
vars:
# This variable gives us a boolean for deciding whether or not to become
# root. It cascades down to any subsequent tasks unless overwritten.
should_be_root: "{{ true if ansible_pkg_mgr != 'brew' else false }}"
# It could also be based on the OS type, but since brew is the main cause
# it's probably better this way.
# should_be_root: "{{ False if ansible_os_family == 'Darwin' else True }}"
tasks:
# Import the tasks from another file, which gives us a chance to pass along
# a `become` context with our variable:
- import_tasks: test_tasks.yml
become: "{{ should_be_root }}"
# Wrapping the tasks in a block will also work:
- block:
- name: ncurses is present
package:
name: [libncurses5-dev, libncursesw5-dev]
state: present
- name: cmatrix is present
package:
name: cmatrix
state: present
become: "{{ should_be_root }}"
Now there is a single logic check for brew
and a single before
directive
(depending on which task pattern above is used). All tasks will be executed as
the root user, unless the package manager in use is brew
.
2) Address packages that need platform-specific treatment as-needed with when
The Package Module is a great convenience but it's quite limited. By
itself it only works for ideal scenarios; meaning, a package that doesn't
require any special treatment or flags from the underlying package manager. All
it can do is pass the literal string of the package to install, the state
, and
an optional parameter to force use of a specific package manager executable.
Here's an example that installs wget
with a nice short task and only becomes
verbose to handle ffmpeg
's special case when installed with brew
:
# playbook.yml
# ...
tasks:
# wget is the same among package managers, nothing to see here
- name: wget is present
when: ansible_pkg_mgr != 'brew'
package:
name: wget
state: present
# This will only run on hosts that do not use `brew`, like linux
- name: ffmpeg is present
when: ansible_pkg_mgr != 'brew'
package:
name: ffmpeg
state: present
# This will only run on hosts that use `brew`, i.e. macOS
- name: ffmpeg is present (brew)
when: ansible_pkg_mgr == 'brew'
homebrew:
name: ffmpeg
# head flag
state: head
# --with-chromaprint --with-fdk-aac --with-etc-etc
install_options: with-chromaprint, with-fdk-aac, with-etc-etc
The play above would produce this output for ffmpeg
against a Linux box:
TASK [youtube-dl : ffmpeg is present] ******************************************
ok: [localhost]
TASK [youtube-dl : ffmpeg is present (brew)] ***********************************
skipping: [localhost]
3) Map package name discrepancies with variables
This isn't specifically part of your question but it's likely to come up next.
The Package Module docs also mention:
Package names also vary with package manager; this module will not "translate"
them per distro. For example libyaml-dev, libyaml-devel.
So, we're on our own to handle cases where the same software uses different
names between package manager platforms. This is quite common.
There are multiple patterns for this, such as:
- Use separate variable files for each OS/distro and import them
conditionally - Use a role with its own variables
- Use the same package manager across platforms, such as Homebrew or
Conda - Compile everything from source via git
None of them are very pleasant. Here is an approach using a role. Roles do
involve more boilerplate and directory juggling, but in exchange they provide
modularity and a local variable environment. When a set of tasks in a role
requires more finagling to get right, it doesn't end up polluting other task
sets.
# playbook.yml
---
- name: Demo
hosts: localhost
connection: local
roles:
- cmatrix
# roles/cmatrix/defaults/main.yml
---
ncurses:
default:
- ncurses
# Important: these keys need to exactly match the name of package managers for
# our logic to hold up
apt:
- libncurses5-dev
- libncursesw5-dev
brew:
- pkg-config
- ncurses
# roles/cmatrix/tasks/main.yml
---
- name: cmatix and its dependencies are present
become: "{{ should_be_root }}"
block:
- name: ncurses is present
package:
name: '{{ item }}'
state: latest
loop: "{{ ncurses[ansible_pkg_mgr] | default(ncurses['default']) }}"
- name: cmatrix is present
when: ansible_pkg_mgr != 'brew'
package:
name: cmatrix
state: present
The task for ncurses
looks for an array of items to loop through keyed by the
corresponding package manager. If the package manager being used is not defined
in the variable object, a Jinja default filter is employed to reference thedefault
value we set.
With this pattern, adding support for another package manager or additional
dependencies simply involves updating the variable object:
# roles/cmatrix/defaults/main.yml
---
ncurses:
default:
- ncurses
apt:
- libncurses5-dev
- libncursesw5-dev
# add a new dependency for Debian
- imaginarycurses-dep
brew:
- pkg-config
- ncurses
# add support for Fedora
dnf:
- ncurses-devel
Combining everything into a real play
Here's a full example covering all three aspects. The playbook has two roles
that each use the correct become
value based on a single variable. It also
incorporates an special cases for cmatrix
and ffmpeg
when installed withbrew
, and handles alternate names for ncurses between package managers.
# playbook.yml
---
- name: Demo
hosts: localhost
connection: local
vars:
should_be_root: "{{ true if ansible_pkg_mgr != 'brew' else false }}"
roles:
- cmatrix
- youtube-dl
# roles/cmatrix/defaults/main.yml
ncurses:
default:
- ncurses
apt:
- libncurses5-dev
- libncursesw5-dev
brew:
- pkg-config
- ncurses
dnf:
- ncurses-devel
# roles/cmatrix/tasks/main.yml
---
- name: cmatrix and dependencies are present
# A var from above, in the playbook
become: "{{ should_be_root }}"
block:
- name: ncurses is present
package:
name: '{{ item }}'
state: latest
# Get an array of the correct package names to install from the map in our
# default variables file
loop: "{{ ncurses[ansible_pkg_mgr] | default(ncurses['default']) }}"
# Install as usual if this is not a brew system
- name: cmatrix is present
when: ansible_pkg_mgr != 'brew'
package:
name: cmatrix
state: present
# If it is a brew system, use this instead
- name: cmatrix is present (brew)
when: ansible_pkg_mgr == 'brew'
homebrew:
name: cmatrix
state: head
install_options: with-some-option
# roles/youtube-dl/tasks/main.yml
---
- name: youtube-dl and dependencies are present
become: "{{ should_be_root }}"
block:
- name: ffmpeg is present
when: ansible_pkg_mgr != 'brew'
package:
name: ffmpeg
state: latest
- name: ffmpeg is present (brew)
when: ansible_pkg_mgr == 'brew'
homebrew:
name: ffmpeg
state: head
install_options: with-chromaprint, with-fdk-aac, with-etc-etc
- name: atomicparsley is present
package:
name: atomicparsley
state: latest
- name: youtube-dl is present
package:
name: youtube-dl
state: latest
The result for Ubuntu:
$ ansible-playbook demo.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'
PLAY [Demo] ********************************************************************
TASK [Gathering Facts] *********************************************************
ok: [localhost]
TASK [cmatrix : ncurses is present] ********************************************
ok: [localhost] => (item=libncurses5-dev)
ok: [localhost] => (item=libncursesw5-dev)
TASK [cmatrix : cmatrix is present] ********************************************
ok: [localhost]
TASK [cmatrix : cmatrix is present (brew)] *************************************
skipping: [localhost]
TASK [youtube-dl : ffmpeg is present] ******************************************
ok: [localhost]
TASK [youtube-dl : ffmpeg is present (brew)] ***********************************
skipping: [localhost]
TASK [youtube-dl : atomicparsley is present] ***********************************
ok: [localhost]
TASK [youtube-dl : youtube-dl is present] **************************************
ok: [localhost]
PLAY RECAP *********************************************************************
localhost : ok=6 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0
Related Topics
Swift: Oslog/Os_Log Not Showing Up in Console App
Connecting Hc-05 with iPhone Se iOS(V11.0)
How to Add Icon to a Share Sheet in Swift
How to Handle Parameter Validation Swift
How, Exactly, Do I Render Metal on a Background Thread
Response Struct Does Not Like Codingkeys
How to Initialize a Mlmultiarray in Coreml
Enum Not Working in Custom Initializer
Swiftui Pick a Value from a List with Ontap Gesture
Swift Nsusernotification Doesn't Show While App Is Active
Why Can't I Use Subscripting on a Ckrecord Object in Swift
How to Capitalize First Word in Every Sentence with Swift
Button State Activates on Wrong Cells