I know nix-ld is popular, I'd like to give you an alternative to run binaries that will help you see how nix works from within a little bit...
This is an automated archive made by the Lemmit Bot.

The original was posted on /r/nixos by /u/Combinatorilliance on 2024-09-25 09:00:52+00:00.

I know nix-ld is popular, I’d like to give you an alternative to run binaries that will help you see how nix works from within a little bit better. I’ve been reading Eelco’s PhD thesis, and I’m learning a lot about how Nix works under the hood. So here’s me sharing a bit of what I learned :)

Binaries load libraries at runtime

As you may or may not know, many binaries depend on libraries that are found at runtime. They do this so that you can change the library without having to recompile the binary, for example for security reasons, or because you don’t want to have 10000 instances of the Qt or GTK libraries compiled into every single binary consuming them.

You can inspect runtime libraries

bash $ nix-shell -p bintools $ readelf -p your_binary

For example, here’s a Go binary that I wanted to use:

$ readelf -p ./rmapi

There is no dynamic section in this file.

No dynamic library dependencies! I think that’s because Go tries to make completely self-contained binaries. Neat! It works perfectly without nix-ld or any other modifications

Or here, a binary that’s meant to help me update my axidraw (wonderful devices!).

Note that the output may look a little bit scary, but it’s not that bad if you consider that you only need to look at the lines with (RUNPATH) and (NEEDED) and can ignore the hexadecimal in the tags entirely.

$ readelf -d ./mphidflash-1.6-linux-64

Dynamic section at offset 0x2a8 contains 26 entries:
 Tag Type Name/Value
 0x000000000000001d (RUNPATH) Library runpath: [/nix/store/0d25a94w1d09prkfm3w3pfwx1ah5ym0k-libusb-compat-0.1.8/lib:/nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib]
 0x0000000000000001 (NEEDED) Shared library: [libusb-0.1.so.4]
 0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
 0x000000000000000c (INIT) 0x400cb8
 0x000000000000000d (FINI) 0x402324
 0x0000000000000019 (INIT\_ARRAY) 0x602e00
 0x000000000000001b (INIT\_ARRAYSZ) 8 (bytes)
 0x000000000000001a (FINI\_ARRAY) 0x602e08
 0x000000000000001c (FINI\_ARRAYSZ) 8 (bytes)
 0x000000006ffffef5 (GNU\_HASH) 0x3ffb30
 0x0000000000000005 (STRTAB) 0x3ff498
 0x0000000000000006 (SYMTAB) 0x3ff758
 0x000000000000000a (STRSZ) 699 (bytes)
 0x000000000000000b (SYMENT) 24 (bytes)
 0x0000000000000015 (DEBUG) 0x0
 0x0000000000000003 (PLTGOT) 0x603000
 0x0000000000000002 (PLTRELSZ) 720 (bytes)
 0x0000000000000014 (PLTREL) RELA
 0x0000000000000017 (JMPREL) 0x4009e8
 0x0000000000000007 (RELA) 0x4009a0
 0x0000000000000008 (RELASZ) 72 (bytes)
 0x0000000000000009 (RELAENT) 24 (bytes)
 0x000000006ffffffe (VERNEED) 0x400940
 0x000000006fffffff (VERNEEDNUM) 1
 0x000000006ffffff0 (VERSYM) 0x4008ea
 0x0000000000000000 (NULL) 0x0

What’s relevant is the following (I’ll post the process_readelf.awk in the comments):

bash readelf -d ./mphidflash-1.6-linux-64 | ./process_readelf.awk Type Name/Value (RUNPATH) Library runpath: [/nix/store/0d25a94w1d09prkfm3w3pfwx1ah5ym0k-libusb-compat-0.1.8/lib:/nix/store/3dyw8dzj9ab4m8hv5dpyx7zii8d0w6fi-glibc-2.39-52/lib] (NEEDED) Shared library: [libusb-0.1.so.4] (NEEDED) Shared library: [libc.so.6]

See that? It NEEDS libusb-0.1.so.4 and libc.so.6

If you don’t have these, the binary won’t run!

So um… Where does Linux look for these shared libaries?

This means it needs to be stored somewhere on your computer, because it is resolved at runtime, you can change the runtime dependency by modifying where the binary searches for the dependency.

When you execute a binary, the following locations will be searched for in order:

  1. RPATH of the executable (if present)
  2. Environment variable LD_LIBRARY_PATH (if set)
  3. RUNPATH of the executable (if present)
  4. Cache file /etc/ld.so.cache (if it exists)
  5. Default system directories (/lib, /usr/lib)

(Guess what nix-ld does. Hint: it’s number 2. That wasn’t actually a hint.)

In the usb mphidflash-1.6-linux-64 binary above, you can see the RUNPATH in the executable too. It doesn’t have an RPATH.

Nix often changes the RPATH of a binary if you compile a binary through Nix itself. This means that in practice you cannot change the path anymore. This would be a downside in regular Linux, but in Nix? Not so much, Nix is about guarantees.

Ok so for god’s sake please tell me about the alternative…

A nix shell! The alternative is just a shell!

To run the mphidflash-1.6-linux-64 binary above, we need to make the shared libraries available. So, what we do is we create a shell as follows

  1. We specify the libraries as buildInputs for our mkShell
  2. We add patchelf so we can patch the RPATH of the executable with these libraries
  3. We patch the elf file with patchelf
  4. And we make it available within our shell
$ cat mphidshell.nix  

{ pkgs ? import  {} }:

let
 # Create a wrapper script for the binary
 mphidflashWrapper = pkgs.writeScriptBin "mphidflash-wrapper" ''
 #!${pkgs.stdenv.shell}

Use patchelf to set the RPATH of the binary

${pkgs.patchelf}/bin/patchelf --set-rpath “${pkgs.lib.makeLibraryPath [ pkgs.libusb-compat-0_1 pkgs.glibc ]}” ${builtins.toString ./mphidflash-1.6-linux-64}

Run the binary with sudo

exec sudo ${builtins.toString ./mphidflash-1.6-linux-64} “$@”


'';
in
pkgs.mkShell {
 buildInputs = with pkgs; [
 libusb-compat-0\_1 # This provides libusb-0.1.so.4
 glibc # This provides libc.so.6
 patchelf # For modifying the binary's RPATH
 mphidflashWrapper
 ];

shellHook = ''
 echo "Use 'mphidflash-wrapper' to run the binary with sudo and proper library paths."
 '';
}

Now I hear you thinking… “What the fuck”. “What the actual fuck is all that”.

I agree! There’s a lot to take in here. I’m still learning Nix, and there’s no way I can just write a script like this every time I want to run a binary.

So I don’t! I didn’t write this script, Claude did :)

I just copy the output from readelf -d into claude and ask it to make a shell. It works quite consisently. With LLMs we can automate the entire process of creating a shell for a binary with runtime path requirements.

Does this always work?

No. This only works for compiled binaries, so for example binaries made with systems languages like C, C++, Go and the like. Many applications and programs in the Linux ecosystem are written with higher level languages like Python, JavaScript using Nodejs, Ruby and Perl. Python especially is a whole different mess to get working.

For example, I tried making inkscape extensions work (again, for the axidraw) and I wasn’t able to do so. Instead, I just opted to create a distrobox with Ubuntu, install inkscape in there and run it from within the distrobox. Works flawlessly.

Shortcuts are ok while learning.