genrulify

How to use it?

To record a build, you need Nix with flakes enabled, an x86-64 Linux machine with access to KVM, and Rewind on your PATH. To build the generated rules, use Bazel or Buck2 on a machine with bubblewrap and the Nix store paths from the recording.

Rebuild a Nix package

convert records the Nix build of GNU hello and writes Bazel and Buck2 rules in hello/workspace. To check a step such as compiling hello.c, it runs the compiler again in a sandbox with the inputs GCC used. It checks that GCC exits with the same status and produces the same object file. If a step cannot reproduce a file the build needs, the generated workspace can supply that file from the recording.

nix run github:fzakaria/genrulify -- convert nixpkgs#hello hello
cd hello/workspace

A successful //:archive build with Bazel or Buck2 reproduces the recorded Nix archive (NAR) byte for byte. The sha256sum command below prints the same digest that convert printed for the Nix output.

Bazel

bazel build //:archive
sha256sum bazel-bin/archive.nar

Buck2

buck2 build //:archive --out hello.nar
sha256sum hello.nar

Explore the recorded hello build.

Build from sources you can edit

The repository includes a small C project. Its Makefile compiles main.c and greet.c, then links the two objects. The .#project derivation uses examples/project as its source directory. Run these commands from the root of a genrulify checkout.

nix run . -- convert --source examples/project .#project /tmp/project-recording
cd examples/project
bazel build //:archive

--source tells genrulify where the working sources live. It checks that they match the derivation's source, then writes the rules, their scripts and their manifests under genrulify/. The BUILD.bazel and BUCK it writes beside your sources load those rules. If you already have a build file of your own, genrulify leaves it alone and prints the two lines that load the rules from it. Each compile reads its source from your checkout. The last argument names a new directory for the recording and graph.

Change the greeting in greet.c and run bazel build //:archive again, or use buck2 build //:archive for both builds. The same compile command reads your edited file. The linker uses the new greet.o and the existing main.o. To see the new greeting without packing the archive, run bazel run //:greet or buck2 run //:greet.

//:greet and //:greet.o are named after the files, so a rule you write yourself can depend on them and still build after you convert again. A program is run with the shared libraries its own project built, and the ones those need. If a source sits below a directory that has its own build file, the rules name it by that package, and convert prints what the package has to export.

The generated rules list the files and commands from the build you recorded. If greet.c starts including a new header, its compile rule needs that header as an input. If you add extra.c to the Makefile, the build needs another compile rule. In either case, rerun convert --source from the repository root to regenerate the rules. Give the new recording its own directory, for example /tmp/project-recording-2. The derivation's source should not take in buck-out or Bazel's bazel-* links. A flake in a git repository leaves such untracked files out.

Every command that read one of your sources runs again after an edit. If it stops leaving a file or a directory it left before, leaves a new one in the package or under /build, keeps a file it removed before, changes a file it only read, or exits differently, its rule fails and the message says to convert again. The archive waits for every such command, and a named target like //:greet for those recorded before the command that made its file, so bazel run //:greet fails the same way the archive does. Bazel and Buck2 do not rerun a rule when only a file's mode changes, so chmod +x on a source reaches the output with the next change to the file's text.