How to use it?
To record a build, you need Nix with flakes enabled, an x86-64 Linux machine with access to KVM, and Rewind on your PATH. To build the generated rules, use Bazel or Buck2 on a machine with bubblewrap and the Nix store paths from the recording.
Rebuild a Nix package
convert records the Nix build of GNU hello and writes Bazel and Buck2
rules in hello/workspace. To check a step such as compiling
hello.c, it runs the compiler again in a sandbox with the inputs GCC
used. It checks that GCC exits with the same status and produces the same object
file. If a step cannot reproduce a file the build needs, the generated workspace
can supply that file from the recording.
nix run github:fzakaria/genrulify -- convert nixpkgs#hello hello
cd hello/workspace
A successful //:archive build with Bazel or Buck2 reproduces the recorded Nix archive
(NAR) byte for byte. The sha256sum command below prints the same digest
that convert printed for the Nix output.
Bazel
bazel build //:archive
sha256sum bazel-bin/archive.nar
Buck2
buck2 build //:archive --out hello.nar
sha256sum hello.nar
Explore the recorded hello build.
Build from sources you can edit
The repository includes a small C project.
Its Makefile compiles main.c and greet.c, then links the
two objects. The .#project derivation uses examples/project
as its source directory. Run these commands from the root of a genrulify checkout.
nix run . -- convert --source examples/project .#project /tmp/project-recording
cd examples/project
bazel build //:archive
--source tells genrulify where the working sources live. It checks
that they match the derivation's source, then writes the rules, their scripts
and their manifests under genrulify/. The BUILD.bazel and
BUCK it writes beside your sources load those rules. If you already
have a build file of your own, genrulify leaves it alone and prints the two
lines that load the rules from it. Each compile reads its source from your checkout.
The last argument names a new directory for the recording and graph.
Change the greeting in greet.c and run bazel build //:archive
again, or use buck2 build //:archive for both builds. The same compile
command reads your edited file. The linker uses the new greet.o and the
existing main.o. To see the new greeting without packing the archive,
run bazel run //:greet or buck2 run //:greet.
//:greet and //:greet.o are named after the files, so a rule
you write yourself can depend on them and still build after you convert again.
A program is run with the shared libraries its own project built, and the ones
those need. If a source sits
below a directory that has its own build file, the rules name it by that package,
and convert prints what the package has to export.
The generated rules list the files and commands from the build you recorded.
If greet.c starts including a new header, its compile rule needs that
header as an input. If you add extra.c to the Makefile, the build needs
another compile rule. In either case, rerun convert --source from
the repository root to regenerate the rules. Give the new recording its own
directory, for example /tmp/project-recording-2. The derivation's
source should not take in buck-out or Bazel's bazel-* links.
A flake in a git repository leaves such untracked files out.
Every command that read one of your sources runs again after an edit. If it stops
leaving a file or a directory it left before, leaves a new one in the package or
under /build, keeps a file it removed before, changes a file it only
read, or exits differently, its rule fails and the message says to convert again.
The archive waits for every such command, and a named target like
//:greet for those recorded before the command that made its file, so
bazel run //:greet fails the same way the archive does.
Bazel and Buck2 do not rerun a rule when
only a file's mode changes, so chmod +x on a source reaches the output
with the next change to the file's text.