Who use a build system?

Build systems are mainly used by software developers as part of their workflow. The build system is also used by users of a software to compile it for their system. Although the processes are the same, the goals are different.

Developers want quick iterations to test changes they applied. They also want to configure the build systems in a way that integrated well with their development tools such as their editor and debugger.

However, users of a software just want to configure the build system for their installation on their system. They don't really care about various integration to development tools.

So what do these have in common? Configuration. The way I see it, a build system is a tool in which you can specify a workflow of processes to be done. Typically, the workflow is defined by the developers. However, the workflow itself should be easily configurable by its users to match their needs. e.g., installation sites, installation phases, variants of the software.

What should do a good build system?

It should be easy to use from the command line, requiring absolutely nothing except the scripts found in the project itself. It's self-contained at the exception of the script interpreter.
It's portable across distribution. If the build system is self-contained, it's just a matter of having the interpreter installed on the system.
It should be flexible enough so that developers can add features to it easily. Developers should not fight with their tools.
Users should be able to configure the software easily with it.

Introducing Gbuild

I've been working part time (~1 h) a week on this project of mine. What I want is a tool that allows me to define custom commands for my projects. I also want an easy to read DSL for defining my project in a declarative way.

Gbuild is a collection of Guile modules.

Here's the project definition for Gbuild itself.

(use-modules
 (gbuild project guile)
 (gbuild script))

(define scripts
  (let ((base-script
         (script
          (name "<invalid>")
          (install? #t)
          (file (lambda (this)
                  (string-append
                   "scripts/" (script-name this))))))
        (base-tools
         (script
          (name "<invalid>")
          (install? #f)
          (file (lambda (this)
                  (string-append
                   "tools/" (script-name this)))))))
    (list
     (script
      (specialize base-script)
      (name "gbuild"))
     (script
      (specialize base-tools)
      (name "guix-build")))))

(guile-project
 (authors '("Olivier Dion"))
 (name "gbuild")
 (version "1.0")
 (licenses '(gpl3+))
 (template-files '(("build-aux/pre-inst-env.in" -> "pre-inst-env" +x)))
 (export-configuration '(("gbuild/config.scm" . "gbuild config")))
 (inputs '("guile@3.0.8"))
 (guile-roots '("gbuild"))
 (tests-root "tests")
 (scripts scripts))

The guile-project syntax expands to something that will make an object of type <guix-project>. This type inherit from the <project> type. Since these are GOOPS objects, one can define methods for them. Actually, one define project commands. This is the same as defining a method, but it's tag so that it can be called from the gbuild's REPL.

Here's for example the default build command for a Guile project.

(define-project-command (build (this <guile-project>))

  (define (scm->go pathname)
    (string-replace-substring pathname ".scm" ".go"))

  (define (compile path out)
    (progress "GUILD ~a" path)
    (compile-file
     path
     #:to 'bytecode
     #:output-file out))

  (define scheme-files
    (find-files (project-guile-roots this) '("*.scm")))

  (define go-files (map scm->go scheme-files))

  (with-rules-graph (make-rules-graph)
    (make-rule (lambda () (progress "DONE")) "entrypoint" go-files)
    (for-each
     (lambda (scm go)
       (make-rule (lambda () (compile scm go)) go (list scm)))
     scheme-files go-files)
    (run-graph)))

Since this is just a GOOPS method, developers are free to override it.

The Gbuild REPL

What I also want is to enter a REPL for mananing the project. Typically, one would simply do gbuild repl in the project root to enter a Guile REPL with all project commands imported in the environment.

So now inside that REPL, one can do stuff like this:

scheme@(gbuild)> build
...
scheme@(gbuild)> check
...
scheme@(gbuild)> clear
...
scheme@(gbuild)> custom-command
...

Since we're in a session, the dependencies tree of the build system can be built once and used continuously. One can even use inotify(2) to dynamically inject dependencies in the rules graph.

Talking about rules graph, I've been thinking of using propagators instead.

What about configuration?

I see two way of configuring a project. First there should be presets. That way, developers can defined presets configuration for debugging and for release. Distributions can also define their presets. Typically, a preset is just a file with command line arguments in it.

The second way is to manually pass the arguments to the configure script. The configure script itself should be loading the project with gbuild and configure it!

#!/bin/sh
exec guile --no-auto-compile -e main -s "$0" "$@"
!#

(use-modules
 (gbuild configure))

(define (main args)
  (configure ".project.scm" (cdr args)))

What's next?

Right now all of this are drafted ideas. When I will have more time to give, I will try to implement them and see along the way what's missing.

A build system entirely in Guile