Kennel

Kennel is the deployment platform for ScottyLabs. When you push code to a repository, kennel builds it with Nix, deploys it, and routes traffic to it.

What it does

Builds your project with Nix when you push to any branch
Deploys services as systemd units and static sites via Caddy
Provisions per-deployment databases (PostgreSQL), caches (Valkey), and object storage (Garage)
Resolves secrets from OpenBao via secretspec
Generates HTTPS URLs for every deployment, including PR previews
Tears down deployments and their resources when branches are deleted or PRs are closed

Your project’s devenv.nix declares what it needs to run: processes, databases, secrets, static sites. Kennel evaluates this configuration, builds the Nix packages, and deploys everything with isolated resources per branch.

Every deployment gets a URL at {project}-{branch}.scottylabs.net. Production deployments can also have custom domains.

Getting started

See the deploying a project guide.

Deploying a Project

This guide walks through setting up a ScottyLabs project for deployment with kennel.

Prerequisites

A repository in the ScottyLabs Forgejo organization
devenv installed locally
A flake.nix and devenv.nix in your project root

1. Import the shared module

Add the ScottyLabs devenv input to your devenv.yaml:

inputs:
  scottylabs:
    url: github:ScottyLabs/devenv
  rust-overlay:
    url: github:oxalica/rust-overlay
    inputs:
      nixpkgs:
        follows: nixpkgs
  treefmt-nix:
    url: github:numtide/treefmt-nix
  git-hooks:
    url: github:cachix/git-hooks.nix
    inputs:
      nixpkgs:
        follows: nixpkgs

Import it in your devenv.nix:

{ pkgs, config, inputs, ... }:
{
  imports = [ inputs.scottylabs.devenvModules.default ];

  scottylabs = {
    enable = true;
    project.name = "my-project";
  };
}

2. Set up direnv and .gitignore

Create an .envrc to automatically activate the devenv environment when you enter the project directory:

eval "$(devenv direnvrc)"
use devenv

Then allow it:

direnv allow

Add a .gitignore for generated and local-only files:

# Nix / devenv
.devenv/
.devenv.flake.nix
.pre-commit-config.yaml
result
result-*

# AI
.mcp.json
.claude

# direnv
.direnv/

# Rust
target/
.cargo/

# OS
.DS_Store
rustc-ice-*.txt

Add any project-specific entries as needed (e.g., sites/docs/book/ for mdbook output, node_modules/ for JS projects).

3. Declare what to deploy

Add kennel options to your devenv.nix to tell kennel what your project produces.

For a backend service:

scottylabs.kennel.services.api = {
  customDomain = "api.my-project.scottylabs.org";
};

processes.api = {
  exec = "${pkgs.my-project}/bin/api";
  ready.http.get = { port = 8080; path = "/health"; };
};

For a static site:

scottylabs.kennel.sites.docs = {
  spa = false;
};

The site name (docs) must match a package in your flake.nix outputs. Kennel builds it with nix build .#packages.{system}.docs.

4. Enable infrastructure

If your project needs a database:

scottylabs.postgres.enable = true;

This gives you a local PostgreSQL instance in development and a provisioned per-deployment database in production. Your app reads DATABASE_URL from the environment in both cases.

5. Enable kennel in governance

In the ScottyLabs governance repository, set the kennel flag for your project. Governance provisions the webhook that connects your repository to kennel.

6. Push

Push to any branch. Kennel receives the webhook, builds your project, and deploys it. Your deployment will be available at:

my-project-main.scottylabs.net for the main branch
my-project-pr-42.scottylabs.net for PR #42
my-project-feature-x.scottylabs.net for a feature branch

Flake packages

Your flake.nix must expose packages that kennel can build. The package names must match the keys in scottylabs.kennel.services and scottylabs.kennel.sites:

packages = forAllSystems (system:
  let pkgs = pkgsFor system;
  in {
    api = cargoNix.workspaceMembers.my-project.build;
    docs = pkgs.stdenv.mkDerivation { ... };
    default = self.packages.${system}.api;
  }
);

Kennel builds each package with nix build .#packages.{system}.{name}.

Secrets

Kennel resolves secrets from OpenBao via secretspec. Secrets are injected as environment variables at deploy time and never written to disk or stored in the database.

Declaring secrets

Create a secretspec.toml in your project root:

[project]
name = "my-project"

[secrets]
JWT_SECRET = { description = "JWT signing key", required = true }
STRIPE_KEY = { description = "Stripe API key", required = true }

[profiles.preview]
STRIPE_KEY = { description = "Stripe test key", required = false }

The [secrets] section defines secrets shared across all profiles. Profile sections can override requirements, for example making secrets optional in preview deployments.

Local development

Enable secrets in your devenv.nix:

scottylabs.secrets.enable = true;

This configures devenv’s native secretspec integration. Secrets are resolved and exported into the shell environment when you enter devenv shell. If you haven’t authenticated to OpenBao yet:

bao login -method=oidc

After authenticating, re-enter the shell and your secrets will be available as environment variables.

Managing secrets

Set a secret for the default (dev) profile:

secretspec set JWT_SECRET

Set a secret for a specific profile:

secretspec set -P prod STRIPE_KEY
secretspec set -P preview STRIPE_KEY

Verify all required secrets are present:

secretspec check
secretspec check -P prod

Production

Kennel authenticates to OpenBao with a service token provided via VAULT_TOKEN in its environment file. It resolves secrets for each deployment using the profile matching the branch:

Branch	Profile
`main`	`prod`
`staging`	`staging`
`dev`	`dev`
`pr-*`	`preview`

If a required secret cannot be resolved, the deployment fails.

PR Deployments

Every pull request gets its own isolated deployment with its own database, cache, and URL.

Lifecycle

PR opened: kennel builds the PR branch, provisions resources, and deploys. The deployment is available at {project}-pr-{number}.scottylabs.net.
Push to PR: kennel rebuilds. Unchanged services (same nix store path) are skipped. Changed services are redeployed.
PR closed: kennel tears down all deployments for the branch, deprovisions resources (drops the database, flushes the cache, deletes the storage bucket), and removes the Caddy route.

Resource isolation

Each PR deployment gets:

Its own PostgreSQL database (kennel_{project}_{branch})
Its own Valkey DB number
Its own Garage S3 bucket and API key

Connection strings are injected as environment variables. Your application code does not need to know whether it is running in production or a PR preview.

Expiry

PR deployments that have had no activity for 7 days are hibernated: the process is stopped but the database is kept. After 30 days, the deployment and its resources are fully torn down.

URLs

PR URLs follow the flat scheme {project}-pr-{number}.scottylabs.net, covered by a single wildcard DNS record. No per-deployment DNS management is needed.

Architecture

Kennel runs as a single binary with four main responsibilities: receiving webhooks, building Nix packages, deploying them, and reconciling desired state against actual state.

Request flow

Git push -> Webhook -> Build (nix) -> Deploy (systemd + Caddy) -> Live

Forgejo sends a webhook to kennel’s single /webhook endpoint.
Kennel parses the repository name from the payload, verifies the HMAC signature, and creates a build record.
The build worker clones the repo, runs devenv build scottylabs.kennel.config to discover declared services and sites, then runs nix build for each package.
The reconciler picks up the completed build, provisions resources (database, cache, storage), resolves secrets from OpenBao, starts a systemd transient unit for services, and adds a Caddy route for each deployment.
Caddy serves traffic over HTTPS with on-demand TLS.

Delegation

Kennel delegates process supervision to systemd and HTTP routing to Caddy, keeping the core focused on build orchestration and resource provisioning.

Systemd transient units are created via D-Bus using the zbus crate. Units are placed in the kennel.slice cgroup for aggregate accounting. Transient units survive kennel crashes since they are independent of the kennel process.

Caddy routes are managed via the admin API. Each deployment gets a route identified by @id for individual add/remove operations. Caddy handles TLS certificate provisioning, HTTP/3, static file serving, reverse proxying, and SPA fallback.

Reconciliation

A single reconciliation loop handles all deployment convergence. It runs on startup, when signaled by a webhook or build completion, and on a periodic 30-second timer.

The reconciler compares desired state (deployment rows in the database) against actual state (systemd units and Caddy routes) and converges:

A deployment row with no running unit gets its unit started.
A running unit with no deployment row gets stopped.
All Caddy routes are re-added on each pass since Caddy config is ephemeral.

There are no intermediate deployment states like “deploying” or “tearing down” that could get stuck. A deployment either has a row in the database or it doesn’t, which eliminates stuck-state bugs by construction.

State

Kennel stores state in SQLite with three tables:

projects – registered repositories with webhook secrets
builds – build queue and history (queued, building, built, done, failed, cancelled)
deployments – active deployments with store paths, domains, unit names, and ports

Runtime process state (running, stopped, failed) is owned by systemd and queried via D-Bus. Routing state is owned by Caddy and queried via the admin API. Kennel’s database only tracks intent.

Crate structure

kennel – main binary with webhook, build, deploy, reconcile, caddy client, systemd client, store, secrets
kennel-config – shared types, constants, environment enum
kennel-provision – resource provisioning trait and implementations (PostgreSQL, Valkey, Garage)
entity – SeaORM generated entities
migration – SQLite schema migrations

devenv Options

These options are provided by the shared ScottyLabs devenv module. Import it in your devenv.nix:

imports = [ inputs.scottylabs.devenvModules.default ];

`scottylabs`

`scottylabs.enable`

Enable the shared ScottyLabs development configuration. Required for all other scottylabs.* options to take effect.

Type: bool, default: false

`scottylabs.project.name`

Project name. Used for database naming, log filtering, and secrets path resolution.

Type: str, required when scottylabs.enable = true

`scottylabs.rust`

`scottylabs.rust.enable`

Enable the Rust development toolchain. Configures nightly Rust with cranelift, clippy, rustfmt, the cargo-nix-update pre-commit hook, and crate2nix integration.

Type: bool, default: false

`scottylabs.rust.cranelift.excludePackages`

Crate names forced to the LLVM backend instead of cranelift. Some crates use features that cranelift does not support (FFI symbol emission, linker sections).

Type: listOf str, default: [ "aws-lc-sys" "aws-lc-rs" "rustls" ]

`scottylabs.bun`

`scottylabs.bun.enable`

Enable the Bun/JavaScript development toolchain. Adds bun, oxfmt for formatting, and oxlint for linting.

Type: bool, default: false

`scottylabs.postgres`

`scottylabs.postgres.enable`

Enable a local PostgreSQL 18 instance with Unix socket access. Creates an initial database named after scottylabs.project.name and exports DATABASE_URL into the shell environment.

Type: bool, default: false

`scottylabs.postgres.extensions`

PostgreSQL extensions as a function of the extensions set.

Type: function, default: e: [ e.pg_uuidv7 ]

`scottylabs.sqlite`

`scottylabs.sqlite.enable`

Enable SQLite for local development. Adds the sqlite package and exports DATABASE_PATH pointing to a database file in the devenv state directory.

Type: bool, default: false

`scottylabs.secrets`

`scottylabs.secrets.enable`

Enable secretspec integration for local secret resolution. Configures devenv’s native secretspec support with the ScottyLabs OpenBao server as the provider. Adds the bao CLI for authentication.

Type: bool, default: false

`scottylabs.secrets.host`

OpenBao server hostname. Used to derive both BAO_ADDR (for the bao CLI) and SECRETSPEC_PROVIDER (for secretspec).

Type: str, default: "secrets2.scottylabs.org"

`scottylabs.secrets.profile`

secretspec profile for local development.

Type: str, default: "dev"

`scottylabs.kennel`

`scottylabs.kennel.services`

Backend services deployed by kennel. Each key must match a devenv process name. Kennel builds the corresponding flake package and deploys it as a systemd transient unit.

Type: attrsOf submodule

Each service accepts:

customDomain (nullOr str, default: null) – custom domain for this service

`scottylabs.kennel.sites`

Static sites deployed by kennel. Each key names a site. Kennel builds the corresponding flake package and serves it via Caddy’s file server.

Type: attrsOf submodule

Each site accepts:

spa (bool, default: false) – serve index.html for all routes
customDomain (nullOr str, default: null) – custom domain for this site

`scottylabs.kennel.config`

Read-only. The generated kennel.json derivation that the kennel builder evaluates at build time. You do not set this directly.

Type: package

`scottylabs.claude`

`scottylabs.claude.enable`

Enable Claude Code integration. Generates the .mcp.json configuration with the devenv MCP server.

Type: bool, default: true

NixOS Module

The kennel NixOS module configures the kennel service, Caddy, systemd integration, and resource provisioning on a NixOS host.

{ kennel, ... }:
{
  imports = [ kennel.nixosModules.default ];

  services.kennel = {
    enable = true;
    package = kennel.packages.x86_64-linux.kennel;
    devenvPackage = kennel.packages.x86_64-linux.devenv;
    webhookSecretFile = config.age.secrets.kennel-webhook.path;
    environmentFile = config.age.secrets.kennel.path;

    domains = {
      ephemeral = "scottylabs.net";
      cloudflare.zones."scottylabs.org" = "<zone-id>";
    };

    resources.postgres = {
      enable = true;
      socketDir = "/run/postgresql";
    };

    secrets = {
      enable = true;
      vaultEndpoint = "https://secrets2.scottylabs.org";
    };
  };
}

Options

`services.kennel.enable`

Enable the kennel deployment platform.

Type: bool, default: false

`services.kennel.package`

The kennel package to use.

Type: package

`services.kennel.devenvPackage`

The devenv package. The build worker uses devenv build to evaluate project kennel configs from their devenv.nix.

Type: package

`services.kennel.environmentFile`

Path to an environment file containing secrets like VAULT_TOKEN, CACHIX_AUTH_TOKEN, and GARAGE_ADMIN_TOKEN. Loaded by systemd before the service starts.

Type: nullOr path, default: null

`services.kennel.api.host` / `services.kennel.api.port`

API server bind address and port.

Type: str / port, defaults: "0.0.0.0" / 3000

`services.kennel.webhookSecretFile`

Path to a file containing the HMAC secret used to verify all incoming webhooks. This is a single secret shared across all projects, provisioned by governance.

Type: path

`services.kennel.domains.ephemeral`

Base domain for auto-generated deployment URLs. A wildcard DNS record should point *.{domain} to the kennel server.

Type: str, default: "scottylabs.net"

`services.kennel.domains.cloudflare.zones`

Map of domain names to Cloudflare zone IDs. Used for creating DNS records for custom domains.

Type: attrsOf str, default: {}

`services.kennel.domain`

Public domain for the kennel API and webhook endpoint. The module configures a Caddy virtualhost with automatic TLS for this domain, reverse-proxying to the API server.

Type: str, default: "kennel.scottylabs.org"

`services.kennel.caddy.adminUrl`

Caddy admin API URL.

Type: str, default: "http://localhost:2019"

`services.kennel.builder.maxConcurrentBuilds`

Maximum number of concurrent nix builds.

Type: int, default: 2

`services.kennel.builder.workDir`

Build working directory.

Type: path, default: "/var/lib/kennel/builds"

`services.kennel.builder.cachix.enable` / `services.kennel.builder.cachix.cacheName`

Enable pushing build artifacts to a Cachix binary cache.

`services.kennel.resources.postgres`

Enable PostgreSQL resource provisioning. Kennel creates a database per deployment using the specified socket directory for peer authentication.

enable (bool, default: false)
socketDir (path, default: "/run/postgresql")

`services.kennel.resources.valkey`

Enable Valkey resource provisioning. Kennel allocates a DB number per deployment from the shared instance.

enable (bool, default: false)
socketPath (path, default: "/run/valkey/valkey.sock")

`services.kennel.resources.garage`

Enable Garage S3 resource provisioning. Kennel creates a bucket and API key per deployment. Requires GARAGE_ADMIN_TOKEN in the environment file.

enable (bool, default: false)
adminEndpoint (str, default: "http://localhost:3903")
s3Endpoint (str, default: "http://localhost:3900")

`services.kennel.secrets`

Enable secretspec/OpenBao secret resolution at deploy time.

enable (bool, default: false)
vaultEndpoint (str, default: "https://secrets2.scottylabs.org")

What the module configures

A systemd service for kennel with Delegate=yes for cgroup v2 access
A polkit rule allowing the kennel user to create transient systemd units via D-Bus
A kennel.slice for all managed deployment units
A Caddy virtualhost for the kennel domain with automatic TLS, plus the admin API for dynamic route management
tmpfiles rules for /var/lib/kennel subdirectories
Firewall rules for ports 80 and 443
Cachix binary cache substituter

Keyboard shortcuts

Kennel