technofab/nix-gitlab-ci
1
0
Fork 0
mirror of https://gitlab.com/TECHNOFAB/nix-gitlab-ci.git synced 2025-12-12 02:00:13 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: write docs & improve tooling

Browse source 
Squashed commit of the following:

commit 86eadd3ec42b7bce0dc5716d65798af95d0d8cbc
Author: technofab <admin@technofab.de>
Date:   Fri May 2 17:10:33 2025 +0200

docs(README): fix built with nix badge

commit f50057da69e89974f17bc37b5e140b2ef9f817f6
Author: technofab <admin@technofab.de>
Date:   Fri May 2 16:09:00 2025 +0200

ci: change back rule so docs only get deployed on main

commit ce02b043f4bd83c36285e5620e71701fc3bcc998
Author: technofab <admin@technofab.de>
Date:   Fri May 2 16:08:10 2025 +0200

docs: write docs and improve formatter etc.

commit e996b23cf877d8021759b782aa5996f5e2bf12ac
Author: technofab <admin@technofab.de>
Date:   Fri May 2 16:07:56 2025 +0200

docs: update README

commit 650f97b5608c32cf6cf66cc3fdd0965dc42e4860
Author: technofab <admin@technofab.de>
Date:   Wed Apr 23 21:05:14 2025 +0200

docs: add favicon

commit 67e1bfecbcaf0b8f7dad2eecfaccf774cc560874
Author: technofab <admin@technofab.de>
Date:   Wed Apr 23 20:53:44 2025 +0200

docs: initial setup
This commit is contained in:
technofab 2025-05-02 17:30:06 +02:00
parent fa6c454b14
commit cf80010d07
15 changed files with 631 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

9
README.md
View file

@ -1,4 +1,11 @@
# Nix Gitlab CI
# Nix GitLab CI
[![built with nix](https://img.shields.io/static/v1?logo=nixos&logoColor=white&label=&message=Built%20with%20Nix&color=41439a)](https://builtwithnix.org)
[![pipeline status](https://gitlab.com/TECHNOFAB/nix-gitlab-ci/badges/main/pipeline.svg)](https://gitlab.com/TECHNOFAB/nix-gitlab-ci/-/commits/main)
![License: MIT](https://img.shields.io/gitlab/license/technofab/nix-gitlab-ci)
[![Latest Release](https://gitlab.com/TECHNOFAB/nix-gitlab-ci/-/badges/release.svg)](https://gitlab.com/TECHNOFAB/nix-gitlab-ci/-/releases)
[![Support me](https://img.shields.io/badge/Support-me-orange)](https://tec.tf/#support)
[![Docs](https://img.shields.io/badge/Read-Docs-orange)](https://nix-gitlab-ci.projects.tf)
Flake module which allows generating a `.gitlab-ci.yml` from Nix.

50
docs/caching.md Normal file
View file

@ -0,0 +1,50 @@
# Caching
Nix GitLab CI supports several caching mechanisms to speed up your pipelines.
## GitLab Runner Cache
The runner cache strategy copies the new store paths into a directory `.nix-cache`,
which is then saved in the regular GitLab cache (technically runner cache).
It's also configured as a substituter automatically.
To enable, set the cache strategy to `runner`.
Configure it using these environment variables:
- `RUNNER_CACHE`: path to the runner cache (default `.nix-cache`)
!!! warning
This is very inefficient and should probably only be used for very very small
dependency counts. Otherwise it takes an eternity to save to cache.
## Cachix
Cachix is a hosted binary cache service that can significantly speed up Nix
builds by sharing build results.
To enable, set the cache strategy to `attic`.
Configure it using these environment variables:
- `CACHIX_CACHE`: name of the cache to use
- (`CACHIX_AUTH_TOKEN`): cachix client itself uses this for authentication
!!! warning
Cachix has not been tested. Feedback is appreciated :)
## Attic (Self-Hosted Cache)
Attic is a self-hosted, deduplicating binary cache. It's a great option if you
want more control over your caching infrastructure and to have the cache closer
to your runners.
To enable, set the cache strategy to `attic`.
Configure it using these environment variables:
- `ATTIC_SERVER`: URL of the server
- `ATTIC_CACHE`: name of the cache to use
- `ATTIC_TOKEN`: auth token from the attic server

41
docs/cicd_component.md Normal file
View file

@ -0,0 +1,41 @@
# CI/CD Component
The CI/CD Component has some inputs which configure defaults for Nix GitLab CI.
## `version`
- Type: `string`
Which version of the Nix CI image to use. Using a tag/version is recommended.
Will not do anything if a custom image is specified using `NIX_CI_IMAGE`.
## `cache_strategy`
- Type: `string`
- Default: `"auto"`
- Options: `auto` | `none` | `runner` | `cachix` | `attic`
Sets the default caching strategy.
- `auto`: dynamically selects the best strategy for every job based on env variables
- `none`: disables caching
- `runner`, `cachix` & `attic`: forces every job to use this strategy
Can be overridden by `NIX_CI_CACHE_STRATEGY`, see [Environment Variables](./environment_variables.md).
## `cache_files`
- Type: `array` (of strings)
- Default: `["flake.nix", "flake.lock"]`
Files to use as the cache key for the generated pipeline yaml.
If you use a file like `ci.nix` to define CI, add that here for example.
This makes sure that changes to your Nix CI configuration will invalidate the cache,
otherwise an old pipeline yaml might be used.
!!! warning
The value of this is used in `cache:key:files`, which currently only supports
a max of 2 entries. So use something like `["flake.*", "ci.nix"]` to match
`flake.lock`, `flake.nix` and `ci.nix`.
See [gitlab-org/gitlab#301161](https://gitlab.com/gitlab-org/gitlab/-/issues/301161)

107
docs/environment_variables.md Normal file
View file

@ -0,0 +1,107 @@
# Environment Variables
Nix GitLab CI is mostly controlled using environment variables.
This page outlines all the variables and their use case.
## `NIX_CI_IMAGE`
| | |
| ----------- | -------------------------------------------------------------------------- |
| Default | `registry.gitlab.com/technofab/nix-gitlab-ci/nix-ci@$[[ inputs.version ]]` |
| Description | Image to use for the jobs |
## `NIX_CI_PIPELINE_NAME`
| | |
| ----------- | ------------------------------------------------- |
| Default | N/A |
| Description | Explicitly request a pipeline to be built and ran |
| See also | [Multi Pipeline](./multi_pipeline.md) |
## `NIX_CI_DEFAULT_SOURCES`
| | |
| ----------- | -------------------------------------------------------------------------------------------------------------------------------- |
| Default | `.*` |
| Description | Regex to match `$CI_PIPELINE_SOURCE` against. If it matches, the `default` pipeline will be ran, otherwise `$CI_PIPELINE_SOURCE` |
| See also | [Multi Pipeline](./multi_pipeline.md) |
## `NIX_CI_FORCE_BUILD`
| | |
| ----------- | -------------------------------------------------------------------------------------- |
| Default | N/A |
| Description | Set to any non-empty value to force the `nix-ci:build` job to freshly build the config |
| See also | [Caching](./caching.md) |
## `NIX_CI_DISABLE_CACHE`
| | |
| ----------- | ------------------------------------------------------ |
| Default | N/A |
| Description | Set to any non-empty value to disable caching for jobs |
| See also | [Caching](./caching.md) |
## `NIX_CI_CACHE_STRATEGY`
| | |
| ----------- | --------------------------------------------------------------------------------- |
| Default | `$[[ inputs.cache_strategy ]]` -> defaults to `auto` |
| Description | Caching strategy to use. `auto` will select the strategy based on runner settings |
| See also | [Caching](./caching.md) |
## `NIX_CI_RUNNER_CACHE_STRATEGY`
| | |
| ----------- | ---------------------------------------------------------------- |
| Default | N/A |
| Description | Every runner can set it's own preferred cache strategy with this |
| See also | [Caching](./caching.md) |
## `NIX_CI_DEFAULT_CACHE_STRATEGY`
| | |
| ----------- | ------------------------------------------------------------------------------------------------- |
| Default | `none` |
| Description | If no runner cache strategy is set and the main strategy is set to auto, this will be the default |
| See also | [Caching](./caching.md) |
## `RUNNER_CACHE`
| | |
| ----------- | -------------------------------------- |
| Default | `.nix-cache` |
| Description | Path to directory for the runner cache |
| See also | [Caching](./caching.md) |
## `CACHIX_CACHE`
| | |
| ----------- | ------------------------------- |
| Default | N/A |
| Description | Name of the cachix cache to use |
| See also | [Caching](./caching.md) |
## `ATTIC_CACHE`
| | |
| ----------- | ------------------------------ |
| Default | N/A |
| Description | Name of the attic cache to use |
| See also | [Caching](./caching.md) |
## `ATTIC_SERVER`
| | |
| ----------- | ----------------------- |
| Default | N/A |
| Description | URL of the attic server |
| See also | [Caching](./caching.md) |
## `ATTIC_TOKEN`
| | |
| ----------- | ------------------------------- |
| Default | N/A |
| Description | API token from the attic server |
| See also | [Caching](./caching.md) |

18
docs/examples.md Normal file
View file

@ -0,0 +1,18 @@
# Example Configs
## V2
- [TECHNOFAB/nix-gitlab-ci](https://gitlab.com/TECHNOFAB/nix-gitlab-ci)
See `flake.nix` for some random example jobs.
- [TECHNOFAB/nixlets](https://gitlab.com/TECHNOFAB/nixlets)
- [TECHNOFAB/nixmkdocs](https://gitlab.com/TECHNOFAB/nixmkdocs)
- [TECHNOFAB/tofunix](https://gitlab.com/TECHNOFAB/tofunix)
## Old / V1
- [TECHNOFAB/coder-templates](https://gitlab.com/TECHNOFAB/coder-templates)
!!! note
Feel free to edit this page and add your project if you're using
Nix GitLab CI :)

BIN
docs/images/favicon.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.9 KiB

BIN
docs/images/logo.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 40 KiB

11
docs/index.md Normal file
View file

@ -0,0 +1,11 @@
# Nix GitLab CI
This project provides a Nix flake module that allows you to generate your `.gitlab-ci.yml` file directly from your Nix configuration.
## Features
- **Reproducibility:** Leverage Nix's strength in creating reproducible environments for your CI jobs.
- **Easy Dependency Management:** Easily include any package available in Nixpkgs or your own defined packages within your CI jobs using Nix.
- **Modularity:** Define and manage your CI configurations in a structured and modular way using Nix modules, making it easier to share and reuse CI logic across multiple projects.
This documentation will guide you through setting up and using Nix GitLab CI for your projects.

38
docs/kubernetes_runner.md Normal file
View file

@ -0,0 +1,38 @@
# Kubernetes Runner Setup
Using the GitLab Kubernetes runner allows your CI jobs to run as pods in a Kubernetes cluster.
Nix GitLab CI can be integrated with this setup, and using advanced configuration options like
`pod_spec` makes it easy to add runner specific caching.
Using this Runner configuration ...
```toml
[[runners.kubernetes.pod_spec]]
name = "nix-ci-cache-secrets"
patch = '''
containers:
- name: build
envFrom:
- secretRef:
name: nix-ci-cache-env
'''
```
... and a secret containing ...
```yaml
NIX_CI_RUNNER_CACHE_STRATEGY: attic
ATTIC_SERVER: <in-cluster-url> # example: http://atticd.<ns>.svc.cluster.local:8080
ATTIC_CACHE: ci # name however you want, just needs to exist
ATTIC_TOKEN: <token>
```
... makes your jobs automatically cache their Nix store paths to the in-cluster
attic when running with this runner.
Other runners could use cachix or no cache, you get the idea ;P
!!! note
This of course works with any executor where you can set environment
variables. This is just an example how to do it in Kubernetes easily.

31
docs/multi_pipeline.md Normal file
View file

@ -0,0 +1,31 @@
# Multiple Pipelines
With V2, Nix GitLab CI can generate different pipelines, depending on the
pipeline source (`$CI_PIPELINE_SOURCE`).
By default, no matter which source, the `default` pipeline is built and ran.
`$NIX_CI_PIPELINE` can override that, eg. when manually triggering a run.
To configure which source should be 1-to-1 translated to a pipeline with the
same name, set `$NIX_CI_DEFAULT_SOURCES` to a regex which explicitly does not
match these sources. Or set it to an impossible to match regex, then it will
always run the pipeline named after `$CI_PIPELINE_SOURCE`.
## Example 1: always run default
If you only have a single pipeline, you just have to call it `default`.
Everything else works out of the box.
## Example 2: default and merge_request_event
If you want the source `merge_request_event` to trigger a different pipeline,
name it like that and set `$NIX_CI_DEFAULT_SOURCES` to `^(merge_request_event)$`.
Now a merge request will run this pipeline, while everything else runs `default`.
## Example 3: default, push and web
Set `$NIX_CI_DEFAULT_SOURCES` to `^(push|web)$`.
## Example 4: always run the specific pipelines, never default
Set `$NIX_CI_DEFAULT_SOURCES` to any regex that never matches the sources,
like `a\A` or `nothing`.

75
docs/setup.md Normal file
View file

@ -0,0 +1,75 @@
# Setup
To integrate Nix GitLab CI into your project, you need to make two main changes:
1. Add the `nix-gitlab-ci` flake module to your `flake.nix`.
1. Include the necessary component in your `.gitlab-ci.yml`.
## Adding to `flake.nix`
In your project's `flake.nix`, add `nix-gitlab-ci` as an input and import its
flake module within your `flake-parts` configuration.
```nix title="flake.nix"
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; # Or your preferred nixpkgs branch/version
flake-parts.url = "github:hercules-ci/flake-parts";
# Add nix-gitlab-ci as an input
# recommendation: pin to a specific release/version
nix-gitlab-ci.url = "gitlab:TECHNOFAB/nix-gitlab-ci/<version>?dir=lib";
};
outputs = { nixpkgs, flake-parts, ... }@inputs:
flake-parts.lib.mkFlake { inherit inputs; } {
imports = [
# Import the nix-gitlab-ci flake module
inputs.nix-gitlab-ci.flakeModule
];
systems = [
"x86_64-linux"
"aarch64-linux"
# Add other systems you need
];
perSystem = { pkgs, ... }: {
# define your CI pipelines
# ci = { ... };
# pipelines."merge_request_event" = { ... };
};
};
}
```
Replace `<version>` with the specific version or commit hash of `nix-gitlab-ci`
you wish to use. Pinning to a specific version is highly recommended for
reproducibility and compatibility.
!!! warning
While the flake input is locked through `flake.lock`, the CI/CD component
will always use the latest commit of the reference. This means that by using
a branch like `main` as version for both, the CI/CD component will always use
the latest commit while your flake uses a fixed one.
This could result in drift between both, potentially breaking stuff.
## Including in `.gitlab-ci.yml`
Your `.gitlab-ci.yml` file will be minimal. Its primary role is to include the
`nix-gitlab-ci` component, which will then generate the full CI configuration
based on your Nix code.
```yaml title=".gitlab-ci.yml"
include:
- component: gitlab.com/TECHNOFAB/nix-gitlab-ci/nix-gitlab-ci@<version>
inputs:
# This input sets the Docker image tag used for the CI jobs.
# Use the same version as you pinned in your flake.nix for consistency.
version: <version>
```
Again, ensure `<version>` matches the version used in your `flake.nix`.
This component includes a job (`build:nix-ci`) that will evaluate your Nix
configuration and generate the `.gitlab-ci.yml` used for the pipeline run.

33
docs/usage.md Normal file
View file

@ -0,0 +1,33 @@
# Usage
To create a basic pipeline, configure it by setting `ci` in `perSystem`.
The schema is similar to the `.gitlab-ci.yml`, only jobs are defined differently:
```nix
ci = {
# Nix GitLab CI specific config, see `configType` in `flakeModule.nix`
config = {};
jobs = {
"job-a" = {};
"job-b" = {};
};
};
```
For every job, there are a couple of settings you can adjust aswell:
```nix
"job-a" = {
# see `jobType` in `flakeModule.nix`
nix = {
enable = true; # is this a nix-based job?
deps = []; # dependencies to install for this job
# for gitlab runner cache:
enable-runner-cache = false;
runner-cache-key = "";
};
};
```
Since V2 multiple pipelines are supported.
See [Multiple Pipelines](./multi_pipeline.md) for more.

59
docs/utilities.md Normal file
View file

@ -0,0 +1,59 @@
# Utilities
Nix GitLab CI provides a couple of utilities to help with development and
debugging.
## Disabling caching temporarily
Nix GitLab CI often utilizes caching mechanisms to speed up your pipelines
(see [Caching](./caching.md)).
However, there might be situations where you need to temporarily disable these
caches for a specific pipeline run, for example, to debug a caching issue or
ensure a clean build.
To disable most of the provided caches for a pipeline, set the environment
variable `NIX_CI_DISABLE_CACHE` to any non-empty value (e.g., `yes`, `true`, `1`)
when triggering the pipeline in the GitLab UI or via the API.
## Forcing a rebuild of the CI pipeline definition
The job responsible for generating the `.gitlab-ci.yml` from your Nix code
(`build:nix-ci`) might itself be cached. If you've made changes to your Nix CI
configuration and the pipeline doesn't seem to pick them up, the cached job
definition might be the reason.
You should first double check if all the Nix files you defined the CI config in
are specified in the `cache_files` CI/CD-component input
(see [CI/CD Component](./cicd_component.md) for more).
To force this specific job to rebuild and re-evaluate your Nix configuration,
set the environment variable `NIX_CI_FORCE_BUILD` when triggering the pipeline.
## Running jobs locally
One of the benefits of defining your CI jobs with Nix is the ability to run them
locally in an environment that closely mirrors the CI environment. This can
significantly speed up debugging and development.
You can run the script of any defined job locally using the `nix run` command.
The syntax is:
```sh
nix run .#gitlab-ci:pipeline:<pipeline name>:job:<job name>
```
Replace `<pipeline name>` with the name of the pipeline the job belongs to
(e.g., `default` for jobs defined under the `ci` attribute) and `<job name>`
with the name of the job you want to run.
This command will set up the environment with the specified `nix.deps` and
execute the job's `script`.
There is also an attribute `.#gitlab-ci:pipeline:<pipeline name>:job-deps:<job name>`.
Building this derivation will generate a shell script which exports the required
environment variables for the job, such as the `PATH` including all dependencies
and any custom environment variables that contain store paths (ensuring they are
correctly resolved across different architectures).
You can use this to inspect the environment that would be set up for a job without
running the full script.

34
flake.lock generated
View file

@ -169,6 +169,21 @@
"type": "github"
}
},
"mkdocs-material-umami": {
"locked": {
"lastModified": 1745840856,
"narHash": "sha256-1Ad1JTMQMP6YsoIKAA+SBCE15qWrYkGue9/lXOLnu9I=",
"owner": "technofab",
"repo": "mkdocs-material-umami",
"rev": "3ac9b194450f6b779c37b8d16fec640198e5cd0a",
"type": "gitlab"
},
"original": {
"owner": "technofab",
"repo": "mkdocs-material-umami",
"type": "gitlab"
}
},
"nix": {
"inputs": {
"flake-compat": [
@ -202,6 +217,23 @@
"type": "github"
}
},
"nix-mkdocs": {
"locked": {
"dir": "lib",
"lastModified": 1745175318,
"narHash": "sha256-eJOTw3SK4psqBPP2hNJ4D/13/oi/FLoM0tYKoCGVFP8=",
"owner": "technofab",
"repo": "nixmkdocs",
"rev": "6d6e0139060c896ae14de4b9c82335655a384643",
"type": "gitlab"
},
"original": {
"dir": "lib",
"owner": "technofab",
"repo": "nixmkdocs",
"type": "gitlab"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1733212471,
@ -301,6 +333,8 @@
"inputs": {
"devenv": "devenv",
"flake-parts": "flake-parts_2",
"mkdocs-material-umami": "mkdocs-material-umami",
"nix-mkdocs": "nix-mkdocs",
"nixpkgs": "nixpkgs_4",
"systems": "systems",
"treefmt-nix": "treefmt-nix"

128
flake.nix
View file

@ -8,6 +8,7 @@
imports = [
inputs.devenv.flakeModule
inputs.treefmt-nix.flakeModule
inputs.nix-mkdocs.flakeModule
./lib/flakeModule.nix
];
systems = import systems;
@ -25,7 +26,15 @@
mdformat.enable = true;
yamlfmt.enable = true;
};
settings.formatter.yamlfmt.excludes = ["templates/nix-gitlab-ci.yml"];
settings.formatter = {
yamlfmt.excludes = ["templates/nix-gitlab-ci.yml"];
mdformat.command = let
pkg = pkgs.python3.withPackages (p: [
p.mdformat
p.mdformat-mkdocs
]);
in "${pkg}/bin/mdformat";
};
};
devenv.shells.default = {
containers = pkgs.lib.mkForce {};
@ -40,9 +49,97 @@
};
};
};
doc = {
path = ./docs;
deps = pp: [
pp.mkdocs-material
(pp.callPackage inputs.mkdocs-material-umami {})
];
config = {
site_name = "Nix GitLab CI";
repo_name = "TECHNOFAB/nix-gitlab-ci";
repo_url = "https://gitlab.com/TECHNOFAB/nix-gitlab-ci";
edit_uri = "edit/main/docs/";
theme = {
name = "material";
features = ["content.code.copy" "content.action.edit"];
icon.repo = "simple/gitlab";
logo = "images/logo.png";
favicon = "images/favicon.png";
palette = [
{
scheme = "default";
media = "(prefers-color-scheme: light)";
primary = "deep orange";
accent = "orange";
toggle = {
icon = "material/brightness-7";
name = "Switch to dark mode";
};
}
{
scheme = "slate";
media = "(prefers-color-scheme: dark)";
primary = "deep orange";
accent = "orange";
toggle = {
icon = "material/brightness-4";
name = "Switch to light mode";
};
}
];
};
plugins = ["search" "material-umami"];
nav = [
{"Introduction" = "index.md";}
{"Setup" = "setup.md";}
{"Usage" = "usage.md";}
{"CI/CD Component" = "cicd_component.md";}
{"Environment Variables" = "environment_variables.md";}
{"Caching" = "caching.md";}
{"Multiple Pipelines" = "multi_pipeline.md";}
{"Utilities" = "utilities.md";}
{"Kubernetes Runner Example" = "kubernetes_runner.md";}
{"Example Configs" = "examples.md";}
];
markdown_extensions = [
{
"pymdownx.highlight".pygments_lang_class = true;
}
"pymdownx.inlinehilite"
"pymdownx.snippets"
"pymdownx.superfences"
"fenced_code"
"admonition"
];
extra.analytics = {
provider = "umami";
site_id = "28f7c904-db22-4c2b-9ee4-ed42e14b6db9";
src = "https://analytics.tf/umami";
domains = "nix-gitlab-ci.projects.tf";
feedback = {
title = "Was this page helpful?";
ratings = [
{
icon = "material/thumb-up-outline";
name = "This page is helpful";
data = "good";
note = "Thanks for your feedback!";
}
{
icon = "material/thumb-down-outline";
name = "This page could be improved";
data = "bad";
note = "Thanks for your feedback! Please leave feedback by creating an issue :)";
}
];
};
};
};
};
# should set the "default" pipeline
ci = {
stages = ["test"];
stages = ["test" "build" "deploy"];
jobs = {
"test" = {
stage = "test";
@ -73,6 +170,31 @@
"echo \"This job will not be modified to use nix\""
];
};
# -- actually useful jobs --
"docs" = {
stage = "build";
script = [
# sh
''
nix build .#docs:default
mkdir -p public
cp -r result/. public/
''
];
artifacts.paths = ["public"];
};
"pages" = {
nix.enable = false;
image = "alpine:latest";
stage = "deploy";
script = ["true"];
artifacts.paths = ["public"];
rules = [
{
"if" = "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH";
}
];
};
};
};
pipelines."non-default" = {
@ -145,6 +267,8 @@
systems.url = "github:nix-systems/default-linux";
devenv.url = "github:cachix/devenv";
treefmt-nix.url = "github:numtide/treefmt-nix";
nix-mkdocs.url = "gitlab:technofab/nixmkdocs?dir=lib";
mkdocs-material-umami.url = "gitlab:technofab/mkdocs-material-umami";
};
nixConfig = {