technofab/nix-gitlab-ci
1
0
Fork 0
mirror of https://gitlab.com/TECHNOFAB/nix-gitlab-ci.git synced 2025-12-11 17:50:08 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: add remaining docs back with some additions/improvements

Browse source
This commit is contained in:
technofab 2025-09-02 11:49:35 +02:00
parent f147295418
commit 0bd75fd1bb
No known key found for this signature in database
12 changed files with 538 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

50
docs/caching.md Normal file
View file

@ -0,0 +1,50 @@
# Caching
Nix GitLab CI supports several caching mechanisms to speed up your pipelines.
## GitLab Runner Cache
The runner cache strategy copies the new store paths into a directory `.nix-cache`,
which is then saved in the regular GitLab cache (technically runner cache).
It's also configured as a substituter automatically.
To enable, set the cache strategy to `runner`.
Configure it using these environment variables:
- `RUNNER_CACHE`: path to the runner cache (default `.nix-cache`)
!!! warning
This is very inefficient and should probably only be used for very very small
dependency counts. Otherwise it takes an eternity to save to cache.
## Cachix
Cachix is a hosted binary cache service that can significantly speed up Nix
builds by sharing build results.
To enable, set the cache strategy to `attic`.
Configure it using these environment variables:
- `CACHIX_CACHE`: name of the cache to use
- (`CACHIX_AUTH_TOKEN`): cachix client itself uses this for authentication
!!! warning
Cachix has not been tested. Feedback is appreciated :)
## Attic (Self-Hosted Cache)
Attic is a self-hosted, deduplicating binary cache. It's a great option if you
want more control over your caching infrastructure and to have the cache closer
to your runners.
To enable, set the cache strategy to `attic`.
Configure it using these environment variables:
- `ATTIC_SERVER`: URL of the server
- `ATTIC_CACHE`: name of the cache to use
- `ATTIC_TOKEN`: auth token from the attic server

41
docs/cicd_component.md Normal file
View file

@ -0,0 +1,41 @@
# CI/CD Component
The CI/CD Component has some inputs which configure defaults for Nix GitLab CI.
## `version`
- Type: `string`
Which version of the Nix CI image to use. Using a tag/version is recommended.
Will not do anything if a custom image is specified using `NIX_CI_IMAGE`.
## `cache_strategy`
- Type: `string`
- Default: `"auto"`
- Options: `auto` | `none` | `runner` | `cachix` | `attic`
Sets the default caching strategy.
- `auto`: dynamically selects the best strategy for every job based on env variables
- `none`: disables caching
- `runner`, `cachix` & `attic`: forces every job to use this strategy
Can be overridden by `NIX_CI_CACHE_STRATEGY`, see [Environment Variables](./environment_variables.md).
## `cache_files`
- Type: `array` (of strings)
- Default: `["flake.nix", "flake.lock"]`
Files to use as the cache key for the generated pipeline yaml.
If you use a file like `ci.nix` to define CI, add that here for example.
This makes sure that changes to your Nix CI configuration will invalidate the cache,
otherwise an old pipeline yaml might be used.
!!! warning
The value of this is used in `cache:key:files`, which currently only supports
a max of 2 entries. So use something like `["flake.*", "ci.nix"]` to match
`flake.lock`, `flake.nix` and `ci.nix`.
See [gitlab-org/gitlab#301161](https://gitlab.com/gitlab-org/gitlab/-/issues/301161)

107
docs/environment_variables.md Normal file
View file

@ -0,0 +1,107 @@
# Environment Variables
Nix GitLab CI is mostly controlled using environment variables.
This page outlines all the variables and their use case.
## `NIX_CI_IMAGE`
| | |
| ----------- | -------------------------------------------------------------------------- |
| Default | `registry.gitlab.com/technofab/nix-gitlab-ci/nix-ci@$[[ inputs.version ]]` |
| Description | Image to use for the jobs |
## `NIX_CI_PIPELINE_NAME`
| | |
| ----------- | ------------------------------------------------- |
| Default | N/A |
| Description | Explicitly request a pipeline to be built and ran |
| See also | [Multi Pipeline](./multi_pipeline.md) |
## `NIX_CI_DEFAULT_SOURCES`
| | |
| ----------- | -------------------------------------------------------------------------------------------------------------------------------- |
| Default | `.*` |
| Description | Regex to match `$CI_PIPELINE_SOURCE` against. If it matches, the `default` pipeline will be ran, otherwise `$CI_PIPELINE_SOURCE` |
| See also | [Multi Pipeline](./multi_pipeline.md) |
## `NIX_CI_FORCE_BUILD`
| | |
| ----------- | -------------------------------------------------------------------------------------- |
| Default | N/A |
| Description | Set to any non-empty value to force the `nix-ci:build` job to freshly build the config |
| See also | [Caching](./caching.md) |
## `NIX_CI_DISABLE_CACHE`
| | |
| ----------- | ------------------------------------------------------ |
| Default | N/A |
| Description | Set to any non-empty value to disable caching for jobs |
| See also | [Caching](./caching.md) |
## `NIX_CI_CACHE_STRATEGY`
| | |
| ----------- | --------------------------------------------------------------------------------- |
| Default | `$[[ inputs.cache_strategy ]]` -> defaults to `auto` |
| Description | Caching strategy to use. `auto` will select the strategy based on runner settings |
| See also | [Caching](./caching.md) |
## `NIX_CI_RUNNER_CACHE_STRATEGY`
| | |
| ----------- | ---------------------------------------------------------------- |
| Default | N/A |
| Description | Every runner can set it's own preferred cache strategy with this |
| See also | [Caching](./caching.md) |
## `NIX_CI_DEFAULT_CACHE_STRATEGY`
| | |
| ----------- | ------------------------------------------------------------------------------------------------- |
| Default | `none` |
| Description | If no runner cache strategy is set and the main strategy is set to auto, this will be the default |
| See also | [Caching](./caching.md) |
## `RUNNER_CACHE`
| | |
| ----------- | -------------------------------------- |
| Default | `.nix-cache` |
| Description | Path to directory for the runner cache |
| See also | [Caching](./caching.md) |
## `CACHIX_CACHE`
| | |
| ----------- | ------------------------------- |
| Default | N/A |
| Description | Name of the cachix cache to use |
| See also | [Caching](./caching.md) |
## `ATTIC_CACHE`
| | |
| ----------- | ------------------------------ |
| Default | N/A |
| Description | Name of the attic cache to use |
| See also | [Caching](./caching.md) |
## `ATTIC_SERVER`
| | |
| ----------- | ----------------------- |
| Default | N/A |
| Description | URL of the attic server |
| See also | [Caching](./caching.md) |
## `ATTIC_TOKEN`
| | |
| ----------- | ------------------------------- |
| Default | N/A |
| Description | API token from the attic server |
| See also | [Caching](./caching.md) |

13
docs/examples.md Normal file
View file

@ -0,0 +1,13 @@
# Example Configs
- [TECHNOFAB/nix-gitlab-ci](https://gitlab.com/TECHNOFAB/nix-gitlab-ci)
- [TECHNOFAB/nixlets](https://gitlab.com/TECHNOFAB/nixlets)
- [TECHNOFAB/nixible](https://gitlab.com/TECHNOFAB/nixible)
- [TECHNOFAB/nixmkdocs](https://gitlab.com/TECHNOFAB/nixmkdocs)
- [TECHNOFAB/tofunix](https://gitlab.com/TECHNOFAB/tofunix)
- [TECHNOFAB/nixtest](https://gitlab.com/TECHNOFAB/nixtest)
!!! note
Feel free to edit this page and add your project if you're using
Nix GitLab CI :)

11
docs/index.md
View file

@ -1,2 +1,11 @@
# Nix-GitLab-CI # Nix GitLab CI
This project provides a Nix flake module that allows you to generate your `.gitlab-ci.yml` file directly from your Nix configuration.
## Features
- **Reproducibility:** Leverage Nix's strength in creating reproducible environments for your CI jobs.
- **Easy Dependency Management:** Easily include any package available in Nixpkgs or your own defined packages within your CI jobs using Nix.
- **Modularity:** Define and manage your CI configurations in a structured and modular way using Nix modules, making it easier to share and reuse CI logic across multiple projects.
This documentation will guide you through setting up and using Nix GitLab CI for your projects.

38
docs/kubernetes_runner.md Normal file
View file

@ -0,0 +1,38 @@
# Kubernetes Runner Setup
Using the GitLab Kubernetes runner allows your CI jobs to run as pods in a Kubernetes cluster.
Nix GitLab CI can be integrated with this setup, and using advanced configuration options like
`pod_spec` makes it easy to add runner specific caching.
Using this Runner configuration ...
```toml
[[runners.kubernetes.pod_spec]]
name = "nix-ci-cache-secrets"
patch = '''
containers:
- name: build
envFrom:
- secretRef:
name: nix-ci-cache-env
'''
```
... and a secret containing ...
```yaml
NIX_CI_RUNNER_CACHE_STRATEGY: attic
ATTIC_SERVER: <in-cluster-url> # example: http://atticd.<ns>.svc.cluster.local:8080
ATTIC_CACHE: ci # name however you want, just needs to exist
ATTIC_TOKEN: <token>
```
... makes your jobs automatically cache their Nix store paths to the in-cluster
attic when running with this runner.
Other runners could use cachix or no cache, you get the idea ;P
!!! note
This of course works with any executor where you can set environment
variables. This is just an example how to do it in Kubernetes easily.

31
docs/multi_pipeline.md Normal file
View file

@ -0,0 +1,31 @@
# Multiple Pipelines
With V2, Nix GitLab CI can generate different pipelines, depending on the
pipeline source (`$CI_PIPELINE_SOURCE`).
By default, no matter which source, the `default` pipeline is built and ran.
`$NIX_CI_PIPELINE` can override that, eg. when manually triggering a run.
To configure which source should be 1-to-1 translated to a pipeline with the
same name, set `$NIX_CI_DEFAULT_SOURCES` to a regex which explicitly does not
match these sources. Or set it to an impossible to match regex, then it will
always run the pipeline named after `$CI_PIPELINE_SOURCE`.
## Example 1: always run default
If you only have a single pipeline, you just have to call it `default`.
Everything else works out of the box.
## Example 2: default and merge_request_event
If you want the source `merge_request_event` to trigger a different pipeline,
name it like that and set `$NIX_CI_DEFAULT_SOURCES` to `^(merge_request_event)$`.
Now a merge request will run this pipeline, while everything else runs `default`.
## Example 3: default, push and web
Set `$NIX_CI_DEFAULT_SOURCES` to `^(push|web)$`.
## Example 4: always run the specific pipelines, never default
Set `$NIX_CI_DEFAULT_SOURCES` to any regex that never matches the sources,
like `a\A` or `nothing`.

81
docs/setup.md Normal file
View file

@ -0,0 +1,81 @@
# Setup
To integrate Nix GitLab CI into your project, you need to make two main changes:
1. Add the `nix-gitlab-ci` flake module to your `flake.nix`.
1. Include the necessary component in your `.gitlab-ci.yml`.
## Adding to `flake.nix`
In your project's `flake.nix`, add `nix-gitlab-ci` as an input and import its
flake module within your `flake-parts` configuration.
```nix title="flake.nix"
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; # Or your preferred nixpkgs branch/version
flake-parts.url = "github:hercules-ci/flake-parts";
# Add nix-gitlab-ci as an input
# recommendation: pin to a specific release/version
nix-gitlab-ci.url = "gitlab:TECHNOFAB/nix-gitlab-ci/<version>?dir=lib";
};
outputs = { nixpkgs, flake-parts, ... }@inputs:
flake-parts.lib.mkFlake { inherit inputs; } {
imports = [
# Import the nix-gitlab-ci flake module
inputs.nix-gitlab-ci.flakeModule
];
systems = [
"x86_64-linux"
"aarch64-linux"
# Add other systems you need
];
perSystem = { pkgs, ... }: {
# define your CI pipelines
# ci.pipelines."merge_request_event" = { ... };
};
};
}
```
Replace `<version>` with the specific version or commit hash of `nix-gitlab-ci`
you wish to use. Pinning to a specific version is highly recommended for
reproducibility and compatibility.
!!! warning
While the flake input is locked through `flake.lock`, the CI/CD component
will always use the latest commit of the reference. This means that by using
a branch like `main` as version for both, the CI/CD component will always use
the latest commit while your flake uses a fixed one.
This could result in drift between both, potentially breaking stuff.
## Including in `.gitlab-ci.yml`
Your `.gitlab-ci.yml` file will be minimal. Its primary role is to include the
`nix-gitlab-ci` component, which will then generate the full CI configuration
based on your Nix code.
```yaml title=".gitlab-ci.yml"
include:
- component: gitlab.com/TECHNOFAB/nix-gitlab-ci/nix-gitlab-ci@<version>
inputs:
# This input sets the Docker image tag used for the CI jobs.
# Use the same version as you pinned in your flake.nix for consistency.
version: <version>
```
Again, ensure `<version>` matches the version used in your `flake.nix`.
This component includes a job (`build:nix-ci`) that will evaluate your Nix
configuration and generate the `.gitlab-ci.yml` used for the pipeline run.
!!! note
Since V3 [Soonix](https://soonix.projects.tf) is supported, this can
automatically generate the `.gitlab-ci.yml` for you, with the version
automatically following the flake.
See [Soonix Integration](./soonix.md) for more.

20
docs/soonix.md Normal file
View file

@ -0,0 +1,20 @@
# Soonix Integration
[Soonix](https://soonix.projects.tf) can be used to automatically generate the
`.gitlab-ci.yml` for you.
This will by default include the CI/CD component with the same version as the
flake (using the `VERSION` file in this repo).
You can specify some options to configure this, like changing the component URL
or adding extra data to your `.gitlab-ci.yml` (like this repo does to bootstrap
Nix-GitLab-CI). See [Options](./options.md#configsoonix) for all the config options.
You can use it like this:
```nix
let
ci = cilib.mkCI { ... };
in {
soonix.hooks."ci" = ci.soonix;
}
```

73
docs/usage.md Normal file
View file

@ -0,0 +1,73 @@
# Usage
## Usage (with flake-parts)
```nix
# flake.nix
{
...
inputs.nix-gitlab-ci.url = "gitlab:TECHNOFAB/nix-gitlab-ci/<version>?dir=lib"; # recommendation: pin to the latest release/version
outputs = {...}: flake-parts.lib.mkFlake {...} {
imports = [
inputs.nix-gitlab-ci.flakeModule
];
...
perSystem = {pkgs, ...}: {
ci = {
config = {
# configure Nix-GitLab-CI here, see docs for options
};
pipelines."default" = {
stages = ["test"];
jobs = {
"test" = {
stage = "test";
nix.deps = [pkgs.unixtools.ping];
script = [
"ping -c 5 8.8.8.8"
];
};
};
};
# runs on a merge request for example
pipelines."merge_request_event" = {
stages = ["some_stage"];
jobs = { ... };
};
};
...
}
}
}
```
Now either use this in your .gitlab-ci.yml or setup Soonix to auto generate this
file for you with the right version (see the [docs][docs-soonix] for more).
```yaml
# .gitlab-ci.yml
include:
- component: gitlab.com/TECHNOFAB/nix-gitlab-ci/nix-gitlab-ci@<version> # recommendation: pin to the latest release/version (don't use "main" etc.)
inputs:
version: <version> # docker image tag, use the same version as a above
```
## Usage (directly)
```nix
let
cilib = inputs.nix-gitlab-ci.lib {inherit pkgs;};
in
cilib.mkCI {
config = ...;
pipelines."default" = ...;
};
# exposes `soonix` for the soonix hook and `packages` which contain the configs, jobs etc.
```
______________________________________________________________________
Since V2 multiple pipelines are supported.
See [Multiple Pipelines](./multi_pipeline.md) for more.

64
docs/utilities.md Normal file
View file

@ -0,0 +1,64 @@
# Utilities
Nix GitLab CI provides a couple of utilities to help with development and
debugging.
## Disabling caching temporarily
Nix GitLab CI often utilizes caching mechanisms to speed up your pipelines
(see [Caching](./caching.md)).
However, there might be situations where you need to temporarily disable these
caches for a specific pipeline run, for example, to debug a caching issue or
ensure a clean build.
To disable most of the provided caches for a pipeline, set the environment
variable `NIX_CI_DISABLE_CACHE` to any non-empty value (e.g., `yes`, `true`, `1`)
when triggering the pipeline in the GitLab UI or via the API.
## Forcing a rebuild of the CI pipeline definition
The job responsible for generating the `.gitlab-ci.yml` from your Nix code
(`build:nix-ci`) might itself be cached. If you've made changes to your Nix CI
configuration and the pipeline doesn't seem to pick them up, the cached job
definition might be the reason.
You should first double check if all the Nix files you defined the CI config in
are specified in the `cache_files` CI/CD-component input
(see [CI/CD Component](./cicd_component.md) for more).
To force this specific job to rebuild and re-evaluate your Nix configuration,
set the environment variable `NIX_CI_FORCE_BUILD` when triggering the pipeline.
## Running jobs locally
One of the benefits of defining your CI jobs with Nix is the ability to run them
locally in an environment that closely mirrors the CI environment. This can
significantly speed up debugging and development.
You can run the script of any defined job locally using the `nix run` command.
The syntax is:
```sh
nix run .#gitlab-ci:pipeline:<pipeline name>:job:<job name>
```
Replace `<pipeline name>` with the name of the pipeline the job belongs to
(e.g., `default` for jobs defined under the `ci` attribute) and `<job name>`
with the name of the job you want to run.
This command will set up the environment with the specified `nix.deps` and
execute the job's `script`.
There is also an attribute `.#gitlab-ci:pipeline:<pipeline name>:job-deps:<job name>`.
Building this derivation will generate a shell script which exports the required
environment variables for the job, such as the `PATH` including all dependencies
and any custom environment variables that contain store paths (ensuring they are
correctly resolved across different architectures).
You can use this to inspect the environment that would be set up for a job without
running the full script.
## Viewing generated config in YAML
Since the GitLab CI config is generated simply using JSON, it's hard to read and
debug. For debugging V3 now adds another package `gitlab-ci:pipeline:<pipeline name>:pretty`.

10
nix/repo/docs.nix
View file

@ -47,6 +47,16 @@ in
}; };
nav = [ nav = [
{"Introduction" = "index.md";} {"Introduction" = "index.md";}
{"Setup" = "setup.md";}
{"Usage" = "usage.md";}
{"CI/CD Component" = "cicd_component.md";}
{"Environment Variables" = "environment_variables.md";}
{"Caching" = "caching.md";}
{"Multiple Pipelines" = "multi_pipeline.md";}
{"Soonix Integration" = "soonix.md";}
{"Utilities" = "utilities.md";}
{"Kubernetes Runner Example" = "kubernetes_runner.md";}
{"Example Configs" = "examples.md";}
{"Options" = "options.md";} {"Options" = "options.md";}
]; ];
markdown_extensions = [ markdown_extensions = [