services.llama-cpp.modelsPreset

Models preset configuration as a Nix attribute set. This is converted to an INI file and passed to llama-server via --model-preset. See llama-server documentation for available options.

Declarations

nixos/modules/services/misc/llama-cpp.nix

Type

null or (attribute set of (attribute set))

Default

null

Example

{
  "Qwen3-Coder-Next" = {
    hf-repo = "unsloth/Qwen3-Coder-Next-GGUF";
    hf-file = "Qwen3-Coder-Next-UD-Q4_K_XL.gguf";
    alias = "unsloth/Qwen3-Coder-Next";
    fit = "on";
    seed = "3407";
    temp = "1.0";
    top-p = "0.95";
    min-p = "0.01";
    top-k = "40";
    jinja = "on";
  };
}