MyNixOS website logo
option

services.tabby.model

Specify the model that tabby will use to generate completions.

This model will be downloaded automatically if it is not already present.

If you want to utilize an existing model that you've already downloaded you'll need to move it into tabby's state directory which lives in /var/lib/tabby. Because the tabby.service is configured to use a DyanmicUser the service will need to have been started at least once before you can move the locally existing model into /var/lib/tabby. You can set the model to 'none' and tabby will startup and fail to download a model, but will have created the /var/lib/tabby directory. You can then copy over the model manually into /var/lib/tabby, update the model option to the name you just downloaded and copied over then nixos-rebuild switch to start using it.

$ tabby download --model TabbyML/DeepseekCoder-6.7B $ find ~/.tabby/ | tail -n1 /home/ghthor/.tabby/models/TabbyML/DeepseekCoder-6.7B/ggml/q8_0.v2.gguf $ sudo rsync -r ~/.tabby/models/ /var/lib/tabby/models/ $ sudo chown -R tabby:tabby /var/lib/tabby/models/

See for Model Options:

https://github.com/TabbyML/registry-tabby.

Declarations
Type
string
Default
"TabbyML/StarCoder-1B"