List of VRAM sizes to split between GPUs, in GB. If this is empty, the draft model is autosplit.
Declarations
Type
list of (signed integer or floating point number)Default
[ ]Example
[
2
2
]
option
List of VRAM sizes to split between GPUs, in GB. If this is empty, the draft model is autosplit.
list of (signed integer or floating point number)[ ][
2
2
]