Showing entries 1-10 out of 10.
Cache mode for draft models to save VRAM
List of VRAM sizes to split between GPUs, in GB
Drafting mode for exllamav3
Directory to look for draft models
An initial draft model to load
Number of tokens to draft per iteration (default: draft model default)
Rope alpha for draft models, same as alpha_value
Rope scale for draft models, same as compress_pos_emb
Adjust number of draft tokens dynamically based on observed acceptance rates
Minimum match length for exllamav3 n-gram drafting