Inference library for running LLMs locally on modern consumer-class GPUs.
0.3.2
x86_64-linux
x86_64-windows