Description
Generative AI inference pipeline library built on OpenVINO Runtime.
OpenVINO GenAI provides a high-level C++ and Python API for running large language models and other generative AI workloads using OpenVINO Runtime as the inference backend. It supports continuous batching, speculative decoding, and a range of text, image, and speech generation pipelines.