OMLX

Overview

OMLX is an LLM inference server designed for Apple Silicon, featuring continuous batching and SSD caching. It is ideal for developers looking to manage LLMs efficiently from the macOS menu bar.