🐧 PenguinPulse

Linux Graphics & Gaming News

Ollama v0.30.0-rc17 Shifts to llama.cpp, GGUF, Adds MLX for Apple Silicon

Ollama has recently released v0.30.0-rc17, a pre-release introducing a significant architectural overhaul. This version "will change the architecture to directly support llama.cpp instead of building on top of GGML, and allows for compatibility with GGUF file format." This move aims to streamline how Ollama interacts with large language models, leveraging the widely adopted llama.cpp project. Additionally, the update integrates MLX to accelerate model inference specifically on Apple Silicon hardware, enhancing performance for users on those platforms. The development team is soliciting feedback on performance, errors, and memory utilization improvements or degradations in this release candidate.

Sources