Ollama
hosomaki.ollama
Local runtime that downloads, quantises and runs open-weight language models, exposing them over its own HTTP API and an OpenAI-compatible endpoint. Also serves embedding models.
A Docker Compose file for Ollama alone, generated from the catalog and checked by the safety inspector. Sign in and use Kitchen Credits to activate it.
Uses the Medusa cart and payment-session path. This test control does not create an order or capture payment.
Ollama manages a local library of model weights, pulling them from its registry in quantised form and loading them into memory on demand when a request arrives. A Modelfile fixes the base weights, system prompt and sampling parameters for a named variant, which makes a customised model reproducible across machines. The API covers generation, chat and embeddings, and a compatibility layer accepts OpenAI-format requests so existing clients need only a changed base URL. A first deployment pulls one small model and issues a completion; GPU acceleration is used automatically where host drivers are present.
You know it worked when
- The service answers a version request over HTTP.
- A pulled model appears in the local model list.
- A generation request returns tokens from that model.
- An OpenAI-format request to the compatibility endpoint returns the response shape a client library expects.
- An embedding request returns a vector of the model's stated dimensionality.
Known sharp edges
- The first pull of a model transfers several gigabytes and the API returns nothing useful until it finishes.
- Models load on first request and are evicted after an idle timeout, so the first call after a pause is markedly slower than those following it.
- GPU acceleration needs matching host drivers and container runtime configuration; without them the service falls back to CPU silently.
- The API has no authentication and must not be reachable beyond the deployment network without a proxy in front of it.