
This blog is written by AI for SEO
Build an Ontology With a Local LLM: Ollama and LM Studio
Yes, you can run an ontology on a local LLM. An ontology in an LLM context is a structured model of your business (the types of things you have, how they link, and the rules between them) that a model drafts and agents then work against. Helix Foundry builds one over your databases, tools and documents, and it can draft with a hosted key, Ollama or LM Studio on your own machine.
The model is a choice, not a given. If your data has to stay in-house, or you want to bring your own inference, an ontology local LLM setup is a real option. This guide covers what the drafting step asks of a model, how to pick where it runs, how to point Foundry at Ollama or LM Studio, and where smaller models draft less well. Foundry traces every value to a source row or document quote, so you can check a weak draft instead of trusting it.
What an ontology needs from an LLM (and what it doesn't)
An ontology needs a model that can read your tables and documents and propose types, links and mappings between them. That is the whole job. In Foundry, a generation sends your synced tables and sample documents to the ontology agent, which drafts a version. You review the draft. The agent does not get the final say.
The model does three things. It reads structure (column names, keys, sample values). It reads prose (contracts, notes, PDFs) and pulls out facts. And it proposes how entities in different sources are the same thing. A customer in Stripe, a contact in your CRM and a name in a contract are one entity, and the model drafts that link.
The model does not need to be the source of truth. Every value in the ontology traces back to a source row or a quote from a document. The model proposes, the evidence decides. That split makes a smaller model a workable choice rather than a reckless one, because a wrong draft is visible against its source.
It also does not need your data to leave the building. Foundry never writes to your sources, and which model reads your tables is a setting you control.
Local, hosted or your own server: choosing where the model runs
Choose by where your data may go and how much draft quality you need. Foundry supports several providers: Claude, OpenAI, Azure OpenAI, Amazon Bedrock, Cohere, Ollama, LM Studio, or any OpenAI-compatible server.
Here is how the options line up.
| Option | Where inference runs | Pick it when |
|---|---|---|
| Hosted key (Claude or OpenAI) | The provider's servers | You want the strongest drafts and the data can leave |
| Azure OpenAI or Amazon Bedrock | Your cloud account | Your org already approves one of those clouds |
| Ollama or LM Studio | Your own machine | Data must stay in-house, or you want to bring your own model |
| Any OpenAI-compatible server | A server you run | You already host inference and want Foundry to use it |
Locally, Foundry uses an ANTHROPIC_API_KEY or OPENAI_API_KEY from the shell that ran docker compose up if one is set, and otherwise falls back to a local model in Ollama or LM Studio.
The rule is simple. If nothing forces your data to stay put, a hosted frontier model will draft best. If something does, run local and plan to review more. For the infrastructure side of running Foundry itself, see Self-Hosted Ontology Platform: Run Foundry in Your Cloud.
Set up Helix Foundry on your computer
Foundry runs locally with Docker. You need Docker with Compose v2, ports 3000 and 8333 free, and room for a few GB of images on the first start.
From a terminal:
git clone https://github.com/helixdb/helix-foundry.git
cd helix-foundry
docker compose up -d --waitThen open http://localhost:3000.
The local install is for one person on their own computer. It is single-user on localhost with no sign-in, so do not expose it to a network. A shared deployment is a different setup from this one.
Until a provider is set, drafting, uploading documents and asking questions wait. Nothing breaks. Foundry is holding for a model. That is the cue for the next step.
If you would rather skip local setup, you can use Helix Cloud at foundry.helix-db.com. This guide stays on the local route because that is where the model choice matters most.
Point Foundry at Ollama or LM Studio
Open Settings, then AI models, to see or change the provider. That screen is where you pick Ollama or LM Studio, and you can choose models per task: drafting, asking questions, document extraction and verification. Locally, Foundry can also detect a running Ollama or LM Studio model on its own.
Ollama, bundled. To run Ollama beside Foundry, start the stack with:
docker compose --profile ollama up -dPull a model first if you do not have one, for example ollama pull qwen3.5:4b. Bundled Ollama is reachable by Foundry through its shared network namespace, but it is not reachable by other containers or from outside Docker.
Ollama, on your machine. If you already run an Ollama server on your computer, point Foundry at that instead of starting the bundled one.
LM Studio. Load and serve a model in LM Studio. Foundry runs in Docker, so it has to reach LM Studio on the host, at http://host.docker.internal:1234/v1. In LM Studio, turn on Serve on Local Network so the container can connect.
The small model in that example is a starting point, not a recommendation. Pick one your hardware runs comfortably, run a first generation on a small slice of data, and read the draft before connecting more sources.
What smaller local models do worse, and how to check their drafts
Smaller local models generally draft less well than frontier models, and you should plan for that rather than discover it late. The weak spots are the hard ones: proposing the right links between entities across sources, pulling clean facts out of long or messy documents, and keeping a consistent naming scheme across a large schema. Expect more missed links, odd type names and thinner extraction from prose.
That is a reason to review, not a reason to avoid local models. The evidence trail stays safe. Every value in the ontology is traced to the source row or document quote it came from, so a bad draft is checkable.
A practical review loop:
Start small. Generate over one or two tables and a few documents before widening.
Open the types and links the draft proposes and ask: would someone in the business recognise these names?
For links, spot-check pairs. Take a customer in one source and confirm the draft matched it to the right record in another.
For anything pulled from a document, follow it back to the quote. If the quote does not say it, the fact is wrong.
Fix or reject, then regenerate. Foundry keeps the schema and graph in step as your data changes.
Since you can set models per task, a useful split is a local model for the first pass over sensitive data and a hosted one only where the data allows it. And Foundry never writes to your sources, so a poor draft costs you review time, not data.
Helix Cloud and Foundry Pro: bringing your own model there
That is the route when you want your own provider relationship but do not want to run Foundry yourself.
The split is simple. Local Foundry gives you the widest choice: a hosted key from the shell, Ollama, LM Studio or any OpenAI-compatible server, all in a single-user install on your own machine. Helix Cloud trades that range for a managed service and shared use.
If your requirement is strict, such as inference that never leaves hardware you control, the local route with Ollama, LM Studio or your own OpenAI-compatible server meets it. If your requirement is a named provider your company already contracts, bringing your own model or running a local setup covers it.
Pick the model for your data, then review the draft
The model that drafts your ontology should follow your data rules, not the other way round. Hosted drafts best and local keeps data home. Because every value traces to a source row or quote, a weaker local draft is something you check rather than something you have to trust. Clone the repo, run docker compose up -d --wait, open http://localhost:3000, set a provider under Settings, AI models, and generate your first ontology over one table. Star helix-foundry on GitHub if it earns it.