AI Agents
Models that run on hardware you own
Open-weight models deployed inside your boundary. No inference call leaves your infrastructure, removing a category of vendor and residency questions.
The hardest questions in an AI assessment are usually not about the model. They are about where the data went: which vendor processed it, under what terms, in which jurisdiction, with what retention, and whether a subprocessor was involved.
Running open-weight models on your own hardware removes that entire category of question. There is no inference vendor, because there is no inference call leaving your boundary.
We deploy the Hermes family from Nous Research among others. Hermes 4, released in August 2025, supports function and tool calling interleaved with its reasoning inside a single assistant turn, and is trained for schema adherence and structured JSON output — which is what makes it usable as an agent rather than only as a chat model.
A licensing caution: the Hermes 4 70B and 405B models are released under the Llama 3 licence, not Apache or MIT. They are not permissively licensed, and the licence carries conditions worth reading before you build a product on them. Anyone who tells you the family is fully permissive has not checked.
Why self-host
What it changes
Self-hosting is not always the right answer. When it is, these are the reasons.
Data residency becomes trivially answerable
The honest answer to \"where is our data processed?\" becomes a rack you can point at. That single sentence closes more assessment findings than any contractual assurance.
No subprocessor chain
Every commercial inference vendor brings a subprocessor list that you inherit and must diligence. Self-hosting removes the list.
Cost stops scaling with usage
Per-token pricing punishes success: the more useful the agent, the larger the bill. Owned hardware converts that into capital cost plus power, and the marginal query approaches free.
Model stability on your terms
A hosted model can be deprecated, re-tuned, or silently changed underneath you, invalidating your evaluations. A model you host changes when you decide it does.
Tool calling that holds its shape
Agents need structured output that parses reliably. Hermes 4 is trained for schema adherence and can interleave tool calls with reasoning in one turn, which materially reduces the glue code and the failure modes.
Honest limits
Self-hosted open-weight models still trail the best frontier models on the hardest reasoning. For many regulated workflows that gap does not matter. Where it does, we will say so and route that work elsewhere.
Questions
Frequently asked
Are open-weight models good enough for real work?
For most regulated workflows, yes — document retrieval and summarization, structured extraction, classification, routing, drafting against a template. Where frontier models still lead is open-ended multi-step reasoning on novel problems. The right architecture often runs both, with a routing policy that keeps regulated data local and sends only non-sensitive work outward.
What hardware do we need?
It depends entirely on model size and concurrency, and we would rather size it against your actual workload than quote a number here. The useful framing is that this is a capital decision with a payback period, and we will model that with you in the Savings Workshop before you buy anything.
What licence do these models carry?
It varies by model and it is worth checking individually. Hermes 4 70B and 405B are under the Llama 3 licence — not Apache, not MIT, and with conditions attached. Other open-weight families differ. We confirm the licence for every model we deploy and hand you the terms, because inheriting a licensing problem into a product is an expensive way to discover this.
Does self-hosting satisfy the EU AI Act?
It helps with some obligations and is irrelevant to others. It makes data-governance and confidentiality questions much easier. It does not change your classification: if your system is high-risk under Annex III, self-hosting does not make it less so. Note that the high-risk obligations were deferred to 2 December 2027 for Annex III and 2 August 2028 for Annex I by the Digital Omnibus.
Interested in this agent?
Let's scope it against your compliance requirements and agree what it returns.