SovereignGrid

Self-hosted inference, no per-token bill

Your team already owns the GPUs. Start using all of them.

SovereignGrid pools idle GPUs across your team into a single inference API. Anyone can send a request, the grid routes it to whoever has capacity, and every completion is credited back to the machine that served it. No cloud contract, no per-token invoice, no data leaving machines you control.

Talk to us about your setup See how it works

Runs with no database for the first ten minutes. Self-hosted on hardware you already own.

Zeroper-token API fees

Minutesto add a new machine

Automaticfailover across nodes

Per-nodeusage and credit accounting

The problem

Inference is either expensive or idle.

Most teams end up paying twice: once for API tokens, and again for workstation GPUs that sit unused most of the day.

The bill scales with success

Background agents, batch jobs, and evaluation runs are exactly the workloads that burn the most tokens and matter the least per call. Usage-based pricing punishes the work you want more of.

Good hardware sits idle

A team of ten developers has a lot of GPU sitting at a desktop, doing nothing overnight and nothing at lunch. There is no simple way to make that capacity available to everyone.

Data leaves your control

Sending prompts to a third-party endpoint means your code, your documents, and your customers' details are processed on infrastructure you cannot inspect.

How it works

Three moving parts, one endpoint.

Providers install a daemon. Consumers call a standard endpoint. The grid handles everything in between.

Install the daemon

Each contributor runs a small daemon that binds to their local Ollama, vLLM, or Aphrodite server. It reports which models it serves and how many jobs it can handle at once.

Join the guild

The daemon authenticates with a wallet signature, and an administrator admits it to the guild. No passwords are shared, and the private key never leaves the contributor's machine.

Send requests

Applications call one OpenAI-shaped endpoint. The grid picks the least-loaded machine serving the requested model, runs the job, and returns the result.

What changes for you

What your team notices in the first week.

The change is less about raw speed and more about what stops being a decision you have to make.

Situation	How it works today	With SovereignGrid
Running a large batch job	Estimate the token cost first, then decide whether the experiment is worth running at all.	Queue it against the pool and let it run on hardware you already paid for.
A contributor closes their laptop	In-flight work fails and someone has to notice and retry it by hand.	Jobs are requeued automatically and picked up by another machine within seconds.
Adding a new machine	A new endpoint, a new URL, and every application updated to know about it.	Install the daemon, get admitted to the guild, and it starts taking work. No client changes.
Working with sensitive material	A review of the provider's terms and a decision about what may leave the building.	Prompts and outputs stay on machines your organisation owns and can audit.
Accounting for who used what	A single invoice with no breakdown by person, project, or machine.	Every completion recorded with the model, the machine, and the credits earned.

In practice

A normal day on a shared grid.

Nothing here needs a person to intervene.

09:00

Six developers start work. Their machines have been serving the overnight evaluation queue since midnight and quietly reduce their own concurrency as local work picks up.

13:20

Someone kicks off a document processing run across four thousand files. It spreads across whichever machines have free slots rather than queueing behind one busy box.

16:45

A contributor reboots without warning. Their three in-flight jobs are requeued and finished elsewhere. The person who submitted them never finds out.

18:00

The hourly settlement runs. Each machine's contribution for the day is totalled and credited, priced by the size of the models it actually served.

What you get

Built for a pool of machines that come and go.

Laptops close. Networks drop. People reboot. The grid is designed for that being normal rather than exceptional.

Nothing gets lost

If a machine disconnects mid-job, times out, or silently vanishes, the work is put back in the queue and picked up elsewhere. Consumers see a slower response, not a failure.

Load-aware routing

Every request goes to the machine with the fewest jobs running that can serve the requested model, so one contributor never becomes the bottleneck while others idle.

Fair usage accounting

Each completed job is recorded and priced by model size, so a contributor running a 70B model earns proportionally more than one running an 8B. Balances settle on a schedule, not on the hot path.

Contributors keep their keys

Authentication is a signature challenge. Only a public address and a signature ever cross the network, and each challenge can only be used once.

Payload screening

Requests are checked against a denylist of shell injection, reverse shell, and file access patterns before they are handed to a contributor's machine.

Async or synchronous

Submit and wait for chat completions, or queue long-running jobs and collect the results later. Both use the same routing and the same failover.

Who it is for

Teams with hardware and a growing inference bill.

SovereignGrid is most useful where the workload is large, background, and not latency critical.

Engineering teams

Give background agents, batch jobs, and evaluation suites somewhere to run that does not appear on a usage invoice.

Agent and automation workloads
Nightly evaluation runs
Bulk document processing

Research groups

Share a small cluster across a lab without building a scheduler, a queue, and a billing system from scratch.

Shared model access
Per-member usage records
Mixed hardware, one endpoint

Privacy-bound work

Keep prompts and outputs on machines your organisation owns and can audit, with no third-party processor in the path.

On-premise only
No external processor
Full request visibility

Under the hood

The details your engineers will ask about.

Everything below is configurable, and documented in full for your engineers.

Engines supportedOllama, vLLM, and Aphrodite through their native or OpenAI-compatible endpoints

API shapePOST /v1/chat/completions plus an async job queue at /v1/jobs

Concurrency1 to 64 simultaneous jobs per machine, set by the contributor

RetriesThree attempts across different machines before a job is marked dead

Recovery sweepStalled jobs are detected and requeued every 15 seconds

StorageRuns in memory for evaluation, or on PostgreSQL for production, switched by one variable

AuthenticationEIP-191 wallet signatures with single-use 60 second challenges

DeploymentPython and FastAPI, self-hosted anywhere you can run a container

Questions

Before you install it.

Do contributors need to trust each other?

Yes, and the design says so plainly. Guild membership is the security boundary: an administrator decides which machines may join. Payload screening catches obvious abuse, but it is a safety net rather than a sandbox. Run the grid inside a team or organisation that already has a trust relationship.

What happens if my machine goes offline mid-job?

The job is requeued and another machine picks it up. This is handled three ways at once, covering a clean disconnect, a silent timeout, and a machine that vanishes without notice, so no single missed signal strands the work.

Can I try it without setting up a database?

Yes. Leave the database variable unset and the whole grid runs in memory. That is enough to install the daemon, connect a local model, and send a real request. Point it at PostgreSQL when you want history and credit balances to persist.

Does it stream responses?

Not yet. Requests return complete results rather than token by token, which suits background and batch workloads. Streaming is the most requested addition and is not in the current build.

How is usage measured and priced?

Every completed job appends a record with the model used and tokens generated, priced by a rate table you control. Balances are settled on an hourly batch so accounting never slows down inference.

What stage is the project at?

Working end to end and in use as an MVP. Known limits are documented rather than hidden: no streaming, token counts are trusted as reported by the contributor, consumer authentication is a bearer key rather than metered billing, and the coordinator currently runs as a single instance.