Inference.net

Inference.net is an LLM inference platform with a gateway, agent tracing, fine-tuning and evals for AI teams on open models, billed by token usage.

Inference.net runs open-source, custom and fine-tuned language models in production and records how every model call behaves. It is for teams that want to move work off closed frontier models without losing sight of latency, cost and quality. Inference R&D, Inc builds it and does business under the same name as an independent company. Serving, tracing, training and evaluation all live in one account behind one signup. Deploy serves catalog models or a team's own on serverless or dedicated infrastructure, in public cloud, private cloud or hybrid setups. Its API covers vision input, function calling, JSON schema outputs, a reasoning_effort setting, and Async and Batch APIs. Observe routes traffic from any provider through the LLM gateway and shows latency, cost and quality per request on one dashboard. Trace uses Catalyst Tracing to capture LLM calls, tool calls and framework steps, and HALO, an open-source agent, reads those traces to say what to fix. Train turns production data into specialized models, while Evaluate and AutoEvals score candidates on real traffic. The platform suits AI-native product teams and the engineers who keep their LLM features fast and affordable. Teams leaving OpenAI, Anthropic or Gemini for open or fine-tuned models gain the most, because their traces feed training and evals directly. A startup can begin on a usage-only account that bills per token, then move to a flat monthly plan for more seats and longer retention. Larger companies sign committed-use contracts that add dedicated infrastructure and custom-trained models. Typical work starts with pointing an existing app at the gateway, so every request is logged before any model changes. Applications built on the OpenAI SDK, the Anthropic SDK or the Vercel AI SDK reach it through those libraries. Agent builders connect HALO to their coding agent and let it find issues in their agents' runs, either locally or on hosted HALO. Over time the logged traffic becomes the training set for a smaller model, which Evaluate checks against the old one before the switch. It fits a team that wants its own production data to decide which model serves each task.