External APIExternal APIExternal APIPrompts leaveYour networkprompts + data
Cloud APIDEMO TELEMETRY
Metered spend~$10,000per month, as reported
100%
Egress> 0prompts leave your network
Last response429RATE LIMITED
External endpoints1+reached per prompt
Cloud APIor your own GPUs.
Your networkClientEndpointGPU
Your own GPUsDEMO TELEMETRY
Run cost~$2,000per month, as reported
20%
Egress0 Bnothing leaves
Last response200OK
External endpoints0reached
A hosted API meters every prompt and sends it to the provider's servers. Open-weight models on your own GPUs answer inside your network. Drag the divider to see both. Cost figures are as reported by customers; the telemetry is a demo.
Same prompts. A different bill, a different perimeter.
Rows marked reported are customer figures from installations we run. They are not a benchmark, and your workload will differ.
| Measure | Cloud API | Your own GPUs with LLM Deploy |
|---|---|---|
| Data flow | Prompts and outputs travel to the provider’s servers. | Prompts and outputs stay inside your network. |
| Cost model | Metered per token, so the bill follows usage. | GPUs and operations. Hardware is sized to your load. We size the hardware and source it at cost, with no markup. Managed ops from $1,500 per GPU node per month. |
| Monthly billNocodo LTD, reported | ~$10,000 | ~$2,000 |
| 240 hackathon submissions scoredInfiano, reported | ≈$6,750 | ≈$3,000 |
| Limits | Rate limits and quotas are set by the provider. | Throughput is what the GPUs you size can serve. |
| LatencyNocodo LTD, reported | 1× 1× | 3× faster 3× 1.1–1.4 s to first token on the 40-GPU Infiano cluster, 0 restarts. |
| Compliance postureNocodo LTD, reported | Baseline 100% | Down ~60% 40% Overhead reported by the customer. Not a certification. |
| Fine-tuning | Only what the provider offers, on its terms. | Open weights can be adapted on your own data without it leaving. Licences differ by model. |
| Air-gap | Needs a route out to the provider’s endpoint. | The endpoint sits inside your network, so inference needs no route out. |
| Transparency | The provider chooses, and can change, the model behind the endpoint. | Open weights you can inspect. You choose the model and when to update. Requests are audit logged. |
A hosted API is the better call when volume is low or spiky, you have no GPUs to run, and nothing in the prompts is sensitive. Owning the stack pays off as volume and data sensitivity grow.
Put your own volume into the cost calculatorChange the base URL. Nothing else.
Same SDK. Change the base URL. Tested-integration matrix on request.
client = OpenAI(
base_url="https://llm.yourcompany.internal/v1", # inside your network
api_key="your-key",
)Talk to an engineer.
Tell us the model, the data and the GPUs you have. An engineer from Limassol or Auckland scopes the 72-hour pilot with you.