LLM_Deploy
Talk to an engineerMenu

Which model fits on your GPUs.

Open-weight models deployed inside your network, on GPUs you control, behind an OpenAI-compatible endpoint. Set the VRAM you have and the models that fit move to the top.

Start the $4,900 pilotTalk to an engineer

0GB60 open-weight models, sized for 13 GPUsEmpty bayempty bay · pick a GPU below

0×

0 of 60 models fit60 models in the catalogue

  • Chat00/28
  • Coding00/06
  • Vision00/08
  • OCR00/08
  • Embed00/08
  • Speech00/06

LoadoutPick the GPU you have. It slots into the bay above, and the table re-sorts.

CPU onlyCPU · tiny models
T416 GB · entry
RTX 4060 Ti 16GB16 GB · consumer
L424 GB · inference
RTX 409024 GB · consumer
A1024 GB · entry DC
RTX 509032 GB · consumer
A100 40GB40 GB · DC, previous gen
L40S / RTX 6000 Ada48 GB · pro
RTX PRO 6000 Blackwell96 GB · pro
H10080 GB · DC
H200141 GB · DC
B200180 GB · DC

VRAM per card from NVIDIA's product pages, datasheets and product briefs, read 2026-10-27. B200 is the per-GPU share of HGX / DGX B200 (1,440 GB for 8). CPU only covers models of about 10B parameters or fewer at 4-bit.

Models we deploy, by popularity

Category
Your GPUPick a GPU in the loadout or drag the amber handle

Drag the amber handle to your VRAM

15 of 60 models · by popularity

Models we deploy: model and vendor, category, total and active parameters, context, estimated VRAM at FP8 and 4-bit, licence, and the smallest GPU configuration it fits on. Ranked by Hugging Face downloads.
ModelSize / activeContextLicenceFits onest. VRAM FP8 / 4-bit
Gemma 4 31BGoogle · Chat + vision31B dense256KApache 2.01× RTX 4090 24GB4-bitFP8 · 1× A100 40GB37.5 GB FP8 · 4-bit 18.8
Qwen3-Coder-30B-A3BAlibaba · Coding30.5B MoE3.3B active256KApache 2.01× RTX 4090 24GB4-bitFP8 · 1× A100 40GB36.6 GB FP8 · 4-bit 18.3
Unlimited-OCRBaidu · OCR3.3B MoE~0.57B active32KMITCPU only4-bitor any GPU4 GB FP8 · 4-bit 2
all-MiniLM-L6-v2sentence-transformers · Embedding22.7M512 tokensApache 2.0CPU only4-bitor any GPU<0.1 GB FP8 · 4-bit <0.1
Whisper large-v3-turboOpenAI · Speech-to-text809M30 s windowsMITCPU only4-bitor any GPU1 GB FP8 · 4-bit 0.5
Qwen3Alibaba · Chat8B / 32B / 235B-A22B22B (235B) active32K (128K with YaRN)Apache 2.0CPU only4-bitFP8 · 1× T4 16GB9.8–282 GB FP8 · across sizes · 4-bit from 4.9
Qwen3-Coder-NextAlibaba · Coding80B MoE3B active256KApache 2.01× L40S 48GB4-bitFP8 · 1× RTX PRO 6000 96GB95.6 GB FP8 · 4-bit 47.8
Gemma 4 26B A4BGoogle · Chat + vision26B MoE~4B active256KApache 2.01× T4 16GB4-bitFP8 · 1× RTX 5090 32GB31 GB FP8 · 4-bit 15.5
DeepSeek-OCRDeepSeek · OCR3.3B MoE~0.57B active8KMITCPU only4-bitor any GPU4 GB FP8 · 4-bit 2
BGE-M3BAAI · Embedding568M8K tokensMITCPU only4-bitor any GPU0.7 GB FP8 · 4-bit 0.3
Whisper large-v3OpenAI · Speech-to-text1.55B30 s windowsApache 2.0CPU only4-bitor any GPU1.9 GB FP8 · 4-bit 0.9
Devstral Small 2Mistral AI · Coding24B dense256KApache 2.01× T4 16GB4-bitFP8 · 1× RTX 5090 32GB28.8 GB FP8 · 4-bit 14.4
Qwen3.8-27BAlibaba · Chat + vision27B dense256K (1M ext.)Apache 2.01× RTX 4090 24GB4-bitFP8 · 1× A100 40GB33.3 GB FP8 · 4-bit 16.7
GLM-OCRZ.ai · OCR0.9B128KMITCPU only4-bitor any GPU1.6 GB FP8 · 4-bit 0.8
bge-reranker-v2-m3BAAI · Reranker568M8K tokensApache 2.0CPU only4-bitor any GPU0.7 GB FP8 · 4-bit 0.3
Voxtral Mini 4B RealtimeMistral AI · Speech-to-text4.4BstreamingApache 2.0CPU only4-bitor any GPU5.3 GB FP8 · 4-bit 2.7
Kimi K2.7 Code on Hugging Face (opens in a new tab)Moonshot AI · Coding1T MoE32B active256KModified MITattribution8× H1004-bitFP8 · multi-node1,232 GB FP8 · 4-bit 616
GLM-5.3-Flash on Hugging Face (opens in a new tab)Z.ai · Chat + vision320B MoE18B active1MMIT4× H1004-bitFP8 · 8× H100386 GB FP8 · 4-bit 193
LightOnOCR-2-1B on Hugging Face (opens in a new tab)LightOn · OCR1B16KApache 2.0CPU only4-bitor any GPU1.2 GB FP8 · 4-bit 0.6
Qwen3-Embedding-0.6B on Hugging Face (opens in a new tab)Alibaba · Embedding0.6B32KApache 2.0CPU only4-bitor any GPU0.7 GB FP8 · 4-bit 0.4
Qwen3-ASR-1.7B on Hugging Face (opens in a new tab)Alibaba · Speech-to-text2.3Baudio, 64KApache 2.0CPU only4-bitor any GPU2.8 GB FP8 · 4-bit 1.4
Qwen3-Coder-480B-A35BAlibaba · Coding480B MoE35B active256KApache 2.04× H1004-bitFP8 · 8× H100576 GB FP8 · 4-bit 288
Qwen3.5-9B on Hugging Face (opens in a new tab)Alibaba · Vision9B dense256K (1M ext.)Apache 2.0CPU only4-bitFP8 · 1× T4 16GB11.6 GB FP8 · 4-bit 5.8
DeepSeek-OCR 2DeepSeek · OCR3.4B MoE~0.57B active8KApache 2.0CPU only4-bitor any GPU4.1 GB FP8 · 4-bit 2
Qwen3-Reranker-4B on Hugging Face (opens in a new tab)Alibaba · Reranker4B32KApache 2.0CPU only4-bitor any GPU4.8 GB FP8 · 4-bit 2.4
Nemotron 3.5 ASR Streaming 0.6B on Hugging Face (opens in a new tab)NVIDIA · Speech-to-text638MstreamingOpenMDW-1.1CPU only4-bitor any GPU0.8 GB FP8 · 4-bit 0.4
gpt-oss-20bOpenAI · Chat~21B MoE3.6B active128KApache 2.01× T4 16GB4-bitFP8 · 1× RTX 5090 32GB25.1 GB FP8 · 4-bit 12.5
North Mini Code 1.0 on Hugging Face (opens in a new tab)Cohere · Coding30B MoE3B active256KApache 2.01× RTX 4090 24GB4-bitFP8 · 1× A100 40GB36.6 GB FP8 · 4-bit 18.3
Qwen3-VL-8B-Instruct on Hugging Face (opens in a new tab)Alibaba · Vision8B dense256K (1M ext.)Apache 2.0CPU only4-bitFP8 · 1× T4 16GB10.5 GB FP8 · 4-bit 5.3
dots.ocr on Hugging Face (opens in a new tab)rednote · OCR3B128KMITCPU only4-bitor any GPU3.6 GB FP8 · 4-bit 1.8
Qwen3-Embedding-8B on Hugging Face (opens in a new tab)Alibaba · Embedding8B32KApache 2.0CPU only4-bitFP8 · 1× T4 16GB9.1 GB FP8 · 4-bit 4.5
Parakeet TDT 0.6B v3 on Hugging Face (opens in a new tab)NVIDIA · Speech-to-text627Mlong-formCC-BY-4.0attributionCPU only4-bitor any GPU0.8 GB FP8 · 4-bit 0.4
Qwen3.6-35B-A3B on Hugging Face (opens in a new tab)Alibaba · Chat35B MoE3B active256K (1M ext.)Apache 2.01× RTX 4090 24GB4-bitFP8 · 1× L40S 48GB43.1 GB FP8 · 4-bit 21.6
Qwen3-VL-235B-A22B-Instruct on Hugging Face (opens in a new tab)Alibaba · Vision235B MoE22B active256K (1M ext.)Apache 2.02× H1004-bitFP8 · 4× H100283 GB FP8 · 4-bit 141
olmOCR-2-7B-1025 on Hugging Face (opens in a new tab)Allen AI · OCR8.3B128KApache 2.0CPU only4-bitFP8 · 1× T4 16GB9.9 GB FP8 · 4-bit 5
EmbeddingGemma 2 on Hugging Face (opens in a new tab)Google · Embedding740M8KApache 2.0CPU only4-bitor any GPU0.9 GB FP8 · 4-bit 0.4
DeepSeek-V4-Flash on Hugging Face (opens in a new tab)DeepSeek · Chat284B MoE13B active1MMIT4× H1004-bitFP8 · 8× H100365 GB FP8 · 4-bit 183
MiniCPM-V 4.7 35B-A3B on Hugging Face (opens in a new tab)OpenBMB · Vision35B MoE3B active256KApache 2.01× RTX 4090 24GB4-bitFP8 · 1× L40S 48GB42.3 GB FP8 · 4-bit 21.1
PaddleOCR-VL-1.6 on Hugging Face (opens in a new tab)Baidu · OCR0.9B128KApache 2.0CPU only4-bitor any GPU1.2 GB FP8 · 4-bit 0.6
Nemotron 3.5 Rerank 8B on Hugging Face (opens in a new tab)NVIDIA · Reranker8.4B256KOpenMDW-1.1CPU only4-bitFP8 · 1× T4 16GB10.1 GB FP8 · 4-bit 5
Qwen3.8-Flash-NextAlibaba · Chat180B MoE6B active256K (1M ext.)Qwen Community License 1.0revenue cap1× H200 141GB4-bitFP8 · 4× H100216 GB FP8 · 4-bit 108
gpt-oss-120bOpenAI · Chat~117B MoE5.1B active128KApache 2.01× H100 80GB4-bitFP8 · 1× H200 141GB140 GB FP8 · 4-bit 70.1
GLM-5.3 on Hugging Face (opens in a new tab)Z.ai · Chat753B MoE40B active1MGLM-5.3 License (MIT-style)conditions8× H1004-bitFP8 · 8× H200904 GB FP8 · 4-bit 452
DeepSeek-V4.1-Flash on Hugging Face (opens in a new tab)DeepSeek · Chat763B MoE8B / 16B active1MMIT8× H1004-bitFP8 · 8× H200916 GB FP8 · 4-bit 458
DeepSeek V3 / R1DeepSeek · Chat671B MoE37B active128KMIT (R1); DeepSeek Model Agreement (V3)use policy (V3)8× H1004-bitFP8 · 8× H200821 GB FP8 · 4-bit 411
Nemotron 3 Super 120B-A12B on Hugging Face (opens in a new tab)NVIDIA · Chat120B hybrid MoE12B active256K (up to 1M)NVIDIA Nemotron Open Model License1× H100 80GB4-bitFP8 · 2× H100148 GB FP8 · 4-bit 74.2
Hy3 on Hugging Face (opens in a new tab)Tencent · Chat295B MoE21B active256KApache 2.01× B200 180GB4-bitFP8 · 8× H100359 GB FP8 · 4-bit 179
Kimi K3 on Hugging Face (opens in a new tab)Moonshot AI · Chat2.8T MoE104B active1MKimi K3 Licenserevenue capMulti-nodebeyond 8× H2003,336 GB FP8 · 4-bit 1,668
DeepSeek-V4-Pro on Hugging Face (opens in a new tab)DeepSeek · Chat1.6T MoE49B active1MMIT8× H2004-bitFP8 · multi-node1,919 GB FP8 · 4-bit 959
Gemma 3Google · Chat270M–27B dense128KGemma Terms of Useuse policy1× RTX 4090 24GB4-bitFP8 · 1× A100 40GB32.9 GB FP8 · 4-bit 16.5
Mistral Small 3.1Mistral AI · Chat24B dense128KApache 2.01× T4 16GB4-bitFP8 · 1× RTX 5090 32GB28.8 GB FP8 · 4-bit 14.4
MiniMax-M3 on Hugging Face (opens in a new tab)MiniMax · Chat~428B MoE~23B active1MMiniMax Community Licenserevenue cap4× H1004-bitFP8 · 8× H100512 GB FP8 · 4-bit 256
Llama 4 ScoutMeta · Chat109B MoE17B active10M (claimed)Llama 4 Community Licenseconditions1× H100 80GB4-bitFP8 · 1× H200 141GB130 GB FP8 · 4-bit 65.2
Mistral Small 4Mistral AI · Chat119B MoE6.5B active256KApache 2.01× H100 80GB4-bitFP8 · 2× H100143 GB FP8 · 4-bit 71.6
Command A+Cohere · Chat218B MoE25B active128KApache 2.01× H200 141GB4-bitFP8 · 4× H100263 GB FP8 · 4-bit 131
Llama 4 MaverickMeta · Chat400B MoE17B active1MLlama 4 Community Licenseconditions4× H1004-bitFP8 · 8× H100482 GB FP8 · 4-bit 241
Mistral Large 2Mistral AI · Chat123B dense128KMistral Research License (MRL)research onlycommercial option: Mistral Large 31× H100 80GB4-bitFP8 · 2× H100147 GB FP8 · 4-bit 73.6
Muse Spark Open on Hugging Face (opens in a new tab)Meta · Chat816B MoE42B active1MMuse Model Licenselimited commercial8× H1004-bitFP8 · 8× H200986 GB FP8 · 4-bit 493
Mistral Large 3Mistral AI · Chat675B MoE41B active256KApache 2.08× H1004-bitFP8 · 8× H200810 GB FP8 · 4-bit 405
Command ACohere · Chat111B dense256KCC-BY-NC-4.0non-commercialcommercial option: Command A+1× H100 80GB4-bitFP8 · 1× H200 141GB133 GB FP8 · 4-bit 66.6
> show all 60 models> show the top 15

est. VRAM = parameters × bytes per weight (BF16 2, FP8 1, 4-bit 0.5) × 1.2 for runtime overhead. Long-context KV cache comes on top. A configuration fits when the estimate is at or below its GPU memory. Estimates, not measurements. Bars: solid to the 4-bit figure, hatched on to FP8, on one scale that is logarithmic between the GPU configurations. Rank within a category by trailing 30-day Hugging Face downloads of the primary repo, read 2026-10-27; embeddings and rerankers are ranked together. We size the real configuration in the pilot.

How it works, and what it costs

Four rows. Start at the first, stop wherever it solves your problem.

Pricing ladder: pilot, production deployment, managed ops, confidential compute

  1. fixed

    Pilot

    • One open-weight model live on your data in 72 hours. The clock starts after access and prerequisites.
    • OpenAI-compatible endpoint.
    • Reference evaluation on your data.
    • Readout and production scoping.

  2. one-time, hardware separate

    Production deployment

    • Model selection and GPU sizing.
    • Quantization, batching, caching.
    • Monitoring, RBAC, audit logging.
    • OpenAI-compatible API inside your network.
    • Handover and documentation.

  3. per GPU node / month

    Managed ops

    • Model and driver updates, monitoring, on-call, SLA tiers.
    • 99.95% uptime SLA and 5-minute incident response.
    • 24/7 across Limassol and Auckland.

  4. Confidential compute

    • Run private AI on GPUs you don't own: NVIDIA confidential computing on H100, H200 and Blackwell.
    • Encrypted VRAM plus CPU TEE, with hardware-verified attestation.
    • The hardware owner sees only ciphertext. Your prompts, your data and your model stay unseen.

We size the hardware and source it at cost, with no markup. One H100 80GB ≈ $25–30k. a 4×H100 node ≈ $180–240k. Smaller models run on far less.

Start the $4,900 pilotTalk to an engineer

Same SDK. Change the base URL.

Your existing OpenAI client keeps working. The endpoint lives inside your network.

Tested-integration matrix on request.

client = OpenAI(
    base_url="https://llm.yourcompany.internal/v1",   # inside your network
    api_key="your-key",
)

Proof from live deployments

Customer figures as reported. Not a benchmark.

Customer results: Nocodo, Infiano hackathon, UpSystems.
SourceResultDetail
Nocodo LTD~$10,000 → ~$2,000 / monthOpenAI API bill moved to their own infrastructure. 3× faster responses. Compliance overhead down ~60%.
Infiano hackathon40 GPUs in 48 hours240 submissions scored for ≈$3,000 vs ≈$6,750 on a cloud API. Time to first token 1.1–1.4 s across all 40 GPUs. Zero infrastructure-caused restarts.
UpSystemsSince 2000 · 99.95% SLA22 engineers, 100+ clients, 24/7, offices in Limassol (Cyprus) and Auckland (New Zealand).
Nocodo, monthly cost
OpenAI API~$10,000
Own infrastructure~$2,000
Infiano, cost to score 240 submissions
Cloud API≈$6,750
40-GPU cluster≈$3,000

Bars start at $0. Each pair is drawn to its own scale: the longer bar is the cloud figure, the shorter is the same ratio of it (2,000 / 10,000 and 3,000 / 6,750).

“LLMDeploy team has unique expertise in understanding our situation and adapting to our needs, while delivering the fastest, reasonable and very affordable winning solutions.”Almas Abulkhairov, CEO, Nocodo LTD

Talk to an engineer.

Tell us the model, the data and the GPUs you have. An engineer from Limassol or Auckland scopes the 72-hour pilot with you.

Start the $4,900 pilotTalk to an engineer
limassol.officeLocal time

 

Limassol HQ

Cyprus, 24/7

+357 25 123877
auckland.officeLocal time

 

Auckland

New Zealand, 24/7

+64 9 887 9088