Skip to main content
📖 The AI Tool Bible

AI private LLM hosting

Editorial picks for "private llm hosting".

48 tools

All Fine-tuning →
TA

Together AI

Featured
Fine-tuning · Llama / Mistral / Qwen / DeepSeek and others
8.6

Fine-tune & serve open-weight models (Llama, Mistral, DeepSeek).

Paid· Pay-per-token; fine-tuning per-tokenopen modelsfine-tuning
MO

Modal

Fine-tuning · Infrastructure (any model you can host)
8.7

Serverless GPUs and infra for training & serving ML.

Freemium· $30/mo free credits; pay-as-you-go GPU ratesserverless GPUfine-tuning
AB

AWS Bedrock

Agents · Multi-model: Anthropic Claude, Meta Llama, Mistral, Cohere, AI21, Amazon Nova/Titan, DeepSeek, Stability, OpenAI GPT
8.6

Build and scale generative AI applications with foundation models

Paid· Standard: Contact sales · Flex: Contact sales · Priority: Contact sales · Reserved: Contact salesEnterprise RAG chatbot over private documentsMulti-step tool-using agents via AgentCore
GV

Google Vertex AI

Agents · Gemini 2.5 (Pro/Flash/Nano), Imagen, Veo, Chirp, plus Model Garden (Llama, Mistral, Claude via partner)
8.6

Google Cloud's unified platform for building, deploying, and scaling enterprise AI agents and models.

Paid· Image Data - Training (Classification): $3.465 / 1 hour · Image Data - Training (Object Detection): $3.465 / 1 hour · Image Data - Deployment and Online Prediction: $1.375 / 1 hour · Image Data - Batch Prediction: $2.222 / 1 hour · Tabular Data - Training (Classification/Regression): $21.252 / 1 hourEnterprise RAG chatbotMulti-agent customer service
IW

IBM watsonx

Agents · IBM Granite (3.x, Code, Time Series), Meta Llama 3.x, Mistral, plus other curated open models
8.6

Portfolio of AI products for building AI applications, managing trusted data, and governing AI.

EnterpriseManaging and integrating trusted dataAutomating AI governance and regulatory compliance
HA

H2O.ai

Agents · Multi-model
8.3

Enterprise AI platform combining AutoML, generative AI, and vertical agents for regulated industries.

Enterprise· Contact sales; live demo on requestenterprise-agentsautoml
LL

Llama

Fine-tuning · Llama 4 (Maverick, Scout), Llama 3.3/3.2/3.1
8.3

Meta's open-weight LLM family covering 1B mobile models up to 405B frontier and natively multimodal 10M-context Llama 4 variants.

Freemiumself-hosted-llmfine-tuning
LS

LM Studio

Agents · Multi-model (gpt-oss, Qwen3, Gemma, DeepSeek-R1, Llama, others)
8.3

LM Studio Bionic is a natively local agent for open models built for creativity, work, and code.

Freemium· Free plan at $0; Bionic+ plan at $20 USD / month; Pro plan at $100 USD / monthCoding tasks and automationsReal-time voice transcription
RU

RunPod

Fine-tuning · Bring-your-own (any open-weight or custom model)
8.3

Paid· Pod H100 SXM: $4.39/hr; Pod RTX 4090: $0.89/hr; Serverless 80 GB H100 PRO: $5.89/hr; Serverless 24 GB 4090 PRO: $1.34/hr; Cluster H100 SXM: $4.79/GPU/hrModel training and fine-tuningServerless inference deployment
VL

vLLM

Fine-tuning · Multi-model (open-weight LLMs: Llama, Qwen, DeepSeek, Mistral, Gemma, Phi, etc.)
8.3

Open-source high-throughput inference engine for serving LLMs with PagedAttention and continuous batching.

Free· Free and open-source (Apache 2.0); self-hosted infrastructure costs applyllm-servingself-hosted-inference
BE

BentoML

Agents · Multi-model
8.2

Open-source framework and managed platform for serving and scaling AI models in production.

Freemium· OSS free (Apache 2.0); managed Bento cloud has free tier + usage-based pricingmodel-servingllm-inference
CO

CoreWeave

Fine-tuning · DeepSeek
8.2

An AI cloud platform purpose-built to turn complex AI workloads into discovery and creation.

Enterprise· Free tier · from $0.015/moAI model trainingAI inference
DA

Daytona

Agents
8.2

Secure and elastic infrastructure for running AI-generated code.

Freemium· Compute vCPU is $0.0504/h; Memory is $0.0162/h per GiB; Storage is $0.000108/h per GiB after the first 5 GiB are free; Windows OS is $0.0858/vCPU/h; $200 in free compute is includedAI evaluationsCode interpreter
SG

SGLang

Fine-tuning · Multi-model (DeepSeek, Qwen, Llama, Mistral, GLM, GPT-OSS)
8.2

Open-source inference framework for large language, vision-language, and diffusion models optimized for agentic workloads and large-scale serving.

FreeAgentic workloadsLarge-scale serving
LA

Lambda

Fine-tuning · NVIDIA VR200 NVL72, NVIDIA GB300 NVL72, NVIDIA HGX B200, NVIDIA HGX B300, NVIDIA H100
8.1

Supercomputers for training and inference

Paid· NVIDIA HGX B200 1-Click Cluster (16 GPUs, 2 weeks – 1 year) is $9.86 per GPU per hour; NVIDIA H100 1-Click Cluster (16 GPUs, 2 weeks – 1 year) is $6.16 per GPU per hour; NVIDIA HGX B200 Instance (8x, 180 GB VRAM) is $7.69 per GPU per hour; NVIDIA HGX H100 Instance (8x, 80 GB VRAM) is $4.59 per GPU per hour; NVIDIA HGX A100 Instance (8x, 80 GB VRAM) is $3.19 per GPU per hour; NVIDIA HGX A100 Instance (8x, 40 GB VRAM) is $1.99 per GPU per hour; NVIDIA Tesla V100 Instance (8x, 16 GB VRAM) is $0.79 per GPU per hourTraining and inferenceTraining foundation models
LO

LocalAI

Writing · Multi-model (llama.cpp, diffusers, whisper, etc.)
8.1

LocalAI runs open models on the user's own hardware.

FreeChatting with open models on local hardwareRunning Python scripts on data files
TA

Together AI Fine-tuning

Fine-tuning · Multi-model (any Hugging Face open-source model)
8.1

Managed fine-tuning for open-source models to improve accuracy and control behavior without managing infrastructure.

Paid· Usage-based; cost estimator in-product, no public price listllm-fine-tuningvision-fine-tuning
AN

Anyscale

Fine-tuning · Infrastructure (any model)
7.9

Anyscale helps AI builders run data-intensive workloads to build and deploy Foundation Models and AI at scale on any cloud, powered by Ray.

PaidMultimodal data curationDistributed model training
FA

Fireworks AI

Fine-tuning · Multi-model (DeepSeek, Qwen, GLM, Kimi, Gemma, Minimax, others)
7.9

A training and inference platform that transforms open models into specialized intelligence.

Paid· Serverless: From $0.05 / 1M input tokens; Managed Training: From $0.50 / 1M training tokens; Dedicated Training API: From $8.00 / GPU-hourOpen model inferenceModel training
LA

Lamini

Fine-tuning · Lamini (built on open base models)
7.7

Memory-tuning platform for grounding LLMs in your facts.

Paid· Enterprise / contact salesenterprise FTfactual recall
MO

Modzy

Agents · Multi-model
7.7

Enterprise ModelOps platform for deploying and running AI/ML models across cloud, on-prem, and edge.

Enterprisemodel-deploymentedge-ai
OM

oMLX

Coding · Multi-model (Qwen, Llama, Mistral, Gemma, DeepSeek, MiniMax, GLM)
7.5

Native macOS LLM inference server built on MLX, with paged SSD KV caching for Apple Silicon agents.

Free· Free, Apache 2.0 open sourcelocal-llm-inferencecoding-agents
FE

FedML

Fine-tuning · Bring-your-own (PyTorch, Hugging Face)
7.3

Distributed training, fine-tuning, and serving platform with federated learning roots.

Freemium· Open-source library free; managed GPU usage pay-as-you-gofine-tuningdistributed-training
SE

Seldon

Agents · Multi-model (bring your own)
7.3

Kubernetes-native MLOps framework for real-time inference, observability, and compliance at scale.

Freemiumreal-time inferenceA/B testing and canary deployments
VA

Valohai

Agents · Multi-model
7.3

MLOps platform for versioned pipelines, distributed training, and LLM evaluation across any cloud.

Enterprise· Free trial; contact sales for pricingmlopsllm-evaluation
BE

Beam

Coding · openai/gpt-oss-20b
7.2

Serverless GPU infrastructure for AI workloads with sub-second cold starts and bring-your-own-cloud support.

Freemium· $30 free credit refreshed monthly; usage-based beyond thatgpu-inferenceagent-sandboxes
DD

Domino Data Lab

Agents · Multi-model
7.2

Enterprise AI platform for building, scaling, and governing AI-powered applications.

EnterpriseBuilding AI systems and applicationsClinical trial submissions
GE

Geniusrise

Agents · Multi-model
7.2

Open-source framework for building, deploying, and scaling AI microservices across text, vision, and audio.

Free· Free, open source; self-hostedinference-servingfine-tuning
OL

Ollama

Coding · Multi-model (Llama, Qwen, Gemma, DeepSeek, Mistral, Phi, etc.)
7.2

The de facto runtime for running open-weights LLMs locally, now with a paid cloud tier for bigger models.

Freemium· Free tier · from $20/molocal-llmself-hosted-inference
PG

Paperspace Gradient

Fine-tuning · Bring-your-own (PyTorch, TensorFlow, Hugging Face)
7.2

MLOps platform for building and scaling AI projects on DigitalOcean.

Freemium· Free tier · from $8/moexploring libraries and datasetstraining and fine-tuning models
WA

Wallaroo.AI

Agents · Multi-model
7.2

Production AI inference platform for deploying and monitoring models across cloud, on-prem, and edge.

Enterprise· Starter: $500 · Wallaroo Community Edition: Free · Ampere Community Edition: Freemodel-deploymentmlops
IG

Iguazio

Agents · Multi-model
7.1

Enterprise MLOps and GenAI platform for taking models from notebook to production at scale.

Enterprise· Contact sales; free trial availablemlopsllm-fine-tuning
NR

Neu.ro

Agents · Multi-model
7.1

Infrastructure-agnostic MLOps platform for the full ML/DL lifecycle across public, hybrid, and on-prem clouds.

Enterprise· Contact sales; no public pricingmlopsmodel-training
SG

Scale GenAI Platform

Fine-tuning · Multi-model (OpenAI, Google, Meta, Mistral)
7.1

Scale GenAI Platform provides tools to build, deploy, and improve agents that reason over enterprise data and scale reliably.

Enterprise· Contact sales; enterprise contracts onlyMulti-agent workflowsHuman feedback loops
AS

Amazon SageMaker

Agents · Multi-model
7.0

The next generation of Amazon SageMaker is the center for all your data, analytics, and AI.

FreemiumModel training and deploymentGenerative AI application development
FO

Forefront

Fine-tuning · Multi-model (Mistral-7B, Mixtral, Phi-2)
7.0

Fine-tune and serve open-source LLMs on your own data without managing GPUs.

Paidfine-tuningopen-source-llms
JS

Jina Serve

Agents · StableLM
7.0

Framework for building and scaling multimodal AI services and pipelines over gRPC, HTTP, and WebSockets.

FreemiumLLM serving with streamingEmbedding services
OP

OpenSandbox

Agents · Model-agnostic
7.0

OpenSandbox provides isolated environments for AI agents to execute code, use tools, and explore the web on user infrastructure.

Freemodel-generated code analysisparallel evaluation and training workloads
PR

PrivateGPT

RAG · Multi-model (BYO local LLM)
7.0

Open-source API layer that turns local models into production AI applications.

Free· OSS free; Zylon enterprise contract (contact sales)Context-aware AI apps based on documentsProduction AI applications
CH

Chassis

Agents
6.9

Open-source tool that auto-packages ML models into production-ready Docker containers with a prediction API.

Free· Free, open source (Apache-style community project)model-packagingedge-deployment
KA

Katonic AI

Agents · Multi-model (2,600+ via AI Gateway)
6.9

Full-stack enterprise platform for building, deploying, and governing AI agents on your own infrastructure.

Enterprise· Contact sales for quote; no public pricingEnterprise agent building and orchestrationSovereign AI deployment on private infrastructure
TR

TrueFoundry

Agents · Multi-model
6.9

Enterprise control plane for deploying, governing, and scaling agentic AI on your own infrastructure.

Enterprise· Developer: $0 · Pro*: $499 · Pro Plus: $2999 · Enterprise: Customagent-deploymentllm-serving
PG

Prediction Guard

Agents · Multi-model
6.6

Self-hosted AI control plane that lets regulated enterprises govern models, agents, and MCP servers behind their firewall.

Enterprise· Contact salesai-governanceself-hosted-llm
AU

AutotuneLLM

Fine-tuning

An open-source optimization layer that sits between your app and Ollama to squeeze more performance out of local LLMs.

Free· Free and open source (MIT licensed).Local LLM inference on Apple SiliconReducing KV cache RAM for Ollama models
LG

LLM Gateway

Agents · Routes to GPT-4o, Claude 3.5 Sonnet, Gemini 1.5, Llama 3.1, Mistral, and 200+ others

Freemium· Free tierAutomatic failover between model vendorsPrompt-injection and PII guardrails
LG

LLM GPU Checker (KO)

Evaluation · Catalog covers open models on Hugging Face (Llama, Qwen, Mistral, Gemma, etc.)

Match LLMs to GPUs and plan multi-model AI stacks by VRAM, bandwidth and precision.

Free· Free (open web tool hosted on GitHub Pages).GPU sizing for self-hosted LLMsMulti-GPU RAG stack planning
OW

Open WebUI

Agents · Backend-agnostic: any Ollama, llama.cpp, vLLM, or OpenAI-compatible API (OpenAI GPT, Anthropic Claude, Llama 3.x, Qwen, Mistral, Gemma, etc.)

Self-hosted, extensible AI chat platform that runs on your infrastructure

Freemium· Free (self-hosted, MIT-style community license via pip/Docker) / Enterprise: custom pricing for SSO, RBAC, audit logs, air-gapped deployment, data-residency guaranteesSelf-hosted ChatGPT alternative for a teamPrivate RAG chatbot over internal documents
QU

QuantProbe

Evaluation

Interactive calculator predicting LLM decode speed and memory fit from the tiered decode law, with charts showing the speed-memory trade-off.

Free· Free and open source; install with `pip install quantprobe` or use the hosted web calculator.Predicting decode speed and memory fit before downloading model weightsSelecting a model and bit-width for a specific machine