AI private LLM hosting

LocalAI

Writing · Multi-model (llama.cpp, diffusers, whisper, etc.)

8.1

Self-hosted OpenAI-compatible API for running LLMs, image, and audio models on your own hardware.

Tabnine

Coding · Proprietary (Tabnine-trained, on-prem capable)

7.6

Privacy-focused AI autocomplete with on-prem options.

Pathway

RAG · Multi-model

7.3

Live data framework for production RAG and streaming ETL pipelines in Python.

Forefront

Fine-tuning · Multi-model (Mistral-7B, Mixtral, Phi-2)

7.0

Fine-tune and serve open-source LLMs on your own data without managing GPUs.

AWS Bedrock

Agents · Multi-model: Anthropic Claude, Meta Llama, Mistral, Cohere, AI21, Amazon Nova/Titan, DeepSeek, Stability, OpenAI GPT

8.6

Build and scale generative AI applications with foundation models

Replicate

Fine-tuning · Thousands of community + first-party models

8.5

One-API platform for running and fine-tuning open-source models.

ChatGLM (Zhipu Qingyan)

Agents · GLM-4, GLM-4V, GLM-4-Air/Flash, CogView, CogVideoX (ChatGLM family, Zhipu AI / Tsinghua KEG)

8.4

Zhipu AI's bilingual ChatGLM assistant with agents, code, image and video generation

H2O.ai

Agents · Multi-model

8.3

Enterprise AI platform combining AutoML, generative AI, and vertical agents for regulated industries.

LM Studio

Agents · Multi-model (gpt-oss, Qwen3, Gemma, DeepSeek-R1, Llama, others)

8.3

Desktop app for discovering, downloading, and running open-weight LLMs locally with an OpenAI-compatible server.

RunPod

Fine-tuning · Bring-your-own (any open-weight or custom model)

8.3

On-demand GPU cloud and serverless inference platform built specifically for AI workloads.

vLLM

Fine-tuning · Multi-model (open-weight LLMs: Llama, Qwen, DeepSeek, Mistral, Gemma, Phi, etc.)

8.3

Open-source high-throughput inference engine for serving LLMs with PagedAttention and continuous batching.

Jan

Writing · Multi-model (local open-weights + OpenAI/Claude/Gemini via API)

8.2

Open-source desktop ChatGPT alternative that runs local LLMs and routes to cloud providers from one app.

Msty

Writing · Multi-model (Ollama, Claude, GPT, Gemini, others)

8.2

Privacy-first desktop AI workspace that runs local and cloud models side by side.

SGLang

Fine-tuning · Multi-model (DeepSeek, Qwen, Llama, Mistral, GLM, GPT-OSS)

8.2

Open-source high-throughput inference engine for LLMs and multimodal models with OpenAI-compatible serving.

Sim

Agents · Multi-model (Claude, OpenAI, Google)

8.2

Open-source visual workspace for building, deploying, and monitoring AI agents.

Vellum

Agents · Multi-model (Claude-class LLMs)

8.2

Personal AI assistant with persistent memory across email, calendar, and chat channels.

Apple Intelligence

Writing · Apple Foundation Models + Private Cloud Compute; optional ChatGPT

8.1

Apple's on-device AI layer baked into iOS, iPadOS, macOS, and visionOS with a rebuilt Siri, writing tools, and image generation.

Athina AI

Evaluation · Multi-model

8.1

Collaborative LLM evaluation and observability platform for teams shipping AI features to production.

Hugging Face AutoTrain

Fine-tuning · Multi-model (Hugging Face Hub)

8.1

No-code fine-tuning and training pipeline that spins up state-of-the-art models on the Hugging Face Hub.

MLflow

Evaluation · Multi-model

8.1

Open-source platform for tracking, evaluating, and deploying ML models and LLM applications.

OpenAI Evals

Evaluation · OpenAI GPT models (extensible)

8.1

OpenAI's open-source framework for benchmarking LLMs against a shared registry of evaluations.

Rivet

Agents · Multi-model

8.0

Open-source visual IDE for building and debugging LLM agent graphs.

Wren AI

RAG · Multi-model (OpenAI, Anthropic, Gemini, self-hosted)

8.0

Open-source GenBI semantic layer that lets AI agents query your warehouse in natural language with governed, accurate SQL.

AnythingLLM

RAG · Multi-model

7.9

Open-source desktop and self-hosted app that turns your documents into a private chat-and-agent workspace.

Deepgram

Audio · Nova, Flux, Speak (proprietary)

7.9

Production-grade speech-to-text, text-to-speech, and voice-agent APIs for real-time and batch audio.

oMLX

Coding · Multi-model (Qwen, Llama, Mistral, Gemma, DeepSeek, MiniMax, GLM)

7.5

Native macOS LLM inference server built on MLX, with paged SSD KV caching for Apple Silicon agents.

Langchain-Chatchat

RAG · Multi-model (GLM-4, Qwen2, Llama 3, etc. via Xinference/Ollama/LocalAI/FastChat)

7.4

Self-hostable RAG and agent framework that wires LangChain to any local open-source LLM and a knowledge base.

Gorilla

Agents · gorilla-openfunctions-v2 (6.91B)

7.3

Open-source LLM purpose-built for function calling and API invocation across thousands of tools.

OpenBB

Agents · Multi-model (Agent Rita is model-agnostic)

7.3

Open-source financial workspace where analysts and AI agents share the same governed data.

Anyword

Writing · Multi-model

7.2

AI marketing copy platform that scores and predicts content performance before you publish.

FinChat (Fiscal.ai)

RAG · Multi-model (proprietary finance-tuned copilot)

7.2

AI copilot for equity research that reads filings, transcripts, and KPI tables across 100,000+ public companies.

Ollama

Coding · Multi-model (Llama, Qwen, Gemma, DeepSeek, Mistral, Phi, etc.)

7.2

The de facto runtime for running open-weights LLMs locally, now with a paid cloud tier for bigger models.

Qwen Chat

Writing · Qwen3 family (Qwen3-Max, Qwen-VL, Qwen-Coder, Qwen-Image)

7.2

Alibaba's flagship chatbot fronting the Qwen family of open-weight LLMs, with vision, code, and image generation in one UI.

WeKnora

RAG · Multi-model

7.2

Tencent's open-source RAG framework that turns raw documents into a queryable knowledge base, ReAct agent, and self-maintaining wiki.

ElizaOS

Agents · Multi-model (BYO via plugins)

7.1

Open-source agentic OS for building and orchestrating multi-agent swarms across Discord, Telegram, X, and onchain.

Respan (formerly Keywords AI)

Evaluation · Multi-model (500+ via gateway)

7.1

LLM engineering platform combining a multi-model gateway with tracing, evals, and prompt management.

SEAL Leaderboard

Evaluation · Multi-model (GPT, Claude, Gemini, Llama, etc.)

7.1

Private, expert-graded leaderboards from Scale AI that rank frontier LLMs on domains contaminated public benchmarks can no longer measure.

Bisheng

Agents · Multi-model

7.0

Open-source enterprise AgentOps platform for building, orchestrating, and governing LLM agents at scale.

CodeRabbit

Coding · Multi-model

7.0

AI pull-request reviewer that lives inside GitHub, GitLab, and your IDE.

Kotaemon

RAG · Multi-model (OpenAI, LlamaCPP, any OpenAI-compatible endpoint)

7.0

Open-source RAG UI for chatting with your own documents, locally or self-hosted.

MaxKB

RAG · Multi-model

7.0

Open-source enterprise RAG and agent platform with built-in workflow engine and multi-LLM support.

PPT Master

Writing · Bring your own LLM

7.0

Open-source AI that turns documents into natively editable PowerPoint decks with real shapes, not flattened images.

PrivateGPT

RAG · Multi-model (BYO local LLM)

7.0

Production-ready, air-gapped RAG framework for querying your documents with local LLMs.

RAGs by LlamaIndex

RAG · Multi-model (OpenAI, Anthropic, Replicate, HuggingFace)

7.0

Open-source Streamlit app that builds a custom RAG pipeline from a natural-language brief.

Superwise

Evaluation · Multi-model

7.0

Agentic management platform for runtime guardrails, policy enforcement, and observability across LLM agents.

Cohere

RAG · Command, Embed, Rerank, Transcribe (proprietary)

6.9

Enterprise-grade LLM platform built for private, secure, and customizable deployment.

DeepSearcher

RAG · Multi-model (DeepSeek, OpenAI o1/o3-mini, Claude, Llama, others)

6.9

Open-source agentic RAG framework for private enterprise data, built by the Zilliz/Milvus team.

Fynk

Writing

6.9

AI-assisted contract lifecycle management with eIDAS-grade e-signatures and risk flagging built in.