Skip to main content
πŸ“– The AI Tool Bible

Braintrust Loop

Braintrust Loop is the AI agent built into Braintrust. It investigates logs, experiments and datasets across a project, and it can create and edit prompts, scorers, datasets, facets, custom views, dashboards and automations. It runs read-only work without interrupting you and pauses for your approval before it creates or edits anything.

Summarised by The AI Tool Bible editorial team from Braintrust's own documentation (linked below), read on Oct 16, 2026. Not hands-on testing. For pricing, pros and cons, see our Braintrust review.

What is Braintrust Loop?

Loop runs in a Braintrust-managed runtime, so long investigations keep going and every conversation is saved as a thread you can return to. You can start without configuring an AI provider: Loop uses built-in models billed against your organization's monthly model credits.

Loop also powers two other Braintrust features, Patterns and the Debugger, which investigate on your behalf without a chat.

What can Braintrust Loop do?

Braintrust groups Loop's abilities into four jobs. Each one is described in its "What Loop can do" documentation, with example prompts.

  • Investigate: query logs, experiments and datasets with SQL, summarize and diagnose individual traces, and search logs for recurring problems, which it records as patterns with supporting evidence. Connected external MCP servers (public preview) let it bring in context such as issue trackers or internal documentation.
  • Build: suggest, create and save scorers (a scorer returns a number, a classifier returns a label), create human review scores, create and update prompts, generate datasets from logs and edit rows, save dataset snapshots and point environments at them, build custom views and dashboard charts, and configure the Topics pipeline. Dataset snapshots, environments and custom dashboards are Pro and Enterprise features; Braintrust's plans page lists none of them on Starter.
  • Run and automate: run an experiment from a dataset, from rows it supplies inline or from a prior experiment, create alerts for matching logs or threshold crossings, and create scheduled Loop automations with their own instructions. Automations can be paused and made active again.
  • Get help: search the Braintrust documentation, resolve object names and URLs, summarize an experiment's metrics against a baseline, and generate shareable links to objects it creates.

How do you use Braintrust Loop?

  1. Open Loop from the Loop page, or from the prompt box on the project overview page. You can also open it from the logs table, SQL sandbox, trace view, playgrounds, datasets, Patterns, individual dashboards and the chart editor; each opens the same agent with that page's data in context.
  2. Choose a model and reasoning effort with the model picker. The picker also sets the provider.
  3. Add context: type @ followed by a name to reference a dataset, experiment, project log, playground or SQL query. You can also attach images, audio, video, PDF, text and JSON files, within the documented limits of 8 files per message, 20 MB per file and 40 MB per message.
  4. Ask your question and press Enter. Review the results in the workspace that opens beside the chat, where several objects can stay open in tabs.

Does Braintrust Loop change things without asking?

No, by default. Before it creates or modifies a Braintrust object, Loop describes the action and asks for approval. In a shared thread it accesses Braintrust with the sender's permissions, and only the sender can approve or cancel the work their message started.

There is an Auto-accept edits setting that skips approvals for tools that support it. Braintrust's documentation warns that auto-accept can allow production changes, such as retagging an environment or creating an online scoring rule, and says to leave it off if you need to review those first.

Which models does Braintrust Loop use, and what limits apply?

Each thread has a model picker with a model, a reasoning effort and a provider. Selecting Braintrust as the provider uses the built-in models and draws on the organization's monthly model credits; the supported provider types for billing inference yourself are OpenAI, Azure, Bedrock, OpenRouter and Azure AI Gateway. On the Starter plan, the built-in models need at least one organization owner with a work email address, or a payment method on file.

Braintrust-hosted Loop limits concurrent active threads for each member, service token and automation: 5 on Starter without on-demand usage, and 50 on Starter with on-demand usage, Pro and Enterprise. At the limit, starting work in another thread returns HTTP 429.

Braintrust's plans page lists monthly model credits of $10 on Starter, $100 on Pro and custom on Enterprise. Credits do not roll over, and on Starter, if you use them up without on-demand usage enabled, built-in model usage is paused until the next monthly cycle or until you add a card. For the plan prices themselves, see the pricing table in our Braintrust review.

What can't Braintrust Loop do?

  • Reach another Braintrust organization's data. A thread is scoped to the project it was opened in.
  • Delete Braintrust objects. It can create and update prompts, scorers, facets, preprocessors, datasets, views and automations, but cannot delete them. It can delete dataset rows or close a pattern, which marks the pattern inactive.
  • Change organization settings: members, permission groups, API keys, AI providers and billing.
  • Read someone else's private threads. Threads are private by default and owners can share them with project members.
  • Run approval-gated tools in an interactive thread unless you approve them or enable auto-accept. Scheduled runs use separate permission rules.

Is Braintrust Loop the same as Braintrust's closed loop?

No. Our Braintrust review describes a closed-loop workflow, where eval datasets, experiments and production monitoring feed each other in one product. Loop is a separate feature inside that product: an AI agent you chat with, or schedule, to investigate your data and build the objects that workflow uses.

Sources

Read Oct 16, 2026. Vendor documentation changes; check the linked pages for current behaviour and limits.

← Back to the Braintrust review