The local control plane for continuously evaluating, supervising, and improving AI systems.
OpenOrbit does not use AI merely to automate work. It automates the operating cycle around an AI system itself: evaluate it on a schedule, retain evidence, review supervisor feedback, and make its improvement history observable.
Run the control room locally, keep the operational record in your own AppData, and decide which changes deserve approval.
Start with a Quick Start, watch the evaluation loop take shape, and review evidence-backed improvements in one place.
An AI feature can look healthy in a demo and still regress after a prompt, model, tool, or product change. OpenOrbit gives that feature a repeatable operating loop rather than a one-off test:
flowchart LR
A[AI system] --> B[Recurring evaluation]
B --> C[Logs, browser evidence, and traces]
C --> D[Supervisor review]
D --> E[Issues and improvement proposals]
E --> F[Human-approved changes]
F --> A
- Define reusable builds from a target, workflow, runner, fixed test cases, manager prompt, and AI model profile.
- Run a one-off test before saving a build, or run its configured lifecycle repeatedly with clear approval boundaries.
- Inspect every phase through process logs, structured evidence, browser screenshots, supervisor responses, and OpenTelemetry traces.
- Review reported issues and proposed improvements in a durable decision history.
- Observe the PDCA cycle across iterations instead of treating a single model response as the whole story.
- Stop active work from one local control room.
OpenOrbit is designed around the information an operator needs at each stage:
| Area | What it answers |
|---|---|
| Dashboard | Is the AI system healthy right now? What changed recently? |
| Builds | What exactly is being evaluated, with which assets and policy? |
| Run detail | What happened in each phase, and what evidence supports the result? |
| Improvements | Are feedback, decisions, and scores actually improving over time? |
The screenshots below follow a customer-support AI through its retained evaluation evidence and improvement cycle.
Open a retained run to review the behavior observed from the evaluated AI at each iteration. Then compare that behavior with the proposed improvement and the evidence supporting it.
Compare feedback volume, accepted changes, scores, and run health across multiple builds. The history makes it clear whether the operating cycle is improving the AI system over time.
Configure a System AI model to use the built-in Chat Assistant for questions about evaluation work, runners, and the control room. Your own AI agent can work with the same local operating data through OpenOrbit's versioned API.
insighta cloud Inc. Development partner |
We are looking for thoughtful collaborators who share our belief that AI systems should be observable, controllable, and continuously improved. Contributions are especially welcome from people working on agent harnesses, browser evaluation (including Playwright), local automation, and evidence-backed operational loops. Start a fork, open a small issue, improve the docs, or help shape a larger idea—every contribution is welcome.
New to the project? Browse good first issues, ask a question or share an idea in Issues, or read the contribution guide before opening a pull request.
Open the Quick starts section on the Dashboard, choose a guided template, fill in the few target-specific fields, and create the build. Each template creates the runner, fixed test cases, environments, manager policy, and model profile configuration together—so you can start with a working operating loop instead of assembling every asset by hand.
| Quick Start | Use it when | Example first run |
|---|---|---|
| User journey smoke test | You need a recurring, read-only browser check for a local product. | Confirm that the home page loads and the primary heading is visible. |
| Site exploration review | You want evidence-backed product feedback from safe same-site exploration. | Explore the documentation or dashboard and retain the visited pages behind each recommendation. |
| Agent self-improvement | You want to improve a managed prompt from real AI responses. | Send a support request to the configured target model, retain its response, and let the supervisor approve only response-backed prompt changes. |
| AI SLO and behavior drift monitor | You already have a structured evaluator for quality, safety, latency, or cost. | Connect its probe command and compare the retained metrics with the configured baseline and thresholds. |
For example, to evaluate a support-agent prompt:
- Choose Agent self-improvement.
- Select the Git repository and managed prompt file, then choose your AI model profile.
- Enter one representative user request and its response-level acceptance criterion.
- Create the build and run it. OpenOrbit retains the actual target-AI response, asks the supervisor to review that evidence, and applies only an adopted, reversible prompt improvement on the following iteration.
Quick Starts never store provider keys. They reference the environment-variable name already configured in the selected model profile.
| Requirement | Version | Used for |
|---|---|---|
| Python | 3.13+ | Local API and runner SDK |
| Git | 2.40+ recommended | Install from Git and repository-backed evaluation cycles |
| Node.js | 24+ | Installing from Git and frontend development |
The runner SDK reference is generated from the Python module and its docstrings. Preview it locally with:
pnpm run docs:serveBuild a static documentation site with pnpm run docs:build, or
run pnpm run build to generate both the SDK docs and the control-room UI.
Install the packaged release:
python -m pip install openorbit
orbit runThe wheel already includes the bundled control-room UI, so Node.js and pnpm are not required at runtime.
PyPI publication is made possible with the support of insighta cloud Inc.
To use the latest development version, install directly from the main OpenOrbit repository. This source installation requires Node.js 24+ and pnpm:
python -m pip install "openorbit @ git+https://github.com/forthfate/openorbit.git@main"Or run it once with npm:
npx openorbit runOpen http://127.0.0.1:3000. If that port is occupied, OpenOrbit selects the next available port and prints its URL. Set ORBIT_PORT and ORBIT_HOST when you need a specific listener:
ORBIT_PORT=8787 ORBIT_HOST=0.0.0.0 orbit rungit clone https://github.com/forthfate/openorbit.git
cd openorbit
uv sync --extra dev
corepack enable
pnpm install
pnpm run build
pnpm run runFor frontend development, start the API and Vite separately:
uv run uvicorn app.main:app --app-dir backend --reload --port 3000
pnpm --filter agent-improvement-console-ui run devThen open the Vite URL shown in the terminal, normally http://localhost:5173.
OpenOrbit is a standalone, local-first control plane. It does not host or resell an AI model, and it does not require an OpenOrbit cloud account.
Bring your own AI: create a model profile for the API provider and model your team already uses, then select that profile for supervisor evaluation, Cycle Improvement AI, or the Chat Assistant. OpenOrbit stores only the environment variable name for a provider credential—not the credential itself—and keeps the operational record in your local AppData.
Only builds that run browser journeys need a Chromium browser and its platform-specific system libraries. This is not required to start OpenOrbit, create assets, review runs, or use non-browser runners.
- Create or choose an AI model profile in Assets.
- Add the runner, workflow, fixed test cases, and target environment that describe the AI system you want to evaluate.
- Create a Build from those assets.
- Use Test to execute the build once and inspect its full run detail without adding it to the evaluation-run history.
- Start a regular run when ready, then review evidence and supervisor results in Runs.
- Use Improvements to compare scores, feedback, decisions, and cycle health over time.
| Concept | Meaning |
|---|---|
| Asset | A reusable model profile, runner, workflow, prompt, test set, or environment. |
| Build | A versioned operating configuration that connects assets to one AI-system evaluation. |
| Test | A transient, one-time execution used to validate a build. |
| Run | A retained execution record, including phases, evidence, logs, and decisions. |
| Supervisor | An AI review step that produces structured evaluation results, issues, and proposals. |
| Improvement cycle | The evidence-backed PDCA loop across multiple evaluations and human decisions. |
OpenOrbit is local-first. Operational state is stored outside the repository in platform AppData:
- Windows:
%LOCALAPPDATA%\\Orbit - macOS:
~/Library/Application Support/Orbit - Linux:
${XDG_DATA_HOME:-~/.local/share}/orbit
Set ORBIT_APP_DATA to use another location. Model profiles store the name of the environment variable that contains a secret, never the secret itself. Review workflow commands, approved workspace boundaries, and network exposure before connecting a production AI system.
OpenOrbit exposes a local, versioned API:
- Swagger UI:
http://localhost:3000/api/docs - OpenAPI document:
http://localhost:3000/api/openapi.json - API base:
http://localhost:3000/api/v1
Read the API reference for endpoint details. To add reusable automation, create a Python runner with explicit lifecycle phases:
from orbit_sdk import runner
@runner.phase("execute")
def verify(ctx):
ctx.log("Run one bounded evaluation step")
if __name__ == "__main__":
runner.main()Runners are intentionally bounded. They provide evidence to the control plane; they do not start their own scheduler or silently modify a target system.
Contributions are welcome: bug reports, evaluation-runner templates, documentation improvements, and product feedback all help.
uv run ruff check orbit/ backend/ tests/
PYTHONPATH=backend uv run pytest -q
pnpm --filter agent-improvement-console-ui run lint
pnpm --filter agent-improvement-console-ui run buildPlease open a pull request rather than pushing directly to main. See CONTRIBUTING.md for development, checks, and release rules.
Copyright © 2026 forthfate and insighta cloud Inc.
Released under the MIT License.