Guides

AI Agent Development

Direct Claude Code or Codex to build TelemetryOS applications through the Developer App's MCP server

AI Agent Development

In this workflow, you describe the application in natural language and an AI coding agent — Claude Code or Codex — writes the code, takes screenshots of the live canvas, seeds store data, reads logs, and iterates visually while the Developer App renders the result. You watch the app come together in real time.

Prefer to write the code yourself? See Manual Development. The two workflows share the same SDK, templates, and publish pipeline — pick the one you'll spend most of your time in, and you can always combine them.

Before You Start

Complete Getting Started first so you have the Developer App installed and a project open on the canvas. This page picks up from there.

What the Developer App Pre-Wires

When you create a project from the Developer App's New Project dialog, three files are placed in the project root. Together they make the project AI-ready.

.mcp.json

Points the agent at the Developer App's built-in MCP server on http://localhost:2025:

{
  "mcpServers": {
    "tos": {
      "type": "http",
      "url": "http://localhost:2025/mcp/<encoded-project-path>"
    }
  }
}

When an MCP-aware agent opens the project, it connects to the tos server automatically — no global install, no account linking, no API keys.

.claude/settings.local.json

Pre-approves the tool permissions the agent needs, so no approval prompts appear during normal work:

{
  "permissions": {
    "allow": [
      "Bash(npm*)",
      "Bash(pnpm*)",
      "mcp__tos__*",
      "WebFetch(domain:docs.telemetryos.com)"
    ]
  },
  "enabledMcpjsonServers": ["tos"]
}

CLAUDE.md

A project context file describing the application, its mount points, store scopes, and build commands. The agent reads this automatically on session start.

Setting Up the Agent

The Developer App must be running with the project open for the MCP server to expose its tools. The server only provides screenshots, store access, and logs while the project is active in the app.

Claude Code

Open a terminal in the project directory (Tools → Open Project in Terminal, Cmd+T) and run:

claude

Claude Code reads .mcp.json automatically and connects to the tos server. The pre-approved permissions in .claude/settings.local.json mean tool calls proceed without prompts, and the agent loads CLAUDE.md as session context.

Codex

Codex uses a different context filename. Copy the scaffolded context file:

cp CLAUDE.md AGENTS.md

Ensure your Codex ~/.codex/config.toml knows about the Developer App MCP server. Either let Codex read the project's .mcp.json, or add a user-level entry pointing at the same URL. Then run codex in the project directory.

Other MCP-Compatible Agents

Any agent that speaks Model Context Protocol can use the same .mcp.json. Most agents look for their own variant of CLAUDE.md — copy it to the filename the agent expects (AGENTS.md, .cursor/rules, etc.) and delete or ignore .claude/ if the agent uses a different permission file.

The Workflow

  1. The Developer App is running with the project open on the canvas.
  2. The agent is running in the project directory.
  3. You describe what you want in natural language. The agent edits files, checks the canvas via screenshots, seeds realistic store data, reads logs to catch errors, and iterates.

Example prompts:

  • "Build a weather dashboard showing the current temperature for a city configured in Settings. Use OpenWeatherMap and fall back to a placeholder if the API fails."
  • "Make the render view layout work for both 16:9 landscape and 9:16 portrait. Screenshot both to verify."
  • "The store sync is glitchy — read the dev server logs and figure out why the Render view isn't updating when Settings changes."

The dev server runs on a dynamic $PORT assigned by the Developer App; the MCP tools abstract this, so the agent never needs the specific URL.

MCP Tools Available

The tos MCP server exposes tools across six categories. The agent discovers them automatically on connection.

  • Visual — take screenshots of the render, settings, or web mount points; sweep all aspect ratio presets in one call; set the aspect ratio, background, or color scheme.
  • Store — get/set/delete keys across all four storage scopes; list keys and namespaces; dump all entries.
  • DOM — click elements, fill inputs, read text, and evaluate JS inside a mount point.
  • Logs — read application console output, dev server stdout/stderr, and internal utility logs, with time and level filtering.
  • Lifecycle — reload the window (soft or hard), restart the dev server, clear all stub data.
  • Project — list projects open in the Developer App.

The key insight: the agent never needs to leave its session. It gets visual feedback (screenshots), data manipulation (store tools), and diagnostics (logs) inside the same context that writes the code.

Customizing CLAUDE.md

The scaffolded context file is a starting point. Add project-specific context as the application grows:

## Project-Specific Context

**External APIs:**
- OpenWeatherMap: https://api.openweathermap.org/data/2.5/weather
  - Authentication: API key in query string (`appid=`)
  - Rate limit: 60 calls/minute on free tier

**Store Keys:**
- `city` (string): selected city for weather lookup
- `units` ('metric' | 'imperial'): temperature unit preference

**Known Quirks:**
- The OpenWeatherMap response field `weather[0].main` is our condition display

The more specific the context, the less the agent has to guess.

Guardrails to Reinforce

A few TelemetryOS-specific gotchas that agents sometimes miss. Worth stating explicitly in CLAUDE.md (the scaffold covers them) or in your prompt:

  • Call configure() in src/index.tsx before createRoot().render(...).
  • Use proxy().fetch() instead of fetch() for external APIs.
  • Avoid store().device in Settings or Web mount points — use store().instance or store().application.
  • Wrap SDK calls in try/catch — they time out at 30 seconds.

Combining with Manual Edits

Nothing stops you from alt-tabbing to your editor mid-session. A common pattern:

  1. Describe the app shape to the agent and let it scaffold the core.
  2. Hand-edit the pieces you care about stylistically — typography, animations, specific prose.
  3. Return to the agent for the next feature, bug, or aspect-ratio pass.

The Developer App's HMR picks up edits from any source, and the agent's next screenshot reflects whatever's on disk.

Troubleshooting

MCP tools aren't connecting

The most common cause is that the Developer App is not running, or the project is open but the dev server isn't ready. Open the project in the Developer App and wait for the canvas to render. In Claude Code, run /mcp to force a reconnect.

Tool calls fail with permission prompts

.claude/settings.local.json should already cover common tools. If a new tool (new MCP method, new shell command) prompts for approval once, approving it adds it to the file for subsequent calls.

Agent writes code that breaks in the simulated runtime

See the Guardrails section above — almost always one of those four. Reinforcing them in your prompt, or adding them to CLAUDE.md, fixes it.

Documentation References for Agents

When adding to CLAUDE.md or prompting an agent manually, these are the pages the agent will find most useful:

Next Steps

  • Code Examples — complete applications to reference or have the agent adapt
  • Platform MCP Server — beyond building apps, connect an agent to your live TelemetryOS account to manage devices, playlists, and media
  • Manual Development — the other workflow, if you want to try it or mix it in

Updated about 1 month ago

What’s Next