Watchtower — Tracing, cost and PII masking for LLM apps
Traces, prompt governance, cost per tenant and PII masked at the source.
One record of everything the agent did.
- Tracing and agent graphs: Every model call, tool, retrieval and decision — not just the LLM request.
- Prompt governance: Prompt identity and a registry, so you know which version ran where.
- Cost per tenant: Allocate spend by tenant and feature — even for models nobody prices.
- PII masked at the source: Masking happens before anything is stored or leaves the process.
- Many backends: Langfuse, Phoenix and LangSmith together, plus your own database — failures isolated per sink.
- A real dashboard: Overview, traces, sessions, costs and budgets, prompts, datasets and evaluations.
Your first real trace in four steps.
- Start the dashboard: Two commands bring up the local UI.
- Copy the connection values: Paste them into your Python project.
- Configure once: Call wt.setup at application start.
- Send a trace: Run your app and watch it appear.
Details
- Language
- Python 3.11+
- Dependencies
- None required
- Works with
- OpenAI, Anthropic, LiteLLM, LangGraph
- Sinks
- Langfuse, Phoenix, LangSmith, your database
- License
- MIT
More from ShipIt
- Vouchline — Research agents that cite and re-check every claim.
- Forge — GitHub coding agent that fixes issues and reviews PRs.
- Sentinel — Autonomous AI penetration testing with real proof.
- shipit-agent — The Python runtime under every ShipIt agent.
- Relay — A self-hosted engineer on your Claude subscription.
- ShipIt Cloud — Multi-tenant console for white-label agents.
- ShipIt CLI — Terminal coding agent with two-way chat bots.
- Pentest skills — One /pentest command for Claude Code.
- ShipitQA — Goal-driven QA in real browsers, with evidence.
- KeyVault — Zero-knowledge password manager. No server.
- MacSweep — Clean, protect and understand your Mac.
- Snappilot — Free screen capture studio, fully on-device.
- shipit-ui-design — Senior UI/UX as a Claude Code plugin.
- ShipIt Palette — One hex to a full design system.