Trajecta.
the agent that remembers ↘A desktop AI agent that learns from experience.
Trajecta is a local-first desktop agent designed to carry useful context across tasks, use tools in a controlled workspace, and turn evidence from previous work into reviewable knowledge. It combines a stateful Deep Agents runtime with a human-controlled memory and skill lifecycle.
not to act without permission.
Why build Trajecta?
The problem
An AI assistant can complete a task and still forget the decisions behind it. Giving that assistant filesystem, browser, and external-service tools also creates risk: the model should not be the authority for sensitive actions, and an apparently completed task should not automatically count as proven success.
The approach
Separate execution, retrieval, evidence capture, reflection, and skill activation. Keep agent state durable, keep permissions deterministic, and require explicit review or evaluation before new procedures become executable skills.
Act with tools
Multi-step agent tasks can use local utilities, selected workspaces, document tools, web tools, subagents and MCP integrations. Risky actions pass through permission checks.
A tool error is an observation for the agent to recover from, not necessarily the end of the whole conversation.Remember selectively
Retain semantic facts, evidence-linked episodes and approved procedures across sessions. Retrieve relevant context with keyword and vector search instead of loading everything.
Unverified experiences and conflicting instructions are treated conservatively.Improve through evidence
Capture execution trajectories and explicit corrections. A bounded reflection worker proposes lessons for human review; reusable skills have a separate, explicit evaluation path.
Completion does not mean verified success, and a suggestion is not an active skill.Keep people in control
A deterministic allow / ask / deny policy controls tool use, while LangGraph checkpoints support interruptions, approvals, and safe resumptions.
Folder boundaries and optional Docker isolation further narrow what tools can access.Inside the architecture
A thin desktop application talks to a FastAPI backend. The backend prepares a stateful agent and mediates model calls, tool permissions, retrieval, and learning. The important boundaries remain visible in the diagram.
Model routing does not grant tool permissions. A memory hit does not bypass workspace scope. A completed run does not automatically activate a skill. Each boundary is enforced by its own subsystem.
What happens when you send a task?
The process is designed to stream progress to the UI while retaining enough state to resume after a user decision or regenerate from an earlier turn.
- 01
Validate the request
FastAPI receives the conversation or scheduled task. Input checks and preflight validation run before SSE headers are emitted; conversation and workspace settings are resolved.
server/src/chat/service.py - 02
Select a model and tools
The runtime resolves the selected model through Bifrost, loads enabled local and MCP tools, takes a permission snapshot, and binds all workspace-sensitive tools to one validated folder.
server/src/chat/runtime.py - 03
Retrieve relevant context
Bounded hybrid retrieval draws from semantic facts, scoped episodes, approved experiences, and active version-matched skills. Retrieval failure is advisory rather than fatal to a chat turn.
server/src/memory/retrieval.py - 04
Execute a stateful agent
Deep Agents and LangGraph handle planning, tools, checkpoints, and branching. Streaming events distinguish output, tool activity, errors, and interruptions.
server/src/chat/runtime.py - 05
Ask before sensitive actions
The policy can hide denied tools or interrupt calls requiring approval. Pending decisions survive through a checkpoint and resume after the user decides.
server/src/guardrails/policy.py - 06
Persist the evidence
The conversation, checkpoint, final outcome, and agent trajectory are recorded. Meaningful tool-using runs can produce scoped episodes and optional reflection jobs.
server/src/chat/service.py
A memory system with boundaries
Trajecta separates ephemeral execution state from durable facts, evidence-linked episodes and executable procedures. Raw trajectories are stored independently so useful history can be inspected without treating every event as reliable memory.
Working state
Live conversation/thread state, active plan, checkpoints, recent actions, and temporary results. It is the context of a task, not a permanent user fact.
server/src/memory/short_term/store.pyDurable knowledge
Useful preferences, project facts, and remembered instructions. The agent store is the source of truth, with searchable SQLite FTS5 and Qdrant mirrors.
server/src/memory/semantic/store.pyEvidence-linked history
Bounded summaries from meaningful, finished trajectories reference the original execution evidence. Each episode remains unverified until supported by independent evidence.
server/src/memory/episodic/store.pyApproved workflows
Reusable procedures and explicitly promoted skills become loadable instructions. Pending, rejected, or obsolete suggestions do not automatically become executable skills.
server/src/memory/procedural/store.pyTwo search strategies, one local source of truth.
SQLite stores relational records and audit history; FTS5 provides keyword search; embedded Qdrant adds vector similarity when configured. Search is scoped and re-checked against authoritative records. When vector search is unavailable, the application can fall back to local keyword retrieval.
Workspace-scoped episodes, bounded ranking, conflicting-memory suppression, and user-controlled automatic memory are handled in the retrieval path.How Trajecta learns
The learning design is intentionally not a self-modifying loop that silently rewrites the agent. It distinguishes observed events, suggested lessons, reviewed knowledge and evaluated executable skills.
Execution becomes evidence
Completed work leaves append-only trajectory events: tool calls, errors, runtime events, and task outcomes. The evidence is useful even when success is not independently proven.
Keep meaningful information
Explicit preferences and corrections can be recorded. Finished tool-using work can be consolidated into episodes; a configured background worker processes meaningful events within budgets.
Suggest, do not silently rewrite
Structured Bifrost-powered reflection generates evidence-grounded lessons and procedure suggestions. The queue supports leases, retries, and restart recovery.
Human decides
Uncertain corrections and procedure proposals are sent for approve / reject review. Approval of a knowledge note is different from activation of executable code or a skill.
Optional skill candidate pathway
A separate, explicitly started workflow can mine repeated patterns, replay candidate skills against baseline and held-out tasks, then assess correctness, regressions, costs, and other outcomes.
Version, activate, or roll back
A verified candidate can be promoted with approval; the registry tracks versions and supports rejection and rollback. Only active, version-matched skill descriptions enter retrieval.
Normal conversations contribute evidence and, when enabled, reflection suggestions. Creating, benchmarking, and activating an executable candidate is a separate manual or opt-in workflow. This is the behavior documented in the current source.
What the evaluation pathway measures
These are evaluation dimensions and checks provided by the project—not published claims of a measured improvement or automatic skill-promotion rate.
Useful autonomy needs limits
Permissions are stored and enforced by the application. Input and model-context checks handle untrusted content, while LangGraph provides resumable human-in-the-loop decisions for tool calls and questions. Docker-based isolation is an optional execution mode, not a blanket guarantee for every connected service.
Policy outside the model
Tool access is determined using persisted allow, ask, or deny settings. This is a deterministic application decision rather than a text instruction asking the model to behave safely.
server/src/guardrails/policy.pyOne folder per conversation
Validated workspace selection is bound at run preparation to file tools, local utilities, and sandbox mounts. Model arguments cannot choose another user’s workspace scope.
server/src/chat/workspace.pyKeyword search survives vector failure
SQLite and FTS5 keep local records available when embedded Qdrant is unavailable. Search checks scopes and authoritative SQLite records; a vector hit alone grants no access.
server/src/memory/storage/vector.pyCheckpoint-backed branching
Editing, resending, and regenerating a message forks a conversation at a recorded checkpoint instead of silently overwriting earlier execution history.
server/src/chat/service.pyBuilt in code, not just diagrams
These short, annotated excerpts come from the project source. The GitHub repository is not currently publicly accessible, so source paths are presented as references rather than broken external links. They show three design decisions: bind tools to a workspace, fail softly on retrieval errors, and keep review decisions explicit.
Binding tools to the validated workspace
The agent does not get to choose a different root for each tool. A permission snapshot and a resolved workspace are used to construct the set of tools for that run.
workspace_root = conversation_workspace(conversation, self._settings)
personal_tools = self._personal_tools.for_workspace(workspace_root)
native_tools = personal_tools.filter_tools(
personal_tools.get_tools(),
policy,
)Fail-soft memory retrieval
Vector or memory outages should reduce available context, not prevent someone from chatting or asking the agent to complete a task.
except Exception:
# Memory search is advisory; it must never break a chat turn.
logger.warning("Unified memory retrieval unavailable", exc_info=True)Review before a lesson becomes active
Approved experiences may be retrieved as contextual guidance. This status change alone does not promote the item as an executable skill.
if decision not in {"approve", "reject"}:
raise ValueError("decision must be approve or reject")
new_status = "active" if decision == "approve" else "rejected"Explore the modules
server/src/chat/runtime.pyserver/src/chat/service.pyserver/src/memory/retrieval.pyserver/src/memory/reflection/worker.pyserver/src/skills/learning/experience.pyserver/src/skills/evaluation/evaluator.pyserver/src/guardrails/policy.pyapps/desktop/srcTrade-offs & current scope
Trajecta is under active development. The useful engineering story includes what the system deliberately does not assume.
Local-first ≠ model-independent
Core state and search live locally, but hosted models and connected services may still receive data according to their configuration. Ollama offers an optional local inference path.
Knowledge ≠ executable skill
An approved experience note can inform context. It does not bypass the candidate, evaluation, and promotion checks required for a reusable executable skill.
Reflection ≠ proof
The reflection model is constrained by evidence and budgets, but its observations still need review. Successful task completion and independent outcome verification remain separate.
Optional isolation ≠ blanket sandboxing
Docker execution depends on the selected setup and permissions. Trajecta does not claim that every tool or connected action always runs in an isolated container.
Agents should learn.
People should stay in control.
Read the implementation, explore the rest of my engineering work, or get in touch to discuss the architecture.