PilotDeck adopts a layered architecture to decouple interaction portals, task execution and basic capabilities. Desktop, web, CLI/TUI, and messaging channels share the same Agent runtime; session, file, and project memories are organized around the WorkSpace, allowing tasks to be continued between different portals.

1. System Panorama

A request enters the Gateway from the interaction portal, and the corresponding WorkSpace and session are located by session routing, and then handed over to the Agent for execution. Agents coordinate context, models, and tools during runtime, and continuously return responses, status, and artifacts to the interface.

2. Layered Design

3. Task Execution Mechanism

Agents translate natural language requests into an ongoing understand-action-feedback loop. The model determines the next action, the tool completes the specific operation, and the execution results are returned to the model context; when the task is completed or the execution is terminated, this round of requests ends.

3.1 Sessions and Tasks

Sessions save the state of ongoing work, including historical messages, model preferences, execution status, and usage. One user input forms a Turn, which is responsible for organizing this round of model invocation and tool execution. Sessions can contain multiple Turns, so users can continue to ask questions, add conditions, or adjust goals based on the results of the previous round.

3.2 Single Round Execution Process

Text, tool calls, and status are continuously output as streaming events. Persistent records are written along with the execution process, allowing the interface to display real-time progress and read the results of this round later.

3.3 Model and Tool Collaboration

The model is responsible for planning and judgment, and the tool is responsible for interacting with the actual environment. The tool call is executed after parameter verification and permission check, and the results are added to subsequent model requests. The Agent continues to iterate when multiple steps are required; cancellation, an unrecoverable error, or a loop limit ends execution. Recoverable request errors are retried according to the run policy.

4. Context Management

Context is the basis for model understanding tasks. PilotDeck organizes core instructions, project instructions, skills, historical messages, tool results and related long-term memory as input for this request, and allocates budget according to the context window of the model.

As conversations grow, the system processes larger tool results and compresses historical context when necessary. Large results can be saved to a file to retain read references; historical compression generates summaries while retaining the information and tool call relationships needed to continue execution. When switching models, the budget is rechecked against the target model.

Session compression is used to continue the current task, and long-term memory is used to reuse knowledge across sessions. The two are maintained separately: the compressed summary is not directly equivalent to a long-term memory, and the long-term memory is formed and updated independently through Recall, Index and Dream.

5. Multi-Model Access

PilotDeck unifies the requests and responses of different services through the model adaptation layer. Agents use consistent message and event formats to perform tasks, and the adaptation layer is responsible for converting them into the interface protocol of the target service. Model selection, routing and fallback can therefore be completed in the same execution flow.

5.1 Unified Messages and Events

Unified protocols describe text, multimodal content, tool calls, streaming responses, and usage information. The upper-layer session does not need to maintain a separate execution process for each model, and the interaction portal can also present responses and tool status in a consistent manner.

5.2 Interface Protocol

Capabilities such as image input, reflection parameters, and context windows are determined by the specific model and its configuration. Protocol adaptation provides a unified access method, and requests are still organized according to the capabilities of the selected model during runtime.

6. Tools and Permissions

Tools enable Agents to read data, modify files, execute commands, and access external services. All tools are registered and called through a unified interface, and parameters, working modes, and permissions are checked before execution. Available capabilities vary depending on the operating environment and configuration.

6.1 Tool Capabilities

6.2 Execution and Scheduling

Concurrent scheduling arranges calls according to the concurrency safety declared by the tool: operations that can be safely concurrently executed in parallel, the remaining operations are executed sequentially, and the results are returned in the order of the original calls. The runtime also supports sequential scheduling to adapt to different execution needs.

File access is based on the working directory and additional authorized directories, and combined with the current permission mode. WorkSpace provides project context, and the specific scope of operations is determined by permission rules and tool execution methods.

6.3 External Tool Access

MCP converts external services into tools that the Agent can call. After the server connection is established, its tools enter the unified execution flow, and the model can use these capabilities just like the built-in tools. Global configuration is located at <PilotHome>/mcp.json, the project configuration is located at <project-root>/.pilotdeck/mcp.json;The server with the same name is subject to the project configuration.

7. Extension Mechanism

PilotDeck extends task capabilities through Skills, Plugins, and lifecycle Hooks. Skills describe the operation methods and knowledge of specific tasks; Plugins provide tools, commands, MCP server and other capabilities; Hooks participate in processing at specified running stages. Extensions can be reused globally or loaded with projects, allowing the universal runtime to adapt to different work scenarios.

WorkSpace is the basic unit for organizing project work in PilotDeck. It is centered on the working directory and links sessions, file operations, project memory and project expansion. Each project has its own work context, so the agent can understand the project in a continuous dialogue and reuse existing experience between different tasks.

1. Workspace Concept

Selecting a project directory determines the execution location of the Agent and the project file entry. File operations are based on this directory and follow permission rules to access projects and additional authorized paths.

Multiple sessions can be created in a WorkSpace: they hold conversation and execution state separately, while using the same project's files and long-term memory. For example, feature development and documentation maintenance can occur in separate sessions, sharing project context and confirmed commitments.

2. Project Identity and Working Directory

PilotDeck uses the real path to the project to identify the WorkSpace, separating the interface display name from the storage identity. The directory name is convenient for display, and the path is used to accurately locate the project; directories with the same name will be distinguished by path information to prevent historical sessions from being mistakenly classified into another project.

The execution directory is responsible for locating files and running commands, and the data directory is responsible for saving sessions and auxiliary records. The two are organized separately, allowing you to simultaneously switch task contexts when switching projects and retain their respective histories.

3. Data Organization

Project files remain in the original working directory. Running data such as session records and file history are managed by PilotDeck's data directory; long-term memory is organized by workspace, and user portraits that can be reused across projects are retained globally.

Conversations are persisted as JSONL for easy appending and restoring in execution order. Directory tags in the project store record the real path and are used to correlate historical data back to the corresponding working directory.

4. Session Management

A session is a unit within a WorkSpace that continuously advances tasks. New sessions start in the project context, and historical sessions restore existing messages and settings. Closing a running session releases runtime resources, and saved conversation records remain available for subsequent opening.

4.1 Session Life Cycle

The session is idle while waiting for input; it enters the running state after receiving the request and continues to output text and tool progress. After normal completion, it returns to the idle state. If it is canceled or fails to execute, the corresponding results will be recorded. The user can continue the task in the same session or create a new session to work on another goal.

4.2 Session Recovery

When you open a historical session, the system restores messages, session settings, and control boundaries from persistent records. For compressed history, restore the most recent valid compressed snapshot and continue subsequent records, allowing new requests to start from a context that can continue execution.

Session recovery retains the task history; whether the command or file operation has taken effect will be based on the status in the actual working directory. Subsequent tasks still read the current environment through the tool and continue processing.

4.3 Dialogue and Execution Records

Transcript records user input, model messages, tool results, task status, file artifacts, and final results by execution. Compression boundaries are also saved along with session metadata, allowing live execution and historical recovery to use a consistent recording basis.

Larger tool results can be dropped independently and have references saved in the context. This not only retains the original information that can be read, but also controls the input size of subsequent model requests.

5. Configuration and Project Expansion

Running parameters such as model, routing and Memory are determined by the currently loaded pilotdeck.yaml management. The default configuration is located at <PilotHome>/pilotdeck.yaml, or via startup parameters or PILOTDECK_CONFIG_PATH Specify the configuration file.

Project-specific capabilities are placed in the working directory .pilotdeck/ Below, load together with global capabilities:

<project-root>/.pilotdeck/
├── plugins/ # Project plugin
├── skills/ # project skills
└── mcp.json # Project MCP connection

A project-level MCP's server with the same name overrides the global setting. Project extensions are selected with the working directory, and run parameters are uniformly provided by the selected YAML configuration.

6. Multi-Project and General

Manage multiple WorkSpaces via project and historical session lists on web and desktop. When switching projects, the execution directory, session and project memory are loaded according to the selected project; user portraits that are common across projects continue to be reused.

General provides an independent working directory for tasks that are not tied to a specific project. Used by default on the web side ~/general, can also be passed WORKSPACES_ROOT Adjust. Its execution directory and PilotDeck data directory are managed separately.

The memory retrieval of the normal WorkSpace revolves around the current project; the General can select relevant project memories from known workspaces based on the task. This allows project knowledge to be continuously accumulated within the project and provides context for cross-project tasks when needed.

Model Routing

Router is PilotDeck’s model routing engine. It automatically selects the most appropriate model based on task complexity and tries an alternate model when a request fails, reducing unnecessary model overhead.

router:
  tokenSaver:
    enabled: true
    judge: openai/gpt-4.1-mini
    defaultTier: medium
    subagent:
      policy: judge
    tiers:
      simple:
        model: openai/gpt-4.1-mini
      medium:
        model: openai/gpt-4.1-mini
      reasoning:
        model: openai/gpt-4.1
      complex:
        model: openai/gpt-4.1

router:
  fallback:
    default:
      - openai/gpt-4.1-mini
      - deepseek/deepseek-chat

router:
  zeroUsageRetry:
    enabled: true
    maxAttempts: 2

router:
  autoOrchestrate:
    enabled: true
    triggerTiers: [complex]

PilotDeck's long-term memory is provided by Recall, Index and Dream Composed of three stages. Recall retrieves relevant knowledge for the current task, Index converts conversation experience into lasting memory, and Dream organizes and updates existing memories. The three together form a closed loop of continuous learning and reuse, allowing project experience to continue across sessions.

1. Core Competencies

2. Three-Stage Working Mechanism

The three stages have different running rhythms: Recall is executed on demand while preparing model requests, Index and Dream are run through background maintenance or manual operations. They work together around the same long-term memory, rather than requiring all stages to be completed simultaneously in each conversation.

2.1 Recall: Recall on Demand

Recall starts from the current user request and recent conversations to determine whether long-term memory is required and whether user personas, project knowledge, or both should be read. For project knowledge, the system first checks the memory directory and item description, then selects the relevant file to read the text, and organizes the results into background information for this request.

General WorkSpace uses the current project as the search scope; General can select related projects from known workspaces. No memory is injected when there is no relevant content to prevent irrelevant knowledge from occupying the context.

2.2 Index: Experience Storage

Index transforms experiences gained from interactions into knowledge that can be reused over the long term. After each round of dialogue, the system first saves the original record and its conversation, time and source information, and then the indexing process processes the content to be refined.

The index combines conversational semantics to refine project status, decisions, and facts, records work rules in user feedback, and classifies stable user information into identity notes. The results are saved in classified file memory for subsequent Recall to read. Raw conversation capture is input preparation for this stage; searchable knowledge is formed after indexing is complete.

2.3 Dream: Memory Organization

As conversations accumulate, multiple memories may be formed on the same topic, and project status and user preferences may change. Dream organizes existing memories as a whole: identifying topics with overlapping or conflicting content, organizing and merging entries by association, revising project meta-information, and summarizing user identity notes to update portraits.

The content to be indexed will be processed before sorting. Then the revision results are generated in the temporary copy, and then submitted to the official memory database, and the snapshot before change tracking and consolidation is saved. The organized knowledge continues to participate in subsequent Recall, allowing long-term memory to remain available as the project progresses.

3. Memory Organization and Storage

Long-term memory is organized by purpose, not just by conversation time:

Markdown files store readable and editable knowledge, and memory directories and project metainformation help Recall locate content. SQLite saves the original conversation, processing status and execution trace, providing the basis for Index and Dream to run. The default data directory is <PilotHome>/memory, each workspace uses independent path identification to organize project data.

4. Collaboration with Context

Memory and session context bear long-term knowledge and current task information respectively. The relevant knowledge read by Recall is encapsulated by the context system and added to System Prompt as a supplementary background; the user's current request and session history continue to constitute the main input of the task.

Explicit instructions in the current request take precedence over historical memory. After the user proposes new rules or corrections, these interactions will become the input of subsequent Index and Dream, so that the memory is gradually updated.

4.1 Memory Life Cycle in Requests

Background maintenance schedules Index and Dream according to configured intervals and pending content. When memory retrieval times out or temporarily fails, the task can continue to use the existing session context; diagnostic information is used for subsequent troubleshooting.

5. Activation and Scheduling

in pilotdeck.yaml Enable Memory in and set the run interval for indexing and collation. The intervals below are sample configurations that can be adjusted based on conversation volume and model cost:

memory:
  enabled: true
  provider: edgeclaw
  captureStrategy: last_turn
  includeAssistant: true
  maxMessageChars: 6000
  retrievalTimeoutMs: 30000
  schedule:
    autoIndexIntervalMinutes: 30
    autoDreamIntervalMinutes: 240

captureStrategy optional last_turn or full_session, controls the capture scope of the original conversation.retrievalTimeoutMs Controls how long a request waits for memory retrieval. The two maintenance intervals control automatic indexing and automatic collation respectively; actual execution also depends on whether there is any pending content.

Memory can be passed model Specify the model pool provider/model, the Agent default model is used when not specified. No Memory or settings configured enabled: false , the built-in memory service is shut down.

6. Observability and Management

The memory management interface provides file viewing and editing, abandonment and recovery, manual indexing and organization, as well as import and export. The trace records of Recall, Index and Dream respectively show how the knowledge is read, how it is formed and how it is revised, making it easy to check whether the memory is consistent with the actual project.

When the new conversation has not yet been reflected in the memory, you can view the records to be indexed and perform manual indexing; when the existing memories are duplicated or need to be reorganized, you can execute Dream and check the results of the changes. Original records, documentary memory, and stage tracking together provide an examineable progression from conversation to knowledge.

PilotDeck provides two background automation capabilities: Always-On (autonomous Agent) and Cron (scheduled tasks). The former lets the Agent decide what to do on its own, while the latter performs specified tasks according to a set schedule.

1. Always-On

Always-On is PilotDeck’s background autonomous Agent function. When enabled, it will automatically browse files and recent conversations in the workspace when PilotDeck is idle, autonomously determine valuable tasks, complete operations and generate reports in an independent copy. Users only need to read the report and decide whether to adopt it.

1.1 Workflow

Always-On automatically runs the following process in each round:

(1) Discover

The Agent browses the files in the workspace and refers to recent chat records to independently judge what is worth doing and write a Plan (including title, overview and reasons).

• Maximum output is 1 plan per round
• The same tasks that have been produced before will not be repeated
• If nothing worth doing is found, the round ends and the Agent goes to sleep.

(2) Prepare a copy of the work

The Agent will first copy the contents of the workspace and operate on this copy - the original files will not be directly touched. There is only one copy of the same workspace at a time, and subsequent plans will reuse the existing copy.

(3) Execution

The agent operates according to the plan in the replica. This step has strict security restrictions: all changes only occur in the copy, and the original file is not affected; the Agent cannot push the content to the remote.

(4) Report

After the execution is completed, the Agent writes a report describing what was done, why it was done, and a summary of the changes.

(5)Application

Steps 1–4 are completely automatic, and the last step requires manual operation: after viewing the plan and reports in the resident panel, click "Apply" and the changes in the copy will be written to the workspace.

1.2 Operational Constraints

Always-On doesn't run unchecked. The following conditions are checked before each start:

The above conditions are checked every 5 minutes by default.

1.3 Sleep and Wake-Up

If the Agent feels that there is "nothing to do" after scanning a circle, it will not run idle repeatedly, but will go to sleep. During hibernation, it will pay attention to file changes in the workspace. Once it detects that a file has been modified or added, it will exit hibernation and return to the normal regular inspection process.

1.4 Configuration Instructions

Just specify the workspace path and turn on enabled. The rest of the parameters have default values and no additional adjustments are required.

2. Cron

Scheduled tasks are used to pre-specify a period of instructions and execution time. PilotDeck will automatically start the Agent for execution after the time is reached. Unlike Always-On, scheduled tasks do not require the Agent to "discover" what to do by itself. Instead, the user defines what to do and when to do it, and it is completed on time.

2.1 Two Timing Methods

2.2 Execution Process

After arriving at the scheduled time:

1. PilotDeck automatically starts an Agent to execute the set instructions
2. Agent completes operations in the background without manual intervention
3. Execution results are sent to the chat channel where the task was created

2.3 Configuration Instructions