Part 1 · CLI Runtime Structure

9. Dissecting the CLI agent runtime — the “physical form” of the harness

Remember: The internal container structure below, including engine codename and OS build, comes from runtime inspection. It is an inference made by identifying internal components in a way similar to prompt injection and matching them to common agent architecture components and roles; it is not official guidance. Codenames, directory structure, and processing methods may change.

9.1 In one line — where does the agent run?

• Session = one isolated container. The OS is Microsoft Azure Linux 3.0 (Microsoft’s own container OS). A new one is provisioned for each session, and the whole container is discarded when the session ends.
• Inside this container, the LLM runs the agentic loop as a low-privilege (sandbox) user and calls real tools such as bash, grep, view, edit, and python.
• In other words, the “coding harness + CLI layer” described in Chapter 1 is physically implemented here.

9.2 Root directory structure

/
├── app/                        ← AI workspace (area used by the agent)
│   ├── uploads/                ← Files coming from outside (KnowledgeSearch downloads)
│   ├── created/                ← Final outputs generated by AI (user downloads)
│   ├── workspace/              ← Temporary workspace and preprocessing artifacts for AI work
│   ├── copilot/                ← AI system internal state
│   │   └── session-store.db    ← (SQLite) in-session memory
│   ├── skills/                 ← Collection of AI skills (tools)
│   └── data/                   ← Node.js runtime cache
├── dracarys/                   ← 🔒 AI engine core (sandbox access blocked)
├── opt/                        ← AI model cache
│   ├── fastembed_cache/        ← Embedding models (bge-small-en, etc.)
│   └── hf_cache/               ← HuggingFace model cache (including xet storage)
├── usr/
│   ├── bin/                    ← System executables
│   ├── lib/python3.12/         ← Python standard library
│   ├── local/bin/              ← Custom tools such as sandbox-exec
│   └── share/{fonts, jupyter}/ ← System fonts (noto-cjk, etc.) and Jupyter data
├── etc/{ssl, adc}/             ← SSL certificates and Azure credential settings
├── home/{sandbox, nonroot}/    ← Executing user home and non-root user home
├── run/dracarys/               ← Runtime socket for communicating with the AI engine
├── var/{log, lib/rpm}/         ← System logs and package DB
└── tmp, dev, boot, mnt, proc, sys/   ← Temporary, device, and kernel virtual FS

Notable points:

9.3 Security and isolation model — implementation of the Chapter 1 “Sandbox”

sandbox user
   ↓ can access
 /app/   /tmp/   /home/sandbox/
   ↓ access blocked 🔒
 /dracarys/   ← AI engine body (core protected area)

• The agent (LLM-driven) runs as the least-privilege sandbox user. Work and outputs are confined to /app.
• Even the agent cannot read the engine body (/dracarys/). Even if prompt injection tries to hijack the agent, it cannot reach another engine or another session.
• This is a typical Azure Container Instance-based isolated execution environment. The phrase “running within the same governance boundary” from Chapter 4 is physically realized as this container boundary.

Analogy: The agent is “an intern playing in the sandbox,” while the engine is “a locked machine room.” Even if the intern makes a mistake or someone tricks the intern, the machine-room door does not open.

9.4 Knowledge handling — the most improved area

The biggest beneficiary of the shift to CLI and containers is how knowledge sources are handled. The area under /app is created and managed per session.

What changed — from referencing simple search results to opening and analyzing real files:

1. KnowledgeSearch (SharePoint knowledge) still uses Graph API-based Copilot Search — same as before up to this point.
2. (New) It returns an answer + download URL together from search result metadata.
3. (New) When needed, it uses that URL to download the actual file into the container (/app/uploads) for processing.
4. (New) It processes the downloaded file at scale with Python libraries — “large data is fine as long as there are enough tokens to write the code” (same direction as Gemini’s code-based processing).

→ While Classic answered from search snippets only, the New CLI agent opens the original file itself and aggregates or transforms tens of thousands of Excel rows or hundreds of PDF pages with code. This is where you can see why the Chapter 2 gains for large files +8.3 points / code interpreter +41.8 points are possible.

Why Classic could not handle Excel properly — limits of RAG (search-based processing):

Classic Copilot Studio processed files through a search (RAG) pipeline.

User question
   ↓ generate search query
Azure AI Search / SharePoint Search
   ↓ return only chunk-level text
Insert chunks into LLM context → generate answer

The key is that it does not receive the whole file.

• Files are indexed (vectorized) in advance
• Only the Top-K chunks similar to the question (usually 512–2048 tokens) are retrieved and injected into the LLM
• It does not directly access the original file ❌

That makes it structurally weak for questions that require aggregating an entire table, as in Excel. If rows are split across chunk boundaries, totals and averages become wrong; if the necessary rows are not in the Top-K results, it cannot answer at all. The fact that “no matter how large the file is, only the Top-K chunks enter the LLM” was both a strength and a limitation.

Same question, different processing — approximation vs exact value:

Classic (RAG):
  "What is the average price by manufacturer?"
  → chunk search → related text fragments → approximation
  → cannot answer if not in the chunks ❌

New CLI (code execution):
  "What is the average price by manufacturer?"
  → Python: df.groupby('Manufacturer')['Price'].mean()
  → aggregate all rows → exact numbers ✅ (sorting and filtering also possible)

Comparison of the two paradigms:

One-line summary: Classic is “search-based (RAG)” — it turns files into an index and answers from chunks. New CLI is “execution-based (Code+Tool)” — it receives files directly and processes them accurately with Python. Both reach the same M365 data, but their processing paradigms are fundamentally different.

Then won’t tokens explode when it receives a large file?:

A natural question, but the answer is “no” — because the LLM does not read the file directly in the first place. If you look at what KnowledgeSearch actually returns:

Title:     02. telecom_device_analysis_dummy_data.xlsx
File path:  /app/uploads/02. telecom_device_analysis_dummy_data.xlsx
Snippets:  (empty — binary, so text extraction is unavailable)

For binaries such as xlsx, pptx, and pdf, snippets are empty and only the file path is provided (text documents include some content). So instead of dumping the file into context, the LLM has Python process it.

# ❌ Bad pattern: file → directly into LLM context → 💥 token overflow
# ✅ Good pattern: file → Python processing → summary only → LLM

df = pd.read_excel('large_file.xlsx')           # processed in Python memory
result = df.groupby('col').agg({'val': 'sum'})  # Python performs the computation too
print(result)   # ← the LLM reads only this summary (tens of tokens)

Supporting strategies (details in the Part 2 hands-on):

• Load selected columns/sheets — use usecols, sheet_name, and nrows to load only what is needed
• Chunk processing — use pd.read_csv(..., chunksize=10000) to split even millions of rows
• Metadata first — inspect the sheet list, dtypes, and a five-row sample first, then decide the next script

Core principle: File size and token consumption are unrelated as long as Python is doing the processing. The LLM reads only the output of print(). In practice, even a 1,000-row × 34-column Excel file did not waste tokens because the LLM did not read the original directly; it read only the aggregate results (hundreds of tokens).

Note: When the session ends, the originals and outputs under /app/uploads are deleted with the container (9.6). Materials are “processed,” but they do not “remain” in the container.

9.5 Python runtime — the reality of the code interpreter

For security, external site access and package installation (pip install) are not allowed. Instead, a rich set of packages is preinstalled.

→ The “Word/PPT/Excel/CSV/PDF outputs in a dedicated container” from Chapter 0 are realized through this preinstalled stack. It is a compromise: block installation to preserve security, but preinstall the libraries needed for real work.

9.6 Memory architecture — implementation of Chapter 6

The “structured notes and hierarchical memory” from Chapter 6 (memory) is actually implemented as /app/copilot/session-store.db (SQLite).

DB table structure:

These tables map exactly to the Chapter 6 principles: checkpoints = Anthropic’s structured notes (claude-progress.txt), forge_trajectory_events = tool trajectory logs, and search_index = just-in-time recall that retrieves past context again.

Memory flow:

User message input
   ↓  stored in turns table
session-store.db
   ↓  used as LLM context → AI response generated
response also stored in turns table
   ↓
FTS indexing into search_index (searchable later)

Memory split into two layers (core):

┌──────────────────────────────────────────┐
│   Microsoft 365 cloud (persistent)        │ ← Maintained across sessions
│   · Copilot memory (server-side)          │
│   · User profile and settings             │
│   · SharePoint / Exchange data            │
└───────────────────┬──────────────────────┘
                    │ Injected at session start
┌───────────────────▼──────────────────────┐
│   Container /app/ (temporary, per session)│ ← Destroyed when session ends
│   · session-store.db (conversation turns) │
│   · uploads/ workspace/ created/          │
│   · Processed files and artifacts         │
└──────────────────────────────────────────┘

Limits (must understand):

• session-state/ and config.json are owned by root → sandbox access blocked 🔒
• When the session ends, even the DB is deleted — this is not persistent memory, but session-scoped memory
• session-store.db-wal (write-ahead log) continues to grow while the session is alive

Analogy: /app/ is RAM (temporary), and the M365 cloud is HDD (persistent). The container is RAM that starts fresh for every session, so cross-session memory must live on the cloud (M365) side. If Chapter 6’s compaction/note-taking are economy techniques inside RAM, the M365 persistence layer above them is long-term memory.

9.7 fastembed_cache — strengthening memory by applying RAG to the conversation

Remember: The search_index in 9.6 is not simple storage. It is a mechanism that searches and retrieves past turns when the context window overflows, and the local embedding engine that enables that search is /opt/fastembed_cache. In other words, this structure applies RAG to the conversation itself.

Role separation between the two embedding caches:

The two model caches under /opt (9.2) have similar names, but completely different roles.

• fastembed_cache = the local embedding engine actually used. It converts user messages into vectors inside the container without internet access and inserts them into search_index (FTS5).
• hf_cache = not a model store, but only transfer-log residue from downloading the model at build time (xet protocol logs). That is why it is only 56 KB.

User message → fastembed(bge-small) → vector → saved to search_index
                     ↑
            runs inside the container without internet (offline)

Two embeddings, two purposes — document RAG vs conversation RAG:

This container contains two different embeddings. Do not confuse this with KnowledgeSearch (document search) from Chapters 4 and 9.4.

┌─────────────────────────────────────────────────┐
│  KnowledgeSearch  (M365 Semantic Index, cloud)  │
│  · SharePoint document vector index             │ ← for file search
│  · Azure large embedding (text-embedding-ada, etc.)│
└───────────────────────┬─────────────────────────┘
                        │ returns search results
┌───────────────────────▼─────────────────────────┐
│  Current container local                         │
│  · fastembed (bge-small-en, ONNX, 65 MB)         │ ← for conversation search
│  · search_index in session-store.db              │
│    └ full-text and semantic search over session turns│
└─────────────────────────────────────────────────┘

Analogy: M365 Semantic Index = the index for the file warehouse; local fastembed = the index for sticky notes from the current conversation. They index completely different data for different purposes, so both exist.

Why it is needed — augmenting the context window with search:

The key is not “storage,” but compensating for context-window overflow. As a session gets longer, earlier turns are pushed out of the window.

Turn 0  ████
Turn 1  ████
...
Turn 50 ████
Turn 51 ████  ← Context window exceeded! Earlier content is truncated

At that point, search_index answers questions such as “what was the filename of the PPT we made earlier?”

User: "What was the filename of the PPT we made earlier?"
   ↓ no longer in the context window
FTS/vector search in search_index  ("PPT filename" → matches turn 4)
   ↓ dynamically inserts that turn into context
"It is telecom_executive_report.pptx"

Three-layer memory reinforcement structure (9.6 + fastembed integrated):

┌─────────────────────────────────────┐
│      Context window (short term)     │  keeps only the latest N turns; fastest
│      Turn N-5 ~ Turn N               │
└──────────────┬──────────────────────┘
               │ when exceeded
┌──────────────▼──────────────────────┐
│  search_index + fastembed (mid term) │  searches all turns in the session
│  FTS5 + local vector search          │  retrieves only relevant turns
└──────────────┬──────────────────────┘
               │ after session ends
┌──────────────▼──────────────────────┐
│      M365 cloud (long term)          │  persistent storage; used in future sessions
│      Copilot memory and summaries    │
└─────────────────────────────────────┘

Conclusion: “Memory enhancement” is the right term. But the goal is not storage; the core is accurately searching and retrieving past turns when the context window is insufficient. Just as RAG is applied to documents (9.4), this structure applies RAG to the conversation itself—the local implementation of Chapter 6’s “structured notes and just-in-time recall.”