Source: lab chronicle — synced 2026-09-23

The SAGA Homelab: A Chronicle

A living history of the SAGA purple-team homelab — from first hardware bolt to full domain compromise.

---

Prologue: Why Build This?

There is a particular kind of ambition that leads someone to build a fully vulnerable Active Directory forest in their home, staff it with pre-SAGA characters, wire up an AI control plane to attack and defend it autonomously, and document every step in a git-tracked knowledge vault. This is that story.

The SAGA homelab is the work of operator kryssar — built on physical hardware, running on Proxmox, populated with five Windows Server VMs across three AD domains, and backed by an increasingly sophisticated AI orchestration layer. What started as a place to practice AD attack techniques became something bigger: a purple-team platform where red and blue operations happen in tandem, evidence is automatically cataloged, and AI agents hand off work to each other through the night.

This chronicle tracks the homelab from its hardware foundation to its current state. It covers the wins, the crashes, the architectural pivots, and one very satisfying root dance.

---

Phase 0 — Hardware Foundation (pre-2026-05-04)

The Iron

Before any software was installed, there was hardware to assemble. The lab runs on three physical machines:

• [internal-host] ([internal-network]) — the Proxmox hypervisor hosting all SAGA VMs
• [internal-host] / ROG ([internal-network]) — an AMD Ryzen 7 7840HS laptop with 30.5 GB RAM and an RTX 4070 Max-Q; the blue-team control plane
• [internal-host] ([internal-network]) — a dedicated AI compute node running LM Studio
• Synology 925+ ([internal-network]) — NAS providing NFS mounts, evidence archives, and backups

Networking is UniFi: a USG 3P gateway and a USW Flex Mini managed switch. The Flex Mini was a deliberate choice — an unmanaged switch would not support the VLAN topology the lab required.

🤔 Decision: Upstream SAGA tooling, not custom Terraform. The Orange-Cyberdefense/SAGA framework deploys with Packer, Terraform, and Ansible — end to end, reproducible, proven. A custom Terraform stack would have taken weeks and would have needed to be maintained independently. The choice was easy.

Deploying SAGA (2026-04-19)

SAGA landed on April 19, 2026. Five Windows VMs appeared on Proxmox:

SAGA defaults to VLAN 56 on vmbr1. The lab uses VLAN 20 on vmbr0 — a VLAN-aware bridge matching the UniFi topology. That adaptation was one of the first things learned: upstream tools assume their own network model, and you have to reconcile that with yours.

The VLAN addressing scheme was deliberately chosen: third octet mirrors VLAN ID. VLAN 20 → [internal-network]. Easy to remember, easy to reason about in firewall rules and nmap output.

---

Phase 1 — The Vault is Born (2026-05-04)

First Commit

On May 4, 2026, the homelab-second-brain repository was created on the NAS at `[internal-path] A git-tracked Obsidian vault as the canonical documentation and AI context source. NFS-mounted so every machine in the lab can read it. Everything from this point forward would be documented here.

The same day: a second audit of the [internal-host] home directory was committed — a snapshot of what existed before the structured era began. Evidence of an older, messier time.

🔥 Problem: Two competing documentation sources. The old ~/projects/[internal-host] git repository had been the original documentation home. But active documentation had moved to the vault months ago without the old repo being retired. Two sources, neither authoritative, causing AI sessions to load contradictory context. The fix was ADR-0006: archive the old repo to ~/archive/. The vault becomes the single source of truth. No exceptions.

Velociraptor Arrives

Velociraptor server was deployed on [internal-host] as the telemetry collection platform. The alternative — traditional Windows Event Forwarding (WEF) — would have required a dedicated Windows collector VM. [internal-host] is Ubuntu. No Windows collector exists. Velociraptor agents were already installed on the SAGA VMs; the server made them useful.

VM100 (DS-WEAPONS-SRV) was the first to be validated: pslist, netstat, and users pulled via Velociraptor. Read-only baseline collected. The data looked clean.

🤔 ADR-0001: Blue-team stack on [internal-host], not a new Proxmox VM. The Proxmox thin-pool Data% was already under pressure from five SAGA VMs. Adding a monitoring VM would have chewed into snapshot headroom. [internal-host] had 1.7 TB free on / and more than enough CPU for Docker containers. The answer was obvious once the disk numbers were on the table.

---

Phase 2 — Blue Team Baseline (2026-05-05 to 2026-05-06)

The Snapshot Ritual

Before touching anything significant, a safety ritual was formalized: create Proxmox snapshots of all SAGA VMs first. This became the lab's equivalent of a pre-flight checklist. The policy was written down and enforced from this point forward. No reset mechanism — no action.

With snapshots in place, Velociraptor bulk enrollment ran across all five VMs without incident.

Sysmon: Deployed, Debugged, Confirmed

Sysmon 15.20 was planned and then deployed to VM100 first, SHA256 verified. Then pushed to VM101 through VM104 (ALDERAAN-CA-SRV, CORUSCANT-DC, ALDERAAN-DC, DS-COMMAND-DC).

🔥 Script bug: backtick regex corrupting base64 in PowerShell. The deploy script was transmitting the Sysmon binary as base64 over a PowerShell remote session. A backtick in the regex pattern was being interpreted as a PowerShell escape character, silently corrupting the decode. The symptom: installation appeared to succeed but the binary wouldn't run.

🔧 Fix: replaced backtick regex with ReadAllLines -join. Once the cause was identified, the fix was a one-liner. Every subsequent VM deployed cleanly. SHA256 verified on each host.

All five VMs confirmed: Sysmon Running/Automatic, Velociraptor telemetry flowing. Event IDs 1, 3, 4, 5, 10, 11, 12, 13, 16, 22 present on all hosts.

🔥 Anomaly: DS-COMMAND-DC EID 10 elevated at 723 rows. Event ID 10 (ProcessAccess) was running hot on DS-COMMAND-DC — 723 rows versus the expected ~100. Not a crash, but a signal. Default Sysmon config needed tuning for this environment. Flagged for review.

Gateway Upgrade Ordered

The USG 3P gateway was showing its age. No WireGuard support. Legacy firmware update path. The UXG-Max was ordered — 2.5GbE ports, modern VPN capability, supported hardware. VPN stayed on L2TP/IPsec until the migration was complete.

---

Phase 3 — The AI Era Begins (2026-05-06 to 2026-05-09)

The [internal-host] Comes Online

The [internal-host] at [internal-network] was wired in as a dedicated AI compute node. LM Studio loaded up: qwen2.5-72b-instruct, deepseek-r1, qwen2.5-coder-14b, and others. A redaction wrapper was built to ensure no secrets left the LAN when using AI routes.

LiteLLM was configured as a routing proxy with four tiers:

• tier-cheap-summary — fast local models for short tasks
• tier-worker — LM Studio for drafts, reports, transforms
• tier-analysis — deeper reasoning for graph queries and detection hypotheses
• tier-execution — Gemini Pro or Claude for planning and orchestration

Claude→Gemini-CLI prompt templates were created for the three session types the lab would need: DEPLOY, DOCS-ONLY, AUDIT. The pre-AI-era PATTERNS.md and manual session scripts were archived. The manual era was officially over.

🤖 AI milestone: The vault transitions from human-maintained docs to AI-native control plane. Future Claude, Gemini, and LM Studio sessions would load context from the vault rather than relying on in-session re-derivation. The investment in documentation structure was starting to pay off.

Architecture Decision Record Blitz (ADR-0006 to ADR-0014)

A blitz of architectural decisions were formalized during this period:

• ADR-0007: Surgical salvage of MCP and LM clients from the old saga-ai-lab repo — the code had value, but the network assumptions were dangerous. Surgical extraction only.
• ADR-0008/0012: Snapshot strategy — pre-attack snapshots mandatory, individual target revert capability required before escalating.
• ADR-0011: MCP servers as the AI tool interface. Standalone scripts replaced by proper MCP wrappers.
• ADR-0013: Infrastructure hardening and HTB-alignment — add a Linux attack target, build full MCP orchestration.
• ADR-0014: Individual target revert capability. The RESET POLICY in one sentence: no offensive action without a deterministic reset mechanism.

The Pro Lab Vision

A document was written: autonomous lab roadmap note. The idea: run HTB Pro Labs style scenarios at home with overnight autonomous AI loops. The lab would attack and defend itself while the operator slept. This was Phase 5's deathstar star.

May 9, 2026: Phase 5 infrastructure declared ready. The first integrated attack session — Phase 5.7, Web Recon — completed and verified. DVWA (VM200, [internal-network]) was live as the first Linux target.

---

Phase 4 — The Great Attack Campaign (2026-05-10)

The Setup

May 10 began with a strong hand: rebel.alliance was fully compromised. Administrator TGT in memory, full NTDS dump on disk, krbtgt hash in hand. The objective was galactic.empire — the forest root. One trust relationship stood between here and there.

Act 1: The Cross-Trust Struggle

The plan looked clean on paper. The GALACTIC$ inter-realm trust key was sitting in the rebel NTDS dump. Forge an inter-realm TGT, exchange it for service tickets on CORUSCANT-DC, then DCSync.

The tickets came. NXC authenticated to CORUSCANT-DC. LDAP reported Pwn3d. Then:

🔥 DRSUAPI returned invalid_checksum — every single time. impacket-secretsdump reached the DRSUAPI endpoint, bound, and then died with an invalid checksum error. Different ccache files. Different service combinations. CIFS, LDAP, HTTP. All the same result.

🔥 LDAP writes returned insufficientAccessRights. NXC ldap -M whoami showed the bind resolving to galactic Administrator DN, with Domain Admins, Enterprise Admins, and Schema Admins group membership. Reads worked. Searches worked. But add-member against Domain Admins: denied. The forged PAC was passing authentication but not authorization.

The diagnosis was a known limitation: when cross-realm service authorization is checked against the local domain's security descriptor, a forged PAC from an external realm doesn't carry the right authorization data for DRSUAPI or LDAP write operations. The reads were allowed by a quirk; the writes were not.

More chaos piled on:

🔥 Kali MCP bridge was pointing at a dead IP. The bridge had been configured for [internal-network]. [internal-host] had been reassigned to [internal-network] (VLAN 30, DHCP) at some point. Every agent trying to use the bridge was silently failing. All attacker operations required direct SSH to [internal-network].

🔥 DVWA Docker container went down mid-session. VM500 at [internal-network] dropped. SSH was open but the dvwa container wasn't running. Restarted via sudo docker restart dvwa after connecting through the Proxmox guest agent.

🔥 Inter-realm ccache accidentally deleted. During a fresh TGT refresh sequence, the inter-realm ccache was removed from disk. A few seconds of horror, then immediate regeneration from the trust key. No permanent loss, but a good reminder that artifact hygiene matters under pressure.

Hours passed. ESC8 relay attempts as a detour. Each dead end reinforced the same wall.

Act 2: BloodHound Reveals the Path

BloodHound had been run against galactic.empire earlier. The data sat in `[internal-path] Now it was time to read it carefully.

The find: [email protected] was a member of the ACROSSTHENARROWSEA group. And ACROSSTHENARROWSEA had GenericAll over the CORUSCANT-DC$ computer object in galactic.empire.

GenericAll on a computer object. That's a Resource-Based Constrained Delegation (RBCD) path.

🏆 The RBCD chain:

1. Create ATTACKSVC$ machine account in galactic.empire using leia.organa credentials
2. Set msDS-AllowedToActOnBehalfOfOtherIdentity on CORUSCANT-DC$ → allow ATTACKSVC$
3. Run getST with S4U2Self + S4U2Proxy to obtain a service ticket as [email protected]
4. Use that ticket with secretsdump -just-dc-user krbtgt → DCSync CORUSCANT-DC

The chain executed cleanly. DCSync returned the galactic krbtgt hash. All domain hashes followed. NXC confirmed: Administrator — Pwn3d.

NTDS dump landed at [internal-path] on hacktop. Evidence preserved at evidence/attack/20260510T2200Z_skvs-da-rbcd/`.

galactic.empire: FULLY COMPROMISED — 2026-05-10.

Also completed that day: DS-WEAPONS-SRV SAM+LSA dump via xp_cmdshell running as sql_svc. The Kerberoast hashes for sql_svc had never cracked (wordlist miss), but LSA secrets gave up the plaintext directly. The Kerberoast path was unnecessary.

Aftermath

ATTACKSVC$ was left in galactic.empire — the machine account used for the RBCD attack. Queued for cleanup via impacket-addcomputer -delete. The cross-trust DRSUAPI invalid_checksum path remains an open, unsolved question. The docs, state, and context pack were updated. The day was done.

---

Phase 5 — The Knowledge Layer (2026-05-11)

The Audit Finding

A review of AI session logs revealed the problem: agents were spending significant time re-deriving state that was already known. Which hosts were compromised? What services ran where? What techniques had been tried? Each new session started from scratch. The solution was structural.

Graph-RAG Knowledge Layer (PR #1)

The 18-knowledge-graph/ directory was built from scratch:

• schema.yaml — entity type definitions (host, domain, service, technique, account, attack-path, vlan)
• 7 entity YAML files covering every lab component
• relations.yaml — 60+ typed relationship edges
• build-graph.py — exports graph.json for RAG pipelines
• graph-query.py — CLI queries: compromised, neighbours, techniques-for, services-on, attack-paths, path

This landed alongside a wave of new MCP servers:

• litellm-server.py — route_prompt, cost ledger, smoke test
• unifi-server.py — read-only UniFi inventory (ADR-0003: never write to UniFi)
• netbox-server.py — full IPAM: VMs, devices, VLANs, prefixes
• n8n-server.py — workflow trigger with safe-allowlist gate

The LM Studio MCP was expanded with tier-aware routing, a deepseek-r1 thinking mode for analysis tasks, a vault document summarizer, and a redaction wrapper. A tri-verify.py tool cross-validated Proxmox + NetBox + UniFi + vault graph with MATCH/IP_DRIFT/MISSING_IN_* verdicts — four sources that could drift independently, now automatically checked.

🤖 AI milestone: 20+ skills in a machine-readable registry. skills/REGISTRY.yaml became the AI discovery manifest — trigger phrases, model tier requirements, required MCPs, hard limits. Agents could now self-discover the right tool for any task without human guidance.

PR #1 merged — the largest structural commit in vault history. The lab had gone from a collection of scripts and docs to a Graph-RAG-enabled, MCP-orchestrated AI control plane.

The overnight n8n orchestration loop was designed: autonomous Red/Blue execution between 00:00 and 07:00. Not yet running — but designed, documented, and ready to activate.

---

Phase 6 — AI-Agnostic Bootstrap (2026-05-12)

The Fragmentation Problem

Claude had CLAUDE.md. Gemini had GEMINI.md. LM Studio sessions had their own prompt templates. Each tool had its own bootstrap path, and they were starting to drift.

🤔 Solution: AGENTS.md — one entry point, all tools. A single AI-agnostic bootstrap file. Every AI tool — Claude, Codex, Gemini, LM Studio routes — reads AGENTS.md first. Tool-specific config files become supplements, not entry points. Tagline: "Built by AI, for AI. Never assume; always verify, test, and check."

GitHub CLI credential source documented. A small commit, but an important one: future sessions would know exactly where gh auth credentials came from without deriving it.

---

Current State (2026-05-12)

The Scoreboard

Telemetry: Velociraptor + Sysmon 15.20 on all 5 VMs.

AI stack: LiteLLM + LM Studio ([internal-host]) + Claude + Gemini, 20+ skills, Graph-RAG knowledge layer, AGENTS.md unified bootstrap.

Open Threads

• deathstar.galactic.empire DA path — constrained delegation via darth.revan → THEWALL identified in BloodHound. Not yet exploited.
• ATTACKSVC$ cleanup — the machine account left in GALACTIC from the RBCD attack.
• Cross-trust DRSUAPI mystery — invalid_checksum with forged inter-realm TGT. Root cause not identified.
• UXG-Max migration — COMPLETE 2026-05-12. USG 3P replaced by UXG-Max. WireGuard evaluation now unblocked.
• n8n purple-team loop activation — designed, documented, waiting to be switched on.
• Sysmon field test — procedure written, not yet executed. Will create the blue-ready-* snapshot baseline.
• DS-COMMAND-DC EID 10 tuning — default Sysmon config still elevated on ProcessAccess.

What's Next

The lab is moving toward autonomous overnight operations. The n8n loop will orchestrate attack-detect-document cycles while the operator sleeps. AI agents will pick up from graph state, execute approved techniques, collect telemetry, and hand off to the next agent. deathstar.galactic.empire is the next target. The UXG-Max will eventually land and bring WireGuard VPN into the mix. The DRSUAPI mystery is a thread that will be pulled eventually.

This document will be updated as the story continues.

---

---

Update — 2026-05-12 — Gateway Migration Complete

The USG 3P was decommissioned and replaced by the UXG-Max as the active network gateway and VPN endpoint. The same IP ([internal-network]) and DDNS hostname (kryssar.servebeer.com) carry over. The Cloud Key Gen2 Plus remains the UniFi controller.

The UXG-Max was ordered in Phase 2 after the USG 3P showed its age: no WireGuard support, legacy firmware path, and limited throughput. The migration was always planned; it happened today.

WireGuard is now available on the UXG-Max. Evaluation and migration from L2TP/IPsec is the next VPN task — requires a dedicated plan and approval before execution.

All network and inventory docs updated. Vault cross-verify (tri-verify.py) requires API credentials in env to query live sources — vault-side references are updated.

---

Phase 7 — The AI Expansion (2026-05-13 to 2026-05-14)

Copilot MCP and the NAS Knowledge Pipeline

Two new capabilities landed in quick succession. First: a GitHub Copilot MCP wrapper (copilot-server.py) that exposes Copilot CLI to the agent orchestration layer. Policy-aware, with integration tests and a CI workflow. Second: a NAS knowledge ingest pipeline — nas-knowledge-scout.py, nas-knowledge-stage.py, and nas-knowledge-queue.py — enabling the lab to index PDFs, writeups, and reference material from the Synology into the Graph-RAG layer. The goal was to make the second brain actually learn from material stored on the NAS, not just from operator commits.

🤔 Decision: Copilot CLI adapter as MCP, not shell passthrough. Direct shell passthrough would bypass the policy layer. The MCP wrapper enforces the same scope gate and approval model as every other bridge. GitHub Copilot joining a table that already had Claude, Gemini, and LM Studio.

Hermes — The Night Operator

The most significant addition of this period was Hermes: a local AI agent (Hermes v0.13.0) wired to Qwen2.5-7B running on [internal-host]'s llama.cpp server (port 8081). Hermes was given a SOUL.md — a persistent identity and operating mandate — and a saga-purple-team skill. Its job: run unsupervised overnight attack-detect-document cycles while the operator slept.

The n8n orchestration layer was extended to support Hermes:

• n8n-hermes-router-workflow.py created the Hermes Router workflow
• n8n-hermes-overnight-workflow.py created the Blue-mode overnight cron (02:00 UTC) + status log (03:30 UTC)
• A manual webhook at /webhook/hermes-manual allowed on-demand invocation

🔥 Problem: Hermes health check false-negative. The /v1/models endpoint on llama-server returns a models key, not the data key the health checker expected. Hermes appeared offline even when running fine. Fix: updated the health check format to match llama-server's actual response schema. POST /webhook/hermes-manual confirmed HTTP 200.

HexStrike — The Attack Bridge

HexStrike MCP bridge (hexstrike-bridge.py) was deployed: 12 offensive technique wrappers over SSH-direct mode to hacktop. Features: scope gate (SAGA VMIDs 100–104 only), hash redaction before any response, dry_run=True by default. The kali-bridge was simultaneously rewritten from a subprocess executor to a proper SSH executor — more reliable, testable, and consistent with the HexStrike architecture.

Both bridges were hardened with dynamic IP resolution via MAC address (0c:54:15:0d:97:ec), using the UniFi Integration API so that [internal-host]'s IP doesn't need to be hardcoded even if DHCP reassigns it.

🤔 Decision: MAC-based dynamic IP resolution over static assignment. [internal-host] is physical hardware on VLAN 30. Static DHCP reservation was possible but added a manual step on every UniFi config change. MAC lookup via the UniFi API is one read-only call at bridge startup — zero maintenance overhead.

The kali-bridge was also extended with a --load flag in capability-query.py, enabling agents to auto-inject the matched skill's full instructions on task startup.

The LAB-MANUAL Rewrite

00-meta/LAB-MANUAL.md was fully rewritten to serve as a senior operator handover document. The old version was sparse and assumed too much context. The new version covers: MCP usage and routing, evidence workflow, n8n loop operation, and the full agent orchestration model. The SAGA rebuild specification was also formalized — a reproducible procedure for tearing down and redeploying the entire lab from scratch.

---

Phase 8 — Malcolm NDR and All Domains Fall (2026-05-15)

Malcolm NDR Goes Live

Malcolm (a multi-tool network traffic analysis platform built on OpenSearch, Zeek, Suricata, and Arkime) was deployed on [internal-host] and reached fully operational status on 2026-05-15. The path was not straightforward.

The [internal-host] was switched from WiFi (wlo1, DHCP) to wired Ethernet (enp2s0, [internal-network], reserved). Malcolm's packet capture was switched to enp2s0. NAS was remounted after the interface change. Route persistence was handled via /etc/netplan/60-lab-wired.yaml with metric 100 default and a static route for [internal-network] (VLAN 20).

Malcolm's Logstash pipeline required authentication (curlrc populated). After that was resolved: all 7 pipelines running, Filebeat→Logstash→OpenSearch confirmed. Kerberoasting TGS-REQ events (Event ID 4769, RC4-HMAC encryption) from SAGA Windows hosts were visible in arkime_sessions3-260515.

🔥 Problem: Arkime container health. Arkime started before OpenSearch reached cluster-green status. Container came up but couldn't connect to the index. Fix: patched config.ini with correct OpenSearch credentials, ran initarkime.sh to create the arkime_stats index, restarted container.

🤔 Decision: Blue stack on [internal-host], not a new VM (ADR-0001, confirmed). Malcolm's 26-container stack (OpenSearch, Logstash, Zeek, Suricata, Arkime, Dashboards, and more) runs on [internal-host]'s 30.5 GB RAM. Proxmox thin-pool headroom was already constrained. The [internal-host] decision from Phase 1 proved correct at scale.

Velociraptor → Malcolm Pipeline (VR→Malcolm)

With Malcolm live, the next problem was getting Windows telemetry into it. WEF (Windows Event Forwarding) would require a dedicated Windows collector. Instead, Velociraptor's EvtxHunter API was used to pull EVTX files from SAGA hosts, transform them to Zeek-compatible JSON, and drop them into Malcolm's Filebeat watch directory (~/lab/blue-team/malcolm/zeek-logs/current/).

vr-sysmon-ingest.py was written to execute this pipeline per host. The n8n Sysmon Defender Telemetry workflow (ID: C9tPlOVgs9d75CMU, cron 0 */6 * * *) was created to automate collection every 6 hours.

First collection results: 28,681 Sysmon events + 34,407 Security events indexed. 25 Kerberoasting EID 4769 RC4-HMAC events confirmed in malcolm_beats_winlog_*. The blue-team stack could now see what the red team was doing.

deathstar.galactic.empire Falls (AP-005)

The last unconquered domain fell on 2026-05-15. The attack path:

1. Kerberoasting against deathstar.galactic.empire SPNs from DS-WEAPONS-SRV foothold
2. PTH to DS-COMMAND-DC as Administrator
3. DCSync → 20 accounts dumped including krbtgt hash for deathstar.galactic.empire
4. Golden Ticket forged from krbtgt hash — unrestricted access to the entire deathstar domain

Evidence path: evidence/attack/2026-05-15_0858_attacker_north-dcsync.md. DS-COMMAND-DC updated to FULLY_COMPROMISED.

Scoreboard after 2026-05-15:

All three AD domains. All five SAGA VMs compromised or foothold. The primary attack campaign was complete.

🔥 Problem: ExtraSID inter-realm golden ticket (AP-004, ATTEMPTED_FAILED). The cross-trust golden ticket obtained CIFS/LDAP service tickets but DRSUAPI returned invalid_checksum. The PAC was signed with the REBEL krbtgt, which GALACTIC DRSUAPI rejected. A real-world PAC checksum constraint that the tool assumed away. Superseded by the RBCD path (AP-003), which succeeded where ExtraSID failed. Documented as a learning: know where the tool's assumptions break.

Docker MCP and the Self-Learning Ingest Pipeline

A Docker MCP server (docker-server.py) was added — read-only visibility into running containers, gated lifecycle operations for Malcolm containers specifically. Separately, a self-learning attack writeup ingest pipeline was built: n8n-attack-writeup-ingest-workflow.py + verifier with JSON parsing, enabling the lab to automatically ingest external writeups from the NAS into the Graph-RAG knowledge layer.

---

Phase 9 — The Automated Purple Team (2026-05-16)

n8n Purple-Team Loop Goes Live

The Purple-Team Loop — the lab's most ambitious automation — became operational on 2026-05-16. Design had been done in May 14; the hardening took the rest of the time.

The final architecture: a 14-node n8n workflow (ID: IQTS3mIRYIq8aQCF) running on n8n 2.19.5 (SQLite).

• Validate & Stage (Code node): validates the session payload, writes a session file via HTTP to a Node.js session-store-container running on 127.0.0.1:19876
• 6 static-command SSH nodes: source `[internal-path] and execute the attack step
• Approval Gate: pauses workflow mid-execution for operator review before proceeding
• Attack Referee (webhook): receives Velociraptor + Malcolm telemetry, scores detection coverage per technique
• Purple-Team Telegram Bot: multi-agent bot on [internal-host] ([internal-network]) — Claude, Gemini, Codex, Local (qwen3) — relays results and alerts to operator

🔥 Problem (1): n8n SSH node expression evaluation. n8n's SSH node wraps expression results in single quotes, breaking shell execution for any command containing variable expansion. Fix: static-command + session-store-container architecture. Commands stored in the session file, sourced at runtime — no expression evaluation inside SSH nodes.

🔥 Problem (2): session-store-container bind address. The Node.js session store was binding 127.0.0.1 only. n8n runs in Alpine Linux inside Docker; its localhost resolves to ::1, not 127.0.0.1. The HTTP write from n8n to the session store was silently failing. Fix: updated the bind address to :: (dual-stack). Both 127.0.0.1 and ::1 now work.

🔥 Problem (3): Attack Referee webhook not auto-registered after n8n restart. n8n 2.19.5 with SQLite has a bug where some workflows with Wait nodes don't activate on startup. Fix: n8n-post-restart.sh — a script that restarts the session-store-container and reactivates all four affected workflows.

Wazuh 4.9.2 Deployed

Wazuh 4.9.2 all-in-one (manager + indexer + dashboard) was deployed as [internal-vm] ([internal-network]). All 5 SAGA agents enrolled in the SAGA group. Dashboard: https://[internal-network].

Custom SAGA detection rules were written (11-detections/saga-wazuh-rules.xml), covering:

Rules deployed via Wazuh REST API (PUT /rules/files/custom_saga_rules.xml) after SSH key-auth to VM 504 was blocked. Manager restarted. Rules 100001–100050 verified loaded.

OpenBao Secrets Management (ADR-0019)

The *.env file era ended. OpenBao (HashiCorp Vault OSS fork) was deployed as a Docker container on [internal-host] with a KV v2 backend. Secrets migrated from lab_creds.env. vault-manager.py provides an env-fallback-compatible shim. The three active MCP bridges (kali-bridge, hexstrike-bridge, proxmox-server) were wired to read credentials from OpenBao.

🤔 Decision: OpenBao over HashiCorp Vault (ADR-0019). Vault changed its license to BSL in 2023. OpenBao is the community MPL-2.0 fork, API-compatible, actively maintained. Same tooling, no license friction.

Malcolm → n8n Alert Pipeline

The detection stack gained automated alerting. Four OpenSearch monitors were deployed via deploy-opensearch-alert.py:

1. Kerberoasting (RC4-HMAC TGS-REQ in Arkime sessions)
2. DCSync (EID 4662 in winlog index)
3. Pass-the-Hash (EID 4624 LogonType=9)
4. RBCD write (EID 5136 msDS-AllowedToActOn)

Each monitor fires to an n8n-saga-webhook destination. n8n workflow SAGA Alert Ingest (ID: 3rBfrSKUMFpH3pBb) receives the alert, writes a timestamped JSON to evidence/audit/, and POSTs to the Purple-Team Referee if a loop is active.

---

Phase 10 — Pro-Lab Expansion and Purple-Team Validation (2026-05-17)

ADCS ESC1: Certificate Template Abuse (AP-007)

The ADCS attack vector was exploited against ALDERAAN-CA-SRV (rebel.alliance certificate authority) using certipy-ad 5.0.4:

1. certipy find: ESC1 template identified on REBEL-CA — enrollee-supplied Subject Alternative Name (SAN) enabled, no manager approval required
2. certipy req: certificate issued with UPN [email protected] via SAN abuse
3. certipy auth: PKINIT TGT obtained, NT hash recovered via UnPAC-the-hash
4. Verified: Pwn3d! on both ALDERAAN-CA-SRV and ALDERAAN-DC via PtH

Evidence: evidence/attack/2026-05-17T1220Z_attacker_adcs-esc1. Walkthrough: 12-attack-walkthroughs/adcs-esc1-alderaan-ca-srv.md.

🔥 Detection gap exposed: ALDERAAN-CA-SRV has no WEF→Malcolm and no Wazuh agent. Zero winlog documents in Malcolm for ALDERAAN-CA-SRV. EID 4886 (Certificate Services received request) fired in Wazuh rule 100050 on ALDERAAN-DC but was unverified at time of execution. The certificate abuse was largely invisible to the blue stack.

Pro-Lab VMs: DEVBOX Jenkins and WINCLIENT

Three additional targets had been provisioned on VLAN 20 alongside the SAGA VMs:

• DEVBOX ([internal-vm], [internal-network], Ubuntu 24.04): Jenkins 2.555.1 in Docker on port 8080, SSH key-auth
• WINCLIENT ([internal-vm], [internal-network], Windows 10): workstation, member of galactic.empire
• DVWA ([internal-vm], [internal-network], Docker): previously compromised in AP-001

AP-006 — DEVBOX Jenkins Pivot (completed 2026-05-17):

1. Jenkins initialAdminPassword retrieved via Proxmox QEMU agent exec (no SSH credentials needed)
2. Jenkins admin authenticated, Groovy script console RCE confirmed as jenkins user in Docker container
3. Host enumeration: NTLM relay toolkit (Responder, ntlmrelayx.py) found on DEVBOX host — it is a dual attacker/target node
4. DS-WEAPONS-SRV MSSQL chain confirmed via Purple-Team Loop execution (xp_cmdshell as deathstar\sql_svc)

🔥 Finding: Jenkins in this SAGA build has no CI/CD jobs. The setup wizard was fresh. No deploy scripts, no SQL credentials in the Jenkins workspace. The sql_svc credential chain came from a prior LSA secrets dump on DS-WEAPONS-SRV (2026-05-10), not from DEVBOX.

🔥 Blue gap: DEVBOX has no Wazuh agent and no WEF→Malcolm. Jenkins Groovy RCE on DEVBOX is completely invisible to the blue stack.

AP-008 — WINCLIENT PtH (completed 2026-05-17):

1. Port scan: 445 filtered (Windows Firewall active)
2. Firewall disabled via Proxmox QEMU agent exec: netsh advfirewall set allprofiles state off
3. PtH using galactic Administrator NT hash via nxc
4. SAM dumped. AV active on host but did not block the PtH + remote exec chain.

All-SAGA VR→Malcolm

The Sysmon Defender Telemetry workflow was extended from DS-COMMAND-DC-only to all 5 SAGA hosts. vr-ingest-all-saga.sh was written to collect Sysmon (EID 1,3,10,11,12,13) and Security events (EID 4769,4624,4662,4768,4672,4776,4732,4728) from all five VR clients (CORUSCANT-DC, DS-COMMAND-DC, ALDERAAN-CA-SRV, DS-WEAPONS-SRV, ALDERAAN-DC) in a single cron-triggered run. Collection runs every 6 hours via n8n.

Purple-Team Loop: 8/8 Techniques Validated

The Purple-Team Loop ran all 8 techniques through the full attack-detect-score cycle:

8/8 techniques executed, scored, and documented.

n8n Night Router Remediation & Container Expansion (2026-05-22)

We remediated the critical integration and routing bugs in the n8n Night Router workflow (PG3AjJFbhF72XhMw):

1. Switch Routing Alignment: Explicitly mapped Switch rules to unique outputs (Offensive to port 0, Blue to port 1, etc.), preventing incorrect routing.
2. Approval Gate Webhook Decoupling: Replaced the failing executeWorkflow subworkflow call with a decoupled httpRequest node posting directly to the Approval Gate webhook (/webhook/approval-gate/request).
3. Container Expansion on [internal-host]: Enhanced the auto-attack-agent container by adding sqlmap to the Dockerfile and pre-baking passwordless SSH authentication (both operators and n8n local orchestration keys).
4. Validation Success: Fully verified both lanes end-to-end. The Blue Lane successfully parses intent and writes to morning-digest.md, and the Offensive Lane triggers the Approval Gate subworkflow (creating waiting execution ID 2815). Remote passwordless SSH verification executed sqlmap --version from [internal-host] in under 1 second.

🏆 Win: Decoupling the workflows through webhooks simplifies data mapping and avoids cross-workflow runner crashes. 🔧 Fix: Storing automation keys directly in ssh_keys/authorized_keys prevents mount overrides from blocking passwordless root execution.

---

Current State (2026-05-17)

The Scoreboard

Blue Team Stack

AI Orchestration Layer

Open Threads

• ALDERAAN-CA-SRV WEF→Malcolm: ALDERAAN-CA-SRV has no Windows event forwarding to Malcolm. Detection gap for ADCS attacks on the CA server. (Resolved 2026-05-24)
• DEVBOX Wazuh agent: DEVBOX has no agent — Jenkins RCE is blind to all blue-team tooling.
• deathstar.galactic.empire KG: attack-paths.yaml shows AP-006 status IN_PROGRESS for the Pro-Lab pivot — KG needs a rebuild to reflect AP-006 COMPLETED, AP-007, AP-008.
• WireGuard evaluation: UXG-Max supports WireGuard. Migration from L2TP/IPsec not started.
• ATTACKSVC$ cleanup verified: machine account removed post-RBCD (confirmed 2026-05-10).
• Malcolm→[internal-host] split: OpenSearch + Dashboards on [internal-host] (RTX 4060, Fedora) to reduce RAM pressure on [internal-host] — design doc pending.

Update — ALDERAAN-CA-SRV ADCS Telemetry Resolution (2026-05-24)

🔧 Fix: Resolved the ALDERAAN-CA-SRV telemetry blind spot for Active Directory Certificate Services (ADCS) CA auditing. Previously, despite GPO audit configurations, the Certificate Authority (REBEL-CA) properties did not have event auditing enabled, preventing Event IDs 4886, 4887, and 4888 from logging to the Security event log.

🏆 Win: Directly queried and corrected the CA Properties registry value AuditFilter to 127 (full CA event auditing) on alderaan-ca-srv ([internal-vm]) via Velociraptor's client execution bridge and restarted the CertSvc service.

🧪 Test: Verified Sysmon and Security log connection and ingestion using vr-sysmon-ingest.py. Direct OpenSearch queries on [internal-host] confirm that 73,441 documents are successfully indexed for alderaan-ca-srv.rebel.alliance, including Network Connections (EID 3), Registry Operations (EID 12/13), Process Creation (EID 1), and critical Security logon and Special Privilege events (EID 4624/4672). Telemetry is now fully flowing and visible in Malcolm NDR!

---

Update — Star Wars Transition Phase 1: Scaffolding & Configuration Bridge (2026-05-24)

🏆 Win: Fully completed Phase 1 of the Revanite Purple-Team Lab transition, replacing the default pre-SAGA schema in the SAGA estate with a high-fidelity Star Wars (KOTOR/SWTOR/Sith) themed directory template. All pre-SAGA domain references have been cleanly expunged in the configuration files, mapping galactic.empire to galactic.empire, deathstar.galactic.empire to deathstar.galactic.empire, and rebel.alliance to rebel.alliance.

🤖 AI milestone: Created the dynamic translation engine apply-starwars-overlay.py and the overlay schema config-starwars-overlay.json to handle automated configuration overlays in the SAGA setup, while ensuring critical internal Ansible host keys remain stable (dc01, dc02, dc03, srv02, srv03) to maintain upstream playbooks' integrity.

🔧 Fix: Implemented a rigorous phase gate validator validate-starwars-phase1.py that confirms config.json is structurally valid, references are clean, and no pre-SAGA naming remnants remain. It successfully passed with 14 active references, 0 missing files, and 0 overlay conflicts.

🏆 Win: Deployed the stateless "Star Forge" droid container pool on [internal-host] ([internal-network]) inside the isolated blue-team-net network. The custom-built containers for hk-47-assassin-droid (exposing command injection vulnerability) and t3-m4-utility-droid (exposing weak database API responses) are fully operational and verified, adding lightweight target surfaces to the cyber-range without extra VM overhead.

🧪 Test: Built verify-starwars-telemetry.py to retrieve SIEM tokens dynamically from OpenBao. End-to-end telemetry checks successfully queried both the Wazuh Manager REST API and the Velociraptor VQL endpoint via Docker, confirming that all 8 hosts (the 5 SAGA VMs + 3 Pro Lab VMs) are correctly spotted and tracked, paving the way for the active VM rebuild stage.

---

Update — Star Wars SAGA Active Rebuild: WinRM Root Cause Found and Fixed (2026-05-24)

🔧 Root cause eliminated: The QEMU/WinRM deploy failure that plagued multiple previous SAGA rebuilds was traced to a combination of three distinct issues — all resolved this session. (1) The WinServer2019 template ([internal-vm]) was built without WinRMSetupPlugin and WinRMListenerPlugin in cloudbase-init.conf, leaving Windows Firewall blocking WinRM from non-VLAN subnets on every fresh clone. (2) Ansible's default NTLM WinRM transport gives UAC-filtered (non-elevated) tokens to local admin accounts, preventing system-wide PowerShell module installation. (3) Two concurrent saga.py processes were spawned by separate agent sessions, causing file lock failures on shared PS module paths.

🔧 Fixes applied: (1) Updated Packer scripts (cloudbase-init.conf + cloudbase-init.ps1) to include WinRM plugins — new template rebuilds will be fully WinRM-ready from boot. (2) Uncommented ansible_winrm_transport=basic in globalsettings.ini — Basic auth bypasses UAC token filtering. (3) Pushed LocalAccountTokenFilterPolicy=1 and Enable-PSRemoting -Force to WinSrv2019 VMs (dc02/dc03/srv02) via Python winrm library using Basic auth. (4) Killed duplicate Ansible processes before clean restart. (5) Stored pre-provision credentials (vagrant/vagrant) in OpenBao at [secret-ref].

🏆 Win: First clean SAGA Ansible run with all 5 Star Wars VMs passing build.yml (failed=0 across dc01/dc02/dc03/srv02/srv03). ad-servers.yml also passed cleanly. Domain promotion (galactic.empire and rebel.alliance) now in progress. ADR-0023 written with pre-provision checklist to prevent recurrence.

---

Update — LM Studio VRAM Safety Hardening & Single-Active-Model Enforcement (2026-05-27)

🔧 Fix: Hardened the lmstudio-server.py MCP bridge to prevent resource contention and host freezes on the [internal-host] ([internal-network]). Previously, loading massive models concurrently would overflow the GPU's 16 GB VRAM capacity, causing the Windows system to freeze and crashing active agent offload workers.

🏆 Win: Implemented software-level VRAM allocation checks that dynamically estimate the footprint of GGUF model files and active KV-caches (large: 150KB/token, medium: 100KB/token, small: 60KB/token). The bridge now proactively aborts load requests with descriptive, graceful errors before hardware limits are breached.

🔧 Fixes applied: Enforced a strict single-active-model policy for large models and proactive "clean slate" model compaction for all medium/large models, resolving memory OOM conditions on tier switching. Built test_lmstudio_vram.py proving full memory bound validation and safety checks run correctly.

---

Phase 11 — The Star Wars SAGA Rebuild (2026-05-24)

Expunging Westeros

On May 24, 2026, the obsolete pre-SAGA naming convention was replaced by the Star Wars SAGA theme across the entire SAGA AD estate. galactic.empire became galactic.empire, rebel.alliance became rebel.alliance, and child domain deathstar.galactic.empire transitioned to deathstar.galactic.empire.

The Relabeled VM layout:

• CORUSCANT-DC (DC01 / [internal-network])
• DS-COMMAND-DC (DC02 / [internal-network])
• ALDERAAN-DC (DC03 / [internal-network])
• DS-WEAPONS-SRV (SRV02 MSSQL / [internal-network])
• ALDERAAN-CA-SRV (SRV03 ADCS / [internal-network])

Resolving the WinRM Provisioning Blockers

Previous rebuild attempts suffered from WinRM failures due to packer base template listener gaps, UAC privilege filtering on local administrator accounts over NTLM, and Ansible process collisions under concurrent agent sessions.

🔧 Fixes: Added WinRM configuration triggers to packer configs, switched remote transport to Basic authentication to bypass local account token filtering, pushed LocalAccountTokenFilterPolicy=1 dynamically via Python winrm, and enforced strict process lock validation.

The result: a clean Ansible execution across all five VMs (failed=0). Active domains were promoted. A baseline snapshot freeze (BLUE-READY-BASELINE) was established across all eight range VMs, securing a deterministic lab reset capability.

---

Phase 12 — The Malcolm Resource Split (2026-05-31)

Saturated Memory Control Plane

By late May, the primary control plane [internal-host] ([internal-network]) hit critical memory saturation, with 30.54 GiB RAM completely used and heavy swap thrashing (7.1 GiB swap consumed). The primary culprit was the monolithic Malcolm NDR stack, consuming over 11 GiB of RAM (OpenSearch ~5.1 GiB, Logstash ~4.2 GiB).

🤔 Decision: Split Capture and Storage. We split Malcolm into a distributed Capture-Storage topology (ADR-0021). [internal-host] retains lightweight capture services (pcap-capture, zeek-live, suricata-live, and arkime viewer) for direct interface binding. JVM-heavy storage/analytics services (opensearch, logstash, dashboards, and postgres) were offloaded to the secondary idle AI/GPU node [internal-host] ([internal-network]), which possessed 26 GiB of free RAM.

Distributed Ingestion & Knowledge Sync

Routing was wired over the local management LAN:

• filebeat ([internal-host]) pushes packets to [internal-network] (Logstash).
• arkime viewer ([internal-host]) queries [internal-network] (OpenSearch).
• nginx-proxy ([internal-host]) forwards UI calls to [internal-network] (Dashboards).

This split reclaimed ~10 GiB of RAM on [internal-host], stabilizing the AI control plane and improving search indexing speeds. In parallel, new attack surfaces (hacklab_cloud and bugbounty_agent) were mapped into the Knowledge Graph database to synchronize public and internal knowledge stores.

---

---

Phase 13 — ICM Consolidation & Documentation Infrastructure (2026-06-04)

The Drift Problem

After Phase 12 the repo had accumulated significant structural debt: the agent load order was defined in seven different files with seven different variants, safety constraints were paraphrased in six places (each differently), five orphaned root Markdown files duplicated state tracked elsewhere, and 886 generated build artifacts were checked into git. An external audit surfaced the core problem: every agent file was copy-pasting safety rules, vault maps, and infrastructure details. When anything changed, 5–7 files needed updating — and they never all got updated.

The ICM principle applied: each layer owns its facts. No fact stated in more than one place. Agents load only what they need.

The Consolidation

AGENTS.md became the sole Layer 0 authority. All five agent shims — CLAUDE.md, GEMINI.md, CODEX.md, HERMES.md, ANTIGRAVITY.md — were slimmed from 100–300 lines each to 43–79 lines, carrying only agent identity, model strategy, and a single pointer to AGENTS.md. Orphaned root files (progress.md, INDEX.md, 00-HOME.md, skill-to-develop.md, SKILLS.md) were archived to 00-meta/archive/. The hermes-dashboard.sh script moved to scripts/. A scripts/ directory formalised what had been scattered root-level tooling.

The vault structure table in AGENTS.md was updated to reflect the ICM folder hierarchy (01-agent-engine/, 02-state/, 03-deployments/, 04-scratchpad/, etc.) — the first time a single file correctly described the full repo layout since the Folders-Over-Agents restructure.

Site Infrastructure Fixes

A consistency audit across the three site surfaces (homelab-experience, README, kryssar.se) found kryssar.se/about returning an empty 39-byte HTML page. The root cause: sync-sites.py was replacing internal NFS paths with [internal-path] but leaving internal file URI] as a live Markdown link href. Go's URL parser treats [...] as an IPv6 address literal — the Hugo build crashed silently on every run, deploying a zero-byte file to Loopia. A targeted regex fix (_INTERNAL_FILE_LINK_RE) stripped the link wrapper before path substitution, the site rebuilt cleanly, and the about page went live at 15K chars.

Mermaid Overhaul

All eight mermaid diagrams across both Hugo sites were broken or missing:

• The graph TB topology in README.md crashed with "Syntax error in text — mermaid version 11.15.0". The graph keyword was deprecated in mermaid v11 in favour of flowchart. Fixed to flowchart TB.
• The Services & AI Infrastructure diagram (block 3 of lab-topology-diagram.md) had three independent parse failures: bullet character • in a node label, --|label| arrow syntax (missing >), and an edge targeting a subgraph container as a node.
• homelab-experience had no topology page at all — /topology/ returned nginx 404.

The fixes were structural: sync-sites.py gained a sync_topology() function that auto-syncs 02-network/lab-topology-diagram.md to both sites on every run (IPs preserved for internal, sanitised for external). The homelab-experience Hugo config gained a "Lab Topology" nav item. Result: 8/8 mermaid diagrams rendering cleanly across both sites.

State

The repo ended this session with a clean git history, all validators passing (ICM 0 errors, curator OK, content drift PASS), and consistent documentation across README, homelab-experience, and kryssar.se. No offensive action, live infrastructure mutation, or credential access occurred.

---

Phase 14 — SAGA Deploy Automation & Template Hardening (2026-06-04)

The Automation Gap

After the Star Wars naming migration and Malcolm resource split, the lab had a clean thematic identity and a working blue team stack — but no reliable automated deploy pipeline. Every fresh SAGA provision required manual WinRM bootstrapping, Ansible ran with default 5 forks, failed playbooks restarted from scratch, and Proxmox template 201900 still lacked WinRM prerequisites baked in. A single deploy could fail midway through a 45-minute DC promotion with no snapshot to roll back to.

SAGA as a Standalone Product

The session began with a standing audit: the older saga-* deploy tools had been renamed to saga-* wrappers, but the SAGA deploy automation was still scattered across the homelab repo with no clear product boundary. A standalone SAGA/ directory was formalised as a product boundary — owning tools, scripts, configs, manifests, overlays, lab-state defaults, and tests. The SAGA/ tree now rejects all legacy naming and defaults to SAGA paths; legacy repo entrypoints delegate into SAGA/ without owning the implementation.

Three parallel research agents investigated best practices across Windows VM provisioning (Packer, WinRM reliability, Ansible Windows modules), Linux/container deployment (cloud-init, Docker Compose patterns, service readiness), and IaC pipeline design (preflight gates, idempotent stages, rollback patterns). Their findings drove six concrete improvements implemented the same session.

The Six Improvements

Concurrent host probing replaced the sequential for host in target_hosts: probe_host(...) loop in deploy-runner.py with ThreadPoolExecutor. For five hosts each requiring ping + TCP + WinRM probes, worst-case preflight dropped from ~100s sequential to ~20s concurrent.

Stage checkpoint + --resume wired a SAGA/run-checkpoint.json that records each successfully completed Ansible playbook. A failed run can be resumed from the last successful stage — eliminating the need to re-run a 45-minute DC promotion just because a later user-data playbook failed. provision-sequence.sh forwards --resume through the pipeline.

Snapshot-bracket with auto-rollback (R6) wired SAGA/tools/snapshot.py against the Proxmox host via SSH. Before the first playbook, all target VMIDs are snapshotted to pre-SAGA-{ts}. On any Ansible non-zero exit, the runner automatically rolls back all VMs to that snapshot. --rollback-to SNAPNAME was exposed as a standalone operator subcommand. Old bracket snapshots are pruned to a configurable keep limit (default 2).

SAGA ansible.cfg (forks=8, timeout=30, pipelining=true) is applied to every ansible-playbook invocation via ANSIBLE_CONFIG — no system-level config changes required.

Packer templates (R7) replaced the unmaintained WinServer2019 and WinServer2016 templates on Proxmox hacklab. New [internal-vm] 201901 (WinServer2019x64-saga) and [internal-vm] 201601 (WinServer2016x64-saga) were built via SAGA/packer/windows_server2019.pkr.hcl using packer-plugin-proxmox (proxmox-iso builder). Both templates bake:

• cloudbase-init with WinRMSetupPlugin + WinRMListenerPlugin
• winrm-harden.ps1 (sets LocalAccountTokenFilterPolicy=1; copies itself to cloudbase-init LocalScripts so it re-runs on every fresh clone first boot)
• Sysmon64 v15.20 at C:\Tools\Sysmon, Automatic startup

Two build quirks were discovered and fixed during live runs: Sysmon64 kernel driver installation over non-interactive WinRM exits 0xC0000409 even on success (added to valid_exit_codes); WinServer2016 autounattend enables WinRM in WinPE ~2 minutes into boot, causing a spurious EOF on the first setup reboot (boot_wait = "4m" skips past it for 2016 builds).

The legacy templates 201900 and 201600 were retained — VMs 101/102/104 are qcow2 linked clones of 201900, and 100/103 are linked clones of 201600. Deleting either would corrupt those VMs' backing disks.

JSONL Ansible callback (R8) added SAGA/callback_plugins/saga_jsonl.py — a notification-type callback that runs alongside the yaml stdout callback. Every task result writes one structured JSON line to SAGA/logs/deploy-{ts}-{id}.jsonl. A compact progress line ([HH:MM:SS] OK/FAILED stage=... host=... task=... (Xs)) is emitted to stdout simultaneously. Logs are tail-able via tail -f SAGA/logs/deploy-*.jsonl | jq . from n8n or any TUI, giving real-time observability into multi-host provisioning runs without parsing ANSI escape sequences.

State

The session closed with all six improvements committed and pushed, both SAGA-hardened templates live on [internal-host], SAGA/config/lab-state.yaml updated with the full template registry and per-VM template_vmid tracking, and all SAGA naming tests passing (4/4). The deploy pipeline is now faster, safer, and observable. The remaining work is operational: a planned maintenance window to stop VMs 100–104, flatten their qcow2 chains, and migrate them to the hardened 201901/201601 templates.

---

Phase 15 — ICM Router Restructure & Floor-Maps (2026-06-17)

From Lobby to Map

Phase 13 had collapsed the agent bootstrap into a single AGENTS.md authority and slimmed the five per-LLM shims. It was the right first move, but AGENTS.md had quietly grown back to 322 lines — a lobby trying to also be the map, the reference shelf, and the rulebook. The ICM (Information Cartography Method) principle says an agent should always know three things: the map (where), the mission (what), and the support (which tools). One bloated entry file answers none of them cleanly. This session pushed the full ICM model into the repo: a thin lobby, a routing map, per-folder floor-maps, and a reference shelf — each fact living in exactly one place.

The Five Tiers

The restructure landed as ten commits on an icm-restructure branch:

• Lobby — AGENTS.md slimmed 322 → ~80 lines: tagline, mandatory load order, vault-structure table, hard limits. Nothing an agent doesn't need on the first read.
• Map — a new root ROUTER.md gives a task→location table ("I want to… / Go to / Gate before acting"). Find your intent, jump to the destination, honour the gate.
• Floor-maps — _ROUTER.md files in 16-ai-bridge/, 15-ai-agents/, 17-state/, and 03-deployments/. Each scopes its folder then redirects: "for anything not covered here, return to root ROUTER.md." The deployments floor-map became the single owner of the full Hugo edit→sync→build→publish workflow.
• Reference shelf — sixteen how-to files under 00-meta/reference/ (python-env, model-routing, session-lifecycle, reflection, frontmatter, fresh-clone, and more), extracted verbatim from the pre-slim AGENTS.md, plus an index.
• Agent stubs — the five per-LLM files reduced to thin stubs: identity + agent-specific quirks + "Read AGENTS.md first, then ROUTER.md."

Protecting the Root

The reason floor-maps are named _ROUTER.md and not AGENTS.md is not cosmetic. Repo-root detection walks up the tree looking for sentinel files, and several tools anchored on the AGENTS.md + NEXT-ACTIONS.md pair. Naming subfolder maps AGENTS.md would have minted false roots and broken every tool's path resolution. The fix was a dedicated .icm-root sentinel at the repo root, with the finder preferring .icm-root first and the file pair as fallback — hardened and proven by test_icm_root_finder (3/3).

A Map That Checks Itself

A map that lies is worse than no map. icm-layer-check.py gained validate_routers(): it parses every backticked path in ROUTER.md and each _ROUTER.md and flags any that do not resolve on disk. The grep gate it enabled caught real dangling references — generated filenames, generic mentions, a bare sync-sites.py — which were de-backticked or pinned to full paths. One self-inflicted regression surfaced and was fixed: emitting an informational OK result made the text verdict print WARN on a clean repo, because the verdict logic had only FAIL/WARN cases and leaned on emptiness for PASS. The ternary was completed (FAIL / WARN / PASS) so text output matches the JSON status. Final state: validator PASS, 0 errors, 0 warnings, 6/6 ICM tests green.

Closeout

The branch merged to main with all checks passing, then the two unrelated in-flight tool changes were committed alongside: n8n-workflow-health.py and reflection-runner.py gained expected-active workflow metadata (so approval-gated and external workflows are no longer flagged as failures) and read-only LM Studio model-status reporting (reflection no longer warms or loads models). Both sites were rebuilt and published via the publish-all.sh pipeline — internal Hugo (860 pages) restarted, external pushed to Loopia (730 files), verify-sites both PASS. Three learnings (L-046–048) captured the floor-map naming rule, the verdict-ternary mistake, and an importlib sys.modules registration gotcha for dynamically loaded hyphenated test modules.

State

The repo ended this session with the full ICM five-tier structure live on main, all validators passing, both sites published and verified, the learning journal current through L-048, and every step audit-logged. No offensive action, live infrastructure mutation, or credential access occurred.

---

Last updated: 2026-06-17 | Maintained by: kryssar Update trigger: end of any significant session, new phase milestone, or infrastructure change