Canonical source:
ARCHITECTURE.md~6 min readArchitecture & Scaling Invariants
mimikyu is a native Rust agentic browser runtime, two-tier isolation layer, and CLI / Daemon / MCP control plane driving local Chromium (CDP) and Firefox (W3C WebDriver BiDi) browsers.
This document maps the workspace dependency graph and codifies the five non-negotiable invariants that keep mimikyu fast, deterministic, and easy to review as external pull requests scale.
Workspace & Module Topology
mimikyu is split into two Rust crates (edition = "2024", MSRV 1.88) and an opaque-box Python 3 stdlib end-to-end verification suite (e2e/):
crates/mimikyu-core: Pure library crate containing browser discovery, process-group lifecycle, WebSocket CDP & BiDi protocol drivers, two-tier profile/context isolation, coherent environment profiles &ProxyRelay, semantic@eNaccessibility snapshots, structured extraction,ChokepointInput, deterministic waits, the warm Unix-socket daemon, the 31-tool stdio MCP server, the dual-origin local HTTP test harness, and the benchmark suite.crates/mimikyu-cli: Thin binary crate (mimikyu) providingclapCLI argument parsing,--jsonenvelope formatting, auto-spawning daemon client dispatch, and frontends formcp,daemon,doctor, andbench.e2e/: Zero-dependency Python 3 stdlib test suite exercising the compiledmimikyubinary across local127.0.0.1:0HTTP/proxy servers (Tiers 1–4 +--self-test).
flowchart TD
CLI["mimikyu-cli (cli, client, batch, output, commands)"] --> Daemon["daemon (Unix-socket NDJSON server & DaemonPool)"]
CLI --> MCP["mcp (Stdio JSON-RPC 2.0 server, 31 tools)"]
CLI --> Bench["bench (Speed & 5-workflow agentic suite + RSS)"]
CLI --> Session["session (Unified SessionManager)"]
Daemon --> Session
MCP --> Session
Bench --> Session
Bench --> Harness["harness (Embedded 127.0.0.1:0 dual-origin server)"]
Session --> Isolation["isolation (ProfileStore, fs2 ProfileLock, VirtualContextPool)"]
Session --> EnvProfile["env_profile (Presets, GeoIP, Coherence, ProxyRelay, Scripts)"]
Session --> Snapshot["snapshot (AxTree distillation, @eN RefMap, Structural Diff)"]
Session --> Input["input (ChokepointInput, Humanize kinematics, ActionBatcher)"]
Session --> Extract["extract (Markdown, Text, Links, Tables, Screenshot, PDF)"]
Session --> Wait["wait (Idle lifecycle/network/layout, Text, URL, Selector)"]
Session --> Network["network (BlockPolicy, RouteRule interception, Ring telemetry)"]
Isolation --> Driver["driver (BrowserDriver & DriverSession traits)"]
Snapshot --> Driver
Input --> Driver
Extract --> Driver
Wait --> Driver
Network --> Driver
EnvProfile --> Driver
Driver --> CDP["driver::cdp (Chromium DevTools Protocol over WS)"]
Driver --> BiDi["driver::bidi (Firefox W3C WebDriver BiDi over WS)"]
CDP --> Process["process (ProcessGuard, setpgid, deterministic reaping)"]
BiDi --> Process
Process --> Discovery["discovery (Local binary probe & zero-spawn macOS Firefox)"]
Module Directory Reference
| Module Path | Responsibility |
|---|---|
crates/mimikyu-core/src/error.rs |
Typed MimikyuError & Result<T> error taxonomy across transport, protocol, timeout, and config faults |
crates/mimikyu-core/src/discovery.rs |
Local Chromium & Firefox discovery (Override -> EnvOverride -> Playwright/Puppeteer caches -> System/PATH) + run_doctor() |
crates/mimikyu-core/src/process.rs |
ProcessGuard launching debloated browsers in dedicated POSIX process groups (setpgid(0, 0)) with crash-dialog suppression |
crates/mimikyu-core/src/driver/mod.rs |
Engine-agnostic BrowserDriver & DriverSession traits, normalized ProxyConfig, protocol JSON surrogate sanitizer |
crates/mimikyu-core/src/driver/cdp.rs |
Native async WebSocket Chromium DevTools Protocol driver (isolated worlds, OOPIF/Shadow DOM traversal, network fetch routing) |
crates/mimikyu-core/src/driver/bidi.rs |
Native async WebSocket W3C WebDriver BiDi driver for Firefox/Gecko (user contexts, sandbox realms, network intercepts) |
crates/mimikyu-core/src/isolation/ |
Tier-1 ProfileStore / ProfileLock (fs2) / EphemeralDir and Tier-2 VirtualContextPool + cookie/storage JSON import/export |
crates/mimikyu-core/src/env_profile/ |
win11 / macos / linux presets, coherence validator, offline GeoIP lookup, loopback ProxyRelay, window & worker init scripts |
crates/mimikyu-core/src/network/ |
Resource blocking (images, media, fonts), glob/regex URL interception & mocking, bounded ring-buffer console/network logs |
crates/mimikyu-core/src/snapshot/ |
Compact semantic accessibility tree builder, deterministic @eN RefMap locator resolution, and structural snapshot diffs |
crates/mimikyu-core/src/input/ |
Per-session mutex-serialized ChokepointInput, viewport-clamped humanize trajectories/cadence, and atomic ActionBatcher |
crates/mimikyu-core/src/extract/ |
Structured extraction (markdown, text, links, tables), viewport/full-page/element PNG screenshots, PDF, instant animations |
crates/mimikyu-core/src/wait/ |
Composite --idle wait (DOM ready + network quiet + layout stability) plus --text, --url, and --selector --state polling |
crates/mimikyu-core/src/session.rs |
SessionManager unifying lifecycle, isolation modes, @eN state, ChokepointInput, ProxyRelay, and telemetry per session ID |
crates/mimikyu-core/src/daemon/ |
Warm Unix-socket NDJSON RPC server (DaemonServer, DaemonPool), idle expiration, lock-safe auto-spawn, short-socket symlinking |
crates/mimikyu-core/src/mcp/ |
Stateful stdio JSON-RPC 2.0 MCP server (2024-11-05) exposing 11 mimikyu_* and 20 browser_* tools |
crates/mimikyu-core/src/harness/ |
Offline 127.0.0.1:0 primary + cross-origin HTTP test server powering integration tests and reproducible benchmarks |
crates/mimikyu-core/src/bench/ |
Reproducible latency, token-compression, process-tree RSS, and 5-workflow agentic benchmark runner |
crates/mimikyu-cli/src/ |
clap CLI definition (cli.rs), daemon auto-connect (client.rs), --json output envelope (output.rs), command handlers (commands/) |
Non-Negotiable Architectural Invariants
Every pull request—whether authored by core maintainers, external contributors, or coding agents—is reviewed against these five invariants.
Invariant 1: Strict DriverSession Dual-Engine Boundary (driver/cdp.rs + driver/bidi.rs)
- Rule:
BrowserDriverandDriverSession(crates/mimikyu-core/src/driver/mod.rs) are the only boundary between wire protocols and higher-level subsystems. - No upward protocol leaks: Raw CDP method names (
Page.*,DOM.*,Runtime.*), BiDi realm/context internals (browsingContext.*,script.*), or engine-specific JSON shapes must never appear insession.rs,snapshot/,input/,extract/,wait/,daemon/,mcp/, orcrates/mimikyu-cli/. - Parity or explicit capability gating: New browser primitives must implement both
CdpDriverandBidiDriverbehindDriverSession, or explicitly declare support inDriverCapabilitiesand returnMimikyuError::Unsupported { engine, feature }when an engine genuinely lacks a protocol capability.
Invariant 2: ChokepointInput Single Input Serialization Boundary (input/chokepoint.rs)
- Rule: All user-simulating interactions—
click,fill,type,press,scroll,select,upload, and multi-stepActionBatchersequences (input/batch.rs)—must execute exclusively throughChokepointInput. - Why: Concurrent MCP/daemon calls dispatching raw
dispatch_mouse,dispatch_key, ordispatch_wheelevents directly onDriverSessioninterleave modifier chords, corrupt humanized pointer trajectories, and hang on unacknowledged frames. - Guarantees enforced at the chokepoint:
- Per-session
tokio::sync::Mutexserialization so multi-event gestures (move->down->up, orCtrl+A->Backspace->insert_text) are atomic. - Bounded per-step ACK timeouts (
DEFAULT_INPUT_ACK_TIMEOUT_MS = 5_000). - Viewport-clamped coordinates (
clamp_point_to_viewport, accounting for scrollbar gutters) so out-of-bounds clicks never silently drop.
- Per-session
Invariant 3: Two-Tier Isolation & Zero-Orphan Process Groups (isolation/, process.rs)
- Rule: Every browser process is owned by a
ProcessGuard(crates/mimikyu-core/src/process.rs) placed into its own POSIX process group (libc::setpgid(0, 0)insidepre_exec) beforeexec. - Deterministic teardown: Shutdown always executes in order:
- Graceful protocol close (
Browser.closeon CDP /browser.closeon BiDi) within a bounded grace window. - Process-group
SIGTERM(kill(-pgid, SIGTERM)). - Process-group
SIGKILL(kill(-pgid, SIGKILL)) +wait_for_pgid_gone, followed by releasingProfileLock(fs2) and deleting anyEphemeralDir.
- Graceful protocol close (
- Test requirement: Every test binary and E2E run must finish with zero orphan
mimikyu, Chromium, or Firefox helper processes and zero leaked/tmpprofile directories, including on error or panic paths (Drop).
Invariant 4: macOS Zero-Spawn Firefox Probe & Crash-Reporter Suppression (discovery.rs, process.rs)
- Rule: On macOS
.appbundles, Firefox version discovery (crates/mimikyu-core/src/discovery.rs) must readContents/Resources/application.iniandContents/Info.plistdirectly from disk rather than spawningfirefox --versionrepeatedly. - Why: Cold-spawning
firefox --versionon macOS exits in ~50 ms while Firefox's backgroundRegisterFontsthread is still initializing, triggering a race inside__cxa_finalize_ranges(SIGSEGV) and popping macOSCrashReporterdialogs on the user's desktop. - Companion rules:
- Every Firefox process spawn or fallback probe must set
MOZ_CRASHREPORTER_DISABLE=1,MOZ_CRASHREPORTER=0, andMOZ_CRASHREPORTER_NO_REPORT=1. - Chromium version probes must pass
--versiononly—never--help(which hangschrome-headless-shellindefinitely).
- Every Firefox process spawn or fallback probe must set
Invariant 5: Narrow Vertical PR Slices & Three-Tier Offline Verification
- Narrow PR scope (2–3 modules max): Keep each PR focused on a single vertical slice or bug fix touching at most 2–3 modules (plus its dedicated tests). Split trait additions, refactors, and unrelated CLI polish into separate PRs so reviews stay fast and conflict-free.
- 100% offline loopback testing (
127.0.0.1:0): Automated unit, integration, benchmark, and E2E tests must never make live requests to external internet domains (example.com,google.com, etc.). Usecrates/mimikyu-core/src/harness/in Rust tests ande2e/local_server.pyin Python E2E tests. - Required verification tiers:
- Lint & Format:
cargo fmt --all -- --checkandcargo clippy --workspace --all-targets -- -D warnings(0 warnings). - Rust Workspace Tests:
cargo test --workspace(unit + live local harness integration + CLI contract tests). - Opaque-Box E2E Parity:
python3 e2e/run_e2e.py --self-testpython3 e2e/run_e2e.py --tier all --bin ./target/release/mimikyupython3 e2e/run_e2e.py --tier 4 --engine firefox --bin ./target/release/mimikyu
- Lint & Format:
Security & R3 Scope Invariant
Across all modules and tests:
- Default isolated world:
evalexecutes in an isolated world unless--main-world(EvalOptions { main_world: true, .. }) is explicitly requested. - Redacted proxy credentials: Authenticated proxies always route through
ProxyRelayon127.0.0.1:0. - Strict R3 exclusion:
mimikyunever implements, bundles, or tests CAPTCHA solvers, human-verification challenge bypasses, or proof-of-work crackers. See SECURITY.md for full details.