ctOS // Node // SYS_NODE_NETPLAY

01. Executive Summary

An ultra-low-latency, distributed network netplay processing engine built as a Cargo workspace with two core components: contra_core (Rust emulation engine) and contra_ui (Flutter cross-platform UI). This system coordinates high-frequency cross-platform NES emulation frames between isolated desktop and mobile operating environments, linked via flutter_rust_bridge FFI.

The engine features cycle-accurate emulation built on tetanes-core, a custom predictive RollbackManager to synchronize player inputs without gameplay stutters, and direct encrypted QUIC-based P2P networking tunnels orchestrated by the Iroh protocol. The system includes integrated low-latency VoIP communications using the Opus codec.

02. Problem Statement

Real-time peer-to-peer emulation requires synchronizing game states at a locked 60Hz (16.6ms per frame).

Network Jitter & Latency: Standard TCP networking introduces packet delivery delays. Even a 50ms lag spike creates noticeable input stutter in lock-step synchronization.
Heavy OS-Thread Contention: Using standard OS-level mutexes to share frame buffers between the rendering UI (Flutter) and the emulation runner (Rust) causes thread blockages and drops frame rates below 60 FPS.
NAT Traversal Obstacles: Mobile-to-mobile P2P connections are frequently blocked by carrier firewalls (Symmetric NATs), preventing direct socket binding.

The system required a predictive network engine capable of executing inputs instantly, running rollback recovery routines within the 16.6ms frame budget, and bypassing NAT blockages without central game servers.

03. System Architecture

The project is structured as a cargo workspace containing the Rust core FFI library (contra_core) and the Flutter frontend wrapper (contra_ui).

    [ Flutter UI Render Loop (60Hz) ]
                   │
    (Input Events) │ (zero-copy swap) [ Frame Buffer (RGBA) ]
                   ▼
  ════════════ FFI Boundary (C-ABI) ════════════
    [ contra_core API (flutter_rust_bridge) ]
                   │
                   ▼
  [ RollbackManager / RollbackEmulator ]
    (AtomicU16 Inputs / Frame Records)
        │                       │
        ▼ (State Deserialise)   ▼ (Wire serialization)
  [ tetanes-core Deck ]     [ Packet Serializer ]
  (6502 CPU / PPU / APU)      (10-byte Bincode)
                                │
                                ▼
                       [ Tokio Async Task ]
                      [ Iroh P2P Transport ]
                                │
                          (QUIC Tunnel)
                                ▼
                       [ Remote Peer ]

Module Architecture Map (11 Core Modules)

Module	Location	Functional Responsibility
`api`	`src/api.rs`	Public interface layer exposing FFI hooks and control runners.
`emulator`	`src/emulator.rs`	Wrapper deck for `tetanes_core::control_deck` managing snapshot states.
`clock`	`src/clock.rs`	Frame timing coordinator (60Hz NTSC / 50Hz PAL).
`input_system`	`src/input_system.rs`	Compresses active gamepad button combinations into a single 16-bit bitmask.
`ffi`	`src/ffi.rs`	Low-level C-ABI Foreign Function Interface definitions.
`network`	`src/network.rs`	P2P peer discovery, room coordinator, and background event router.
`packet`	`src/packet.rs`	Compact 10-byte P2P packet serialization structure.
`rollback`	`src/rollback.rs`	History frame tracking, snapshot storage, and prediction replayers.
`transport`	`src/transport/`	Separation of `iroh_native` (sockets) and target `wasm` networking interfaces.
`voice`	`src/voice.rs`	In-process Opus audio encoding, capturing, and streaming node.
`bridge_generated`	`src/bridge_generated.rs`	Auto-generated type conversion code compiled by `flutter_rust_bridge`.

04. Engineering Decisions

Iroh over WebRTC for P2P Networking

WebSockets and WebRTC require complex stun/turn server configurations and heavy connection negotiations. Iroh P2P was chosen because it wraps QUIC networking directly:

Bypasses firewalls using automatic NAT traversal (hole punching) and fallback relay servers.
Cryptographically secures tunnels natively via TLS-ring certificates.
Operates on direct raw socket streams without server handshake brokers.

Tetanes-Core for Emulator Core

Rather than building an emulator clone from scratch, we integrated tetanes-core v0.14.2. This cycle-accurate emulator implements:

The full NES 6502 instruction set (including unofficial opcodes).
Pixel-accurate Picture Processing Unit (PPU) rendering.
Audio Processing Unit (APU) synthesis for standard channels. This choice let us focus our engineering on the rollback sync and low-latency transport layer.

05. Scaling Strategy

Cross-Compilation to WebAssembly

To support web browsers, the transport layer is split.

On desktop and mobile, transport::iroh_native uses standard async Tokio thread pools.
On the web, transport::wasm compiles to WASM32 and utilizes the browser’s JavaScript environment (getrandom with wasm_js) to handle inputs, allowing deployment on standard static web servers.

Decoupled VoIP Stack

To prevent high-bandwidth audio from competing with critical frame sync packets, the system separates voice data:

Frame Inputs: Sent over unreliable P2P datagrams for minimum latency.
Voice Data: Sent via low-priority channels using the Opus codec (mono 16kHz, 20ms chunks) to prevent audio packet bursts from delaying input frames.

06. Performance Optimizations

Lock-Free Frame Buffers

To prevent thread contention between the Flutter UI loop and the Rust emulation engine, the system utilizes lock-free atomic swaps. Instead of blocking the UI thread with mutexes while Rust renders a frame:

Rust renders the PPU frame into a background buffer.
An atomic pointer swap (AtomicU16 for input vectors, and raw FFI pointers for buffers) updates the active display frame in under 2 microseconds.
Flutter accesses the buffer pointer directly over the FFI boundary, achieving a stable 60 FPS.

Cargo Workspace Optimization

To maintain emulation speeds during development testing, the Cargo.toml overrides dependencies compilation profiles:

[profile.dev.package."*"]
opt-level = 3

This forces all external packages (like CPU decoding and PPU rendering) to compile with release-level optimizations, even in debug builds.

07. Challenges Solved

Input Rollback Execution & Frame Capture

When a remote input arrives late, the emulator must roll back to the mismatch frame, apply the corrected input, and replay the intermediate frames to match the current local clock. We engineered the RollbackManager:

Caches emulator state snapshots at regular intervals (e.g. every 6 frames).
Serializes the CPU and PPU states using binary bincode encoders.
On input mismatch, the manager restores the closest snapshot, updates the input history, and runs run_frame() sequentially to catch up, all within the 16.6ms frame budget.

Local Clock: Frame 12  ───► (Input mismatch detected at Frame 8)
                             │
                             ▼
                    [ Restore Snapshot 6 ]
                             │
                             ▼
                    [ Replay Frame 7 ]  (Correct local inputs)
                             │
                             ▼
                    [ Replay Frame 8 ]  (Apply correct remote input)
                             │
                             ▼
                [ Replay Frames 9 - 12 ] (Fast catch-up)
                             │
                             ▼
                    [ Resume Realtime ]

08. Lessons Learned

Rust Memory Boundaries over FFI: Allocating large structures in Rust and passing ownership to Dart/Flutter creates memory leak risks. The FFI boundary must utilize raw pointers where the lifetime is explicitly managed by Rust constructors and destructors.
Audio Sync in Rollbacks: Audio packets cannot be rolled back. If the video game state is rolled back 5 frames, replaying those frames must not trigger duplicate audio generation, or it will create static audio pop sounds. The system must mute or discard APU audio generation during catch-up execution.

09. Infrastructure Diagram

Below is the Cargo Workspace structure and FFI integration flow:

graph TD
    subgraph CargoWorkspace ["Cargo Workspace (Rust Core)"]
        subgraph ContraCore ["contra_core (Library)"]
            A["api.rs (FFI Entry)"] --> B["rollback.rs (RollbackManager)"]
            B --> C["emulator.rs (Tetanes Deck)"]
            B --> D["packet.rs (10-byte wire packet)"]
            B --> E["transport::iroh_native"]
            B --> F["voice.rs (Opus Codec)"]
        end
    end

    subgraph FlutterApp ["contra_ui (Flutter Frontend)"]
        UI["Flutter Game View"] -->|Pointer Render| FB["Frame Buffer (256x240 RGBA)"]
        Ctrl["Virtual Controller"] -->|Bitmask Events| UI
    end

    UI -->|API Call| A
    A -->|Raw Pointer Swap| FB
    E -->|QUIC P2P Tunnel| Remote["Remote Peer (Iroh Node)"]

10. Technology Stack

Programming Language: Rust 1.91 (Cargo Workspace)
Emulation Core: tetanes-core 0.14.2 (NES)
FFI Binding Generator: flutter_rust_bridge 1.82.4
P2P Transport Protocol: Iroh 1.0.0 (TLS-ring encrypted)
Async Runtime: Tokio 1.35 (Full feature set)
Audio Codec: Opus 0.3.1
Binary Serialization: bincode 2.0.1 (Legacy config)
Struct Mapping: Serde 1.0
Target Targets: WASM32 (Web), Native (Desktop/Mobile)
User Interface: Flutter SDK 3

11. Future Improvements

Spectator Mode: Multiplexing Iroh P2P datagrams to support multiple viewing peers connecting to a live netplay session.
Auto-Frameskip Adjustment: Dynamically adjusting the emulation clock speed (e.g. to 59Hz or 61Hz) to match minor clock drifts between peers, reducing the need for rollback executions.

12. Architecture Decision Records (ADR)

[ADR-01] Selecting Rust as the Core Engine Language

Context: High-frequency emulation loops requiring sub-millisecond serialization and rollback replays within a tight 16.6ms frame budget.
Decision: Selected Rust instead of C++ or Go.
Rationale: Rust offers strict memory safety guarantees and high-performance execution without a Garbage Collection runtime. This allows us to predict execution times accurately and load bincode snapshots without GC pauses that would introduce input latency and game stutters.

[ADR-02] Cycle-Accurate Emulation over Scanline-Based Emulation

Context: Ensuring deterministic rollback synchronization across different clients.
Decision: Integrated the cycle-accurate tetanes-core emulator engine.
Rationale: Scanline-based emulation is faster but lacks the precision required for precise hardware sync. Minor discrepancies in PPU rendering states would diverge the game state after a few hundred frames, causing rollback loops to desynchronize and crash the netplay session.

[ADR-03] Lock-Free C-FFI Pointer Swap Rendering

Context: Sharing the 256x240 RGBA frame buffer between the Rust emulation thread and the Dart/Flutter rendering UI thread.
Decision: Implemented lock-free pointer swapping over raw C-FFI functions.
Rationale: Protecting the frame buffer with standard thread locks (like Mutex) causes render threads to block, degrading performance below 60 FPS. Swapping pointers atomically allows Rust to render to a background buffer while Flutter displays the front buffer, achieving a smooth 60 FPS.