Core
Three roles work together: Containers store and nest media, Panels render layouts, and the Organizer coordinates where/when content appears.
AI Compositing
Nest two or more panels/containers along the Z-axis; AI proposes a hierarchical output that you can save as a unique, reusable panel grouping.
Z-space Nesting
Stack NMPs and NMCs in three-dimensional order (back→front). Each layer declares a role (viewport, overlay, HUD, subpanel) plus blend/opacity. The result is a structured tree with depth-aware rendering semantics.
AI Hierarchy Proposals
The AI analyzes layer roles, data bindings, and context (SoI, device, task) and proposes a labeled hierarchy (parents, children, constraints). It can auto-group commonly paired panels (e.g., “Threat HUD + Live Video + Ticker”).
User-Defined Groupings
Accept, reorder, or refine the proposal to mint a custom grouping. Save it as a named composite so it can be recalled on desktop, mobile, or headsets/wearables.
type PanelGroup = { id: string;
name: string;
soi: 'Private'|'Social'|'Public';
layers: Array<{
id: string; kind: 'panel'|'container'; role: 'viewport'|'overlay'|'hud'|'subpanel';
z: number; blend?: 'normal'|'screen'|'multiply'|'add'|'none'; opacity?: number;
bindings?: Record<string,string>;
}>;
constraints?: string[]; // e.g., "hud must be above viewport"
origin: 'ai-proposed'|'user-defined';
version: number;};
# Ask AI to propose a hierarchy from selected layers
POST /v1/voyager/hierarchy/propose
{ selection: ['nmp_video','hud_threat','ticker_ops'], context:{ zone:'SOC', role:'analyst' } }
→ { groupDraft: PanelGroup, rationale: [...] }
# Save a custom grouping
POST /v1/moe/groups { ...PanelGroup }
# Rehydrate grouping on a device
GET /v1/moe/groups/{id}?device=mobile
→ { layout: ..., layers: ... }
Flow
Encoders and data sources populate containers; the organizer schedules them; NMPs render the result across visual, aural, and spatial channels.
MoE Integration
MoE discovers panels, selects the right variant, and binds agent actions to panel UI — all governed by SoI policies.
GET /v1/moe/panels?zone=ops-floor&role=soc_analyst
→ { panels: [{ id: "nmp_ops_wall", variants: ["private","social"] }], context: {...} }
POST /v1/moe/panels/{id}/actions
{ label: "Quarantine host", planId: "pl_01H...", soi: "Private", approvals: ["SOC Lead"] }
Orchestration
Drag‑and‑drop schedules, nested playlists, and priority rules control what each panel shows.
I/O
Panels coordinate displays, speakers, and haptics so a single layout can drive multi‑channel delivery.
Visual
Video viewports, image slots, overlay polygons.
Aural
Layered audio racks and voice prompts synchronized to regions.
Spatial / Haptics
Spatial anchors and optional haptic cues for AR/room‑scale ops.
Governance
Least‑privilege exposure of panel variants; every composition decision is logged.
Access control
Role/zone policies determine which NMP variant is visible.
Approvals
Risk‑tiered, human‑in‑the‑loop for high‑impact actions.
Audit & provenance
Trace schedules, overrides, and panel outputs to source tracks.
API
Minimal types for containers, panels, and schedules with organizer endpoints.
type MediaTrack = { id: string; kind: 'video'|'audio'|'image'|'polygon'; src: string; meta?: Record<string, any> }
type Container = { id: string; name: string; tracks: MediaTrack[]; children?: Container[] }
type PanelRegion = { id: string; role: 'video'|'audio'|'image'|'overlay'|'subpanel'; layout:{x:number;y:number;w:number;h:number}; childPanelId?: string }
type Panel = { id: string; name: string; regions: PanelRegion[]; children?: Panel[]; policy?: string[] }
type Assignment = { panelId: string; containerId: string; start: string; end: string; priority?: number }
POST /v1/nmc { name, tracks:[...], children:[...] }
POST /v1/nmp { name, regions:[...], children:[...] }
POST /v1/organizer/schedules { assignments:[...] }
POST /v1/compose { panelId, at }
→ { layout, resolvedAssets, conflicts:[], outputs:['visual','aural','spatial'] }
Build with Nested Media Panels
See authoring, scheduling, and MoE‑surfaced NMPs in action.