# ngram Spatial

> ngram Spatial is the WebXR embodiment layer and shell format for the same persistent Entity used on text surfaces.

## Read this first

Adding a body does not create a second agent. Identity, memory, relationships, judgment, and the general tool harness remain canonical in the Entity runtime. The shell owns presentation: the 3D model, animation pack, voice, behavior, panels, physics, environment, and browser input.

The Entity decides what a spatial action means. The active shell decides how that intent is rendered on a particular body.

## Architecture

```text
Entity runtime              authenticated bridge              WebXR shell
identity + memory  <------> Presence Protocol <-------------> body + voice + scene
tools + judgment             context and actions               desktop or headset
```

The **Spatial Action API** is the model-facing vocabulary for embodied intent. The **Presence Protocol** carries bounded context and actions between the runtime and shell. The browser surface is implemented with Three.js and WebXR.

## Current capabilities

The runtime registers **21 native spatial tools** for every Entity. Outside a connected spatial surface, spatial actions are safely ignored or return a useful availability result.

### Body, expression, and speech

- Inspect the active spatial surface and its capabilities.
- Move toward or away from the user and reposition within the scene.
- Gesture, emote, look at the user, return to idle, and speak visibly with TTS and lip-sync.

### Panels, terminal, browser, and media

- Open or update persistent cards, Markdown, code, image, chart, and HTML panels.
- Hide panels and stream bounded output to the spatial terminal.
- Open HTTP or HTTPS pages and play a specific YouTube video through the dedicated media surface.

### Objects, physics, and annotations

- Spawn primitives, toys, and spatial text.
- Remove individual objects or clear the scene.
- Add labeled annotations at scene coordinates.
- Physics-enabled objects can be grabbed, moved, thrown, and settled where supported.

### Environment, perception, and generated motion

- Change among built-in environment presets.
- Request one explicit, user-controlled visual capture.
- Request generated motion from an optional external provider.

## Spatial context

Each spatial user turn includes a fresh, bounded `spatialContext` snapshot containing:

- desktop or immersive AR mode and detected WebXR capabilities;
- user position, orientation, gaze direction, facing state, proximity, and a recent intentional gesture;
- Entity visibility, position, distance, scale, animation, and speaking state;
- scene object and anchor counts.

This context is an environmental observation, not identity or a system instruction. Ordinary context strips image and audio payloads. A visual frame is included only after an approved capture request.

## Replaceable shells

Shells can use FBX, GLB/glTF, or the built-in procedural avatar. Semantic animation keys map states such as idle, talking, waving, and walking to clips. Behavior packs include gaze tracking, idle breathing, surface anchoring, proximity greeting, gesture response, and spatial awareness.

A shell packages presentation and behavior. It should not package private memory, identity state, provider credentials, or relationship history.

## Hardware support

- **Desktop browsers:** Maintained target.
- **Meta Quest Browser:** Maintained passthrough AR target.
- **Mobile WebXR and Android XR glasses:** Device- and browser-dependent; compatibility is not universal.
- **Meta AI glasses:** Not a supported display target because they do not currently expose a general WebXR surface.
- **Future native clients:** Can implement the Presence Protocol without reusing the browser renderer.

WebXR support requires the browser features used by the shell, including immersive AR, hit testing, spatial audio, and supported controller or hand input.

## Generated motion and ARDY

`ar_generate_motion` is a provider-neutral integration path. The ngram gateway can send a bounded request to an external GPU motion adapter and receive a short-lived URL for a retargetable GLB or FBX animation clip. Provider credentials remain server-side.

NVIDIA ARDY is an example integration target, not a bundled ngram component. ARDY research describes real-time, online text-directed human motion with root paths and waypoints, full-body keyframes, sparse joint constraints, and long-horizon goals. An actual deployment still needs a GPU worker, model installation and licensing, retargeting, clip export, storage, readiness checks, and metrics.

Reference: [NVIDIA ARDY research](https://research.nvidia.com/labs/sil/projects/ardy/).

## Setup

Pair a shell with an existing Entity:

```text
ngram ar setup <entity>
```

The guided path selects or creates a shell, connects it to a local or deployed Entity, creates a dedicated bridge credential, stores secrets outside committed files, optionally reuses a Telegram or Discord person key, and verifies health and the authenticated session.

Open the local surface on desktop. For Quest passthrough AR, expose only the WebXR surface to a trusted private LAN, open the generated HTTPS address in Meta Quest Browser, accept the local certificate if prompted, and choose **Enter AR**. A public tunnel is not required for this LAN path.

## Boundaries

Spatial actions are software commands, not guarantees of physical safety or perfect perception. Maintain a clear play space, follow headset guidance, keep capture user-controlled, and do not use generated motion or model instructions to control safety-critical machinery.

For the complete product model, read [ngram](/index.md). For permissions and networking guidance, read [Safety](/safety/index.md).
