Real-time 3D & XR
Engines for live, interactive 3D: point clouds, VR and AR.
- Unity
- Unreal Engine
- TouchDesigner
- Meta XR SDK
- ZapWorks (AR)
- Mobile AR
Artist · creative technologist
My relationship with art began through technological interfaces — cameras, screens, the physics of game engines. Working backwards from digital to physical taught me to perceive materials differently.
Art
About
Work
A gallery that sees its visitors as live point clouds and talks about them — a distributed system of depth sensors, a brain-encoding model, visual memory and a local vision-language model, running for eight days.
Single-handedly transported, rigged, mapped, ran and struck a 20,000-lumen laser projector for Jadu’s mixer at Augmented World Expo.
Physical → Generated
Films that begin as handmade sets, slides and analog footage, then get reanimated with generative video.
Stack
Engines for live, interactive 3D: point clouds, VR and AR.
Local models and vision pipelines, run offline; generative tools for film.
What the systems are written in.
Real-time 3D in the browser, interfaces and the APIs behind them.
Modelling, drafting and photogrammetry.
Depth sensors, XR headsets, laser projection and live control.
The shop, the printer and the circuit.
Editing, sound, spatial audio and design.
Capabilities
Education
BA, December 2026. Visual arts, interactive technology, virtual environments & simulation; computer science (secondary).
Human and machine should co-exist, not compete.
— who decides which is what?
Senior thesis · Bennington College
Usdan Gallery · 18–26 May 2026
The closer I got to systems that could read and respond to human presence, the more I became interested in what they were missing.
The system view — sensors to wall in seven components. The concept and the voice live on the project page.
System map
Select a component — what it does, where it runs, what broke and the fix.
Does
Depth + RGB at 640 × 480, 30 fps each, angled inward and slightly down so the two views overlap across the room.
What broke · the fix
The sensor frames are aligned by ICP. A “place sensors from blueprint” shortcut snapped both back to rough positions mid-run and tore a seam through the cloud — now flagged as destructive; the saved calibration is reloaded instead.
Does
Merges both clouds, smooths depth over time, fills brief dropouts from a persistent background plate, detects presence and per-person heights, and writes presence.json for the perception loop.
What broke · the fix
The plate kept restoring depth captured at an old 4 m range after the range was raised to 8 m, slicing the far wall off. It now resets itself whenever min or max depth changes.
Does
Pulls the merged RGB stream and runs workers in parallel — 2-second clips at 8 fps for the brain model, single frames for MediaPipe pose (2 Hz) and recall — then assembles seven streams of context into one prompt.
What broke · the fix
It started serial — capture, brain, language model, caption — about 12 s per thought. A single-slot clip producer and an async brain worker let the language model read the freshest result instead of waiting: about 5 s.
Does
TRIBE v2 predicts cortical activation from each 2-second clip (POST /predict). open_clip ViT-B-32 embeds every arrival as a 512-d vector (POST /embed), so the room can recognize a return.
What broke · the fix
Raw activations sat between 0.00 and 0.01, so they’re rescaled against a rolling 10th–90th percentile (300 samples). If the host drops, the client feeds synthetic zeros and re-probes /health every 60 s. Recall matches at cosine 0.84.
Does
Turns the assembled prompt and the current frame into one short line — a noticing, a guess, a question — every thirty seconds or so.
What broke · the fix
It defaulted to help-desk speech. Shaped against the logs: a first-person persona, regex filters for third-person and greeter phrasing, a fixation analyzer that steers off repeated words, and a 30-second gate during long visits.
Does
Two project the merged point cloud, calibrated in-piece; the third shows the room’s line on black, over a top-down view of the cloud.
What broke · the fix
The caption canvas was depth-tested against the cloud, so points drew over the text. It moved to a screen-space overlay with a backing band and outlined glyphs, under one CanvasGroup so the fades still animate together.
Does
Keeps every thought and every visit — arrival, duration, CLIP signature, thumbnail — plus a reflection written at 4 AM with the room empty, which is injected into the next day’s prompt.
What broke · the fix
One night the reflection came back as a chatbot refusal and was stored as memory. Replies with refusal patterns are now discarded before saving, and filtered again at injection.
Why it’s built this way
Measure, then guessWhat we can understand, we can simulate.
It measures everything it can — depth, pose, brain response, a visual signature — and still only guesses. The line it speaks is a noticing, not a reading.
Its own measureMachines can’t be held to our metrics.
The brain signal only meant something once it was scored against its own history — a rolling 10th–90th percentile, not a human scale.
Who decidesReal and rendered — who decides which is what?
The rendered room on two walls, its reading of you on the third. The visitor stands between them.
Jadu × AWE — Augmented World Expo
Mixer · volunteer projection & AV
Volunteering with Jadu, an AR company, I ran the projection for their mixer at Augmented World Expo on my own, end to end — transporting, assembling and rigging a 20,000-lumen laser projector, mapping and running it live, then disassembling and packing it.
What I did
I rigged and configured the projector, drove the mapped content from Unity and TouchDesigner, and aligned it to the venue on site. Output ran at the projector’s native 1920 × 1200, with everything off the target masked to true black — at 20,000 lumens, spill is real. I operated it live through the event, then struck the rig.
Handling. All of the physical work was mine alone — transport, assembly, rigging, disassembly and packing. The PT-MZ20K is sealed and liquid-cooled, so breakdown means the lens and the rig, not the chassis: tool-free lens removal (top slide cover, lock lever, counter-clockwise out), the lens cased on its own, and the body cased level and left to reach room temperature before power-up, to avoid condensation.
Physical → Generated
Film · 2026 · on Flick
My films start in physical material — handmade sets, slides and footage shot by hand — which I then pass to generative video to reanimate and extend, cutting the built and the synthesized back into each other.
Process
Handmade sets and slides.
Analog, hand-held footage of the physical work.
Generative video reanimates and extends the footage.
The built and the synthesized, edited back together.
Trailer
It began as an analog experiment after Stan Brakhage, until AI reanimated it. The trailer follows the same arc: analog flashes first, then the reanimated footage.
Loop Prison — episodic series
The same method, carried across episodes.