work

This is my prototype implementation of pixel-motion voxel projection. Its goal is to fuse motion before detection. Conventional multi-camera triangulation begins by identifying an object independently in each camera image, then deciding which detections correspond. Pixel-motion voxel projection is an inversion of that order. Here, each camera computes the absolute per-pixel difference against an exponential moving background. Pixels above a small noise floor then back-project their difference values into a shared voxel grid, adding them to every voxel their ray intersects. When these rays are aggregated from several independent views, detection almost falls out of the grid as a consequence.

A single weak pixel that would not survive object-level detection in a conventional system can therefore still contribute evidence in 3D. Noise and foreground motion that are uncorrelated between views mostly remain as unsupported lines through the grid, whereas correlated observations concentrate around their common origin. Each additional camera adds another linear pass over its observations while making accidental agreement between independent views progressively less likely.

My embedded implementation runs across both cores of a Canaan K230. The 1.6 GHz C908 runs RT-Smart, captures 1280×720 NV12 frames from an OV5647 through VICAP/ISP, and uses an Odin process to maintain the EMA background, extract motion, and 2×2 max-pool the result. DATAFIFO carries the sparse observations through shared memory to Linux on the 800 MHz C908, where a small C process retains only the latest frame and forwards it to the server over TCP.

The Rust server reconstructs each observation as a world-space ray and walks it through the grid with 3D-DDA, accumulating motion values and distinct-camera support in a sparse voxel table. Voxels above the configured intensity and contributor thresholds are clustered with DBSCAN. The resulting detections are globally associated using the Hungarian algorithm and tracked with constant-velocity Kalman filters.

This is my opinionated Vulkan rendering interface. It targets Vulkan 1.3 with VK_EXT_shader_object and VK_EXT_descriptor_buffer.

Buffer resources are entirely bindless, and addressed through GpuPtr<T>, a shared host/shader abstraction over Vulkan device addresses: it is an opaque typed address on the CPU and a directly dereferenceable pointer in rust-gpu shaders. Images and samplers live in bindless descriptor buffers. Together with shader objects and dynamic rendering, this removes per-draw descriptor-set updates, graphics-pipeline management, render-pass objects, and framebuffers from client code.

My renderer builds on this with persistently mapped per-frame arenas whose reuse is gated by timeline semaphores. Per-frame draw and dispatch data can be copied directly into GPU-visible memory and passed to shaders as device addresses through push constants, without per-object bindings or staging copies.

Asha is the name I have given to the rendering half of my engine. It is less one monolithic renderer and more so a collection of the shaders and GPU systems that have accumulated while making games. It sits on top of my gpu RHI, and assumes bindless resources, typed device addresses, and indirect work from the bottom up. I've also tried to make the host interface as declarative as possible, so the CPU can just describe what it wants drawn.

Geometry is divided into meshlets, which are culled and compacted on the GPU, then submitted with multi-draw indirect commands. Lightweight visibility buffer passes keep expensive material work out of overdraw. This core foundation supports a variety of rendering techniques, including ray-traced shadows, volumetric fog, stylized linework, UI widgets, and an implementation of Eric Lengyel's Slug text-rendering algorithm.

All of the shaders are written in Rust with rust-gpu. Per-frame data, mesh records, materials, and text descriptors are defined in the shared ABI crates and compiled for both the host and SPIR-V, rather than being mirrored across a language boundary. Together, rust-gpu and the gpu RHI have seriously transformed the way that I write graphics code. It's still kind of unbelievable how little friction remains when both the host and GPU share the same typed ABI, and the interface between them is reduced to a minimal set of primitives.