ddidderr
1a827b8b0d
Implements the simulator described in README.md: a 3D room in which a
loudspeaker can be moved and aimed, heard binaurally from the camera's
position, with every acoustic term shown live so the user can read why it
sounds the way it does.
Only README.md was previously committed. An earlier AI-generated
implementation existed in the working tree but was never committed; it is
superseded rather than modified. That implementation did not work:
renderer.setSize() was never called and the stylesheet had no rule for
html, body or the canvas container, so the WebGL view rendered into a
300x150 box in the corner of an unstyled page. Its acoustics were also
wrong in ways that voided the stated goal, which is what motivated a
rewrite rather than a repair:
- AudioContext.listener orientation (forwardX/Y/Z, upX/Y/Z) was never
written, only position. Orbiting the camera therefore produced no
change in the sound field at all.
- The listener was a separate avatar mesh, not the camera, contradicting
the README. Its syncFromCamera() had zero callers.
- The reverb send was tapped after the distance, cone and occlusion
gains, making the direct-to-reverberant ratio mathematically constant.
Walking away sounded like a fader move rather than like distance.
- Doppler was derived from a single-frame velocity finite difference
smoothed with a per-frame (not per-second) lerp, so any drag produced
an octave-wide pitch chirp and the response varied with refresh rate.
Design decisions worth recording, since each has a cheaper alternative
that was rejected:
The camera IS the Web Audio listener. Position and orientation are both
published every frame. OrbitControls panning is disabled on purpose: it
moves the orbit target without moving the camera, which would silently
desync the ear from the view and break that invariant.
Doppler is emergent, not computed. A DelayNode holds distance /
speedOfSound; chasing it resamples the signal exactly as air does. This
removes velocity estimation from the codebase entirely, which is what
eliminated the drag-chirp class of bug. The reported ratio is 1 - dD/dt,
the shift a delay line actually produces. The textbook moving-source form
1/(1 - v/c) agrees only to first order and has a pole that inverts the
sign of the shift when a source closes fast.
The reverb send is tapped BEFORE distance, directivity and occlusion,
because a room's reverberant field is roughly uniform: it depends on how
much power the source radiates, not on where the listener stands. This is
what makes the critical distance audible, and it is the one thing the
simulator teaches that a fader cannot fake.
Room acoustics are derived from geometry via Sabine (T60 = 0.161V/Sa)
rather than from named presets, so the room-size sliders genuinely change
the sound. The impulse response is normalised to unit energy per channel,
not unit peak, because a convolution's output level follows total energy;
peak normalisation would make a long tail far louder than a short one and
the send gain would stop meaning anything.
Source presets are built additively from a bounded harmonic series instead
of being sampled from ideal waveforms, leaving an octave of clean headroom
so Doppler can pitch them up without folding harmonics back down as
aliasing.
User-supplied audio files are summed to mono. A stereo mix fed into the
panner would leave part of the image fixed to the listener's ears no
matter where the source moved, which is precisely the illusion this app
exists to break.
Dark theme only. The viewport is a lit 3D room; a light chrome around it
reads as a white frame on a dark photograph, and re-lighting the scene for
a light theme would need a second set of materials for no real gain.
User-visible behaviour, relative to the uncommitted prior version:
- The 3D view fills its pane and is sized on construction plus via a
ResizeObserver.
- Orbiting the view swings the sound between the ears (measured at
roughly 14 dB of L-R swing across a full orbit in an anechoic room).
- Walking away attenuates the direct path while the reverberant field
holds steady, and a HUD chip reports which of the two dominates.
- A signal-path table lists every stage between source and ear in dB on
one shared -60..0 dB scale. The stage losses sum exactly to the total;
air absorption deliberately reports no dB because it has no broadband
loss, only a cutoff.
- Audio files can be loaded by drop or picker and behave as the source.
- Three guided demos, a first-run explainer, and a keyboard model.
- Leaked agent artifacts are gone: the panel header no longer reads
"M4 OVERLAY", sections are no longer lettered "A.", "B.", and the
milestone scratch directories and docs are deleted.
Known limitations, also listed in README.md:
- Reverb is a single static impulse response per room; it does not vary
with listener position, so walking into a corner is not audible.
- No precedence effect. Real ears localise well beyond the critical
distance because the first wavefront wins; here localisation degrades
with the direct-to-reverberant ratio, so a very live room smears the
image more than it would in life.
- Occlusion tests one obstacle and models diffraction as a fixed
broadband loss rather than as a function of the obstacle's size.
- HRTF quality is whatever the browser's PannerNode provides.
- Loaded files loop with an audible seam; the crossfade used on the
generated presets would be wrong for music.
Test Plan
npm test 116 unit tests. Physics and room acoustics are pure
functions. The audio graph is tested against a mock
AudioContext that records connections, so the suite
can assert topology directly: that the reverb send
hangs off the delay rather than the direct gain, and
that listener forward/up are published and not just
position. Both were silently wrong before.
npm run test:e2e 47 checks driving real Chrome; starts its own dev
server. Covers what a mock cannot: that the view is
actually sized, that orbiting changes the ear balance,
that walking away attenuates the direct path while the
room holds steady, that each demo does what its
caption claims, that the panel follows the demos
rather than silently reverting them, and that a real
WAV decoded by the browser spatialises like any other
source while a non-audio file fails visibly.
npm run build tsc plus a production bundle, split so three.js gets
its own long-lived chunk (470 kB three, 99 kB app).
Manual check: load the page in a browser with headphones, press
"Enable audio & start", then press 3 and walk away with W/S. The Direct
row should fall while Room (wet) stays flat, and the HUD chip should flip
to room-dominant as you cross the ring drawn on the floor.
Refs: ORIGINAL_REQUEST.md
Docs: README.md
152 lines
7.8 KiB
Markdown
152 lines
7.8 KiB
Markdown
# Resonance — a spatial audio simulator in Three.js
|
||
|
||
Move a loudspeaker through a 3D room and hear exactly what distance, direction,
|
||
obstacles and the room itself do to the sound. **Your camera is the microphone**:
|
||
orbiting the view swings the sound around your head, walking away makes the room
|
||
take over, and stepping behind a pillar muffles the source.
|
||
|
||
Every number on screen is the actual value driving the audio graph that frame.
|
||
|
||
```bash
|
||
npm install
|
||
npm run dev # http://localhost:5173
|
||
```
|
||
|
||
Headphones strongly recommended — the binaural simulation collapses on speakers.
|
||
|
||
**Bring your own audio.** Drop a file onto the 3D view, or use *Load a track from
|
||
your computer* in the Source panel — MP3, WAV, FLAC, OGG, M4A, whatever your
|
||
browser decodes. It becomes the loudspeaker: set Motion to Orbit and your song
|
||
circles your head, Dopplering as it goes, muffling when it passes behind the
|
||
pillar. Files are summed to mono, because a loudspeaker standing at one point in
|
||
a room radiates one signal; a stereo mix would leave half the image glued to your
|
||
ears no matter where the source went. Nothing is uploaded — decoding happens
|
||
locally in the page.
|
||
|
||
## What it models
|
||
|
||
| Effect | How |
|
||
| --- | --- |
|
||
| **Distance attenuation** | Inverse / exponential / linear laws. At rolloff 1.0 the inverse law is exact physics: −6 dB per doubling. |
|
||
| **Directivity** | Web Audio's inner/outer cone model. The lobe in the scene dims with the gain you are actually receiving. |
|
||
| **Propagation delay** | A delay line holding `distance / speedOfSound`. Sound genuinely arrives late. |
|
||
| **Doppler** | *Emergent.* Nothing computes a Doppler ratio to feed the audio — the delay line chasing a moving source resamples the signal, exactly as air does. The reported ratio is `1 − dD/dt`, which is what a delay line actually produces. |
|
||
| **Room reverberation** | A synthetic impulse response built from the room's real geometry via Sabine's equation, `T60 = 0.161 V / Sα`. Early reflections land at the actual wall arrival times. |
|
||
| **Direct-to-reverberant ratio** | The reverb send is tapped *before* distance and directivity, because a room's reverberant field is roughly uniform. This is the effect that makes distance audible. |
|
||
| **Occlusion** | A bundle of rays across the head gives fractional occlusion, which sweeps a lowpass on the direct path only — reflected sound still gets through. |
|
||
| **Air absorption** | Distance-dependent treble loss. The default is realistic, which means subtle; the slider goes to 8× if you want to hear it. |
|
||
|
||
### The critical distance
|
||
|
||
The one idea the app is built around. The direct sound falls as `1/r`; the
|
||
reverberant field does not fall at all. They cross at the **critical distance**,
|
||
`rc = √(R / 16π)`, drawn as a ring on the floor:
|
||
|
||
- **Inside the ring** the source dominates. You can point at it with your eyes shut.
|
||
- **Outside it** the room dominates. It does not get quieter as you back away — and that constancy is a large part of how your ears judge distance.
|
||
|
||
Press <kbd>3</kbd> for a guided walk through it.
|
||
|
||
## Controls
|
||
|
||
| | | | |
|
||
| --- | --- | --- | --- |
|
||
| <kbd>W</kbd><kbd>A</kbd><kbd>S</kbd><kbd>D</kbd> | walk (you are the mic) | <kbd>Q</kbd> <kbd>E</kbd> | down / up |
|
||
| <kbd>↑</kbd><kbd>↓</kbd><kbd>←</kbd><kbd>→</kbd> | move the source | <kbd>PgUp</kbd> <kbd>PgDn</kbd> | source height |
|
||
| drag | orbit | <kbd>Shift</kbd> | move 3× faster |
|
||
| <kbd>G</kbd> / <kbd>R</kbd> | gizmo: move / turn | double-click | place the source |
|
||
| <kbd>K</kbd> | play / pause | <kbd>H</kbd> | hide panels |
|
||
| <kbd>1</kbd><kbd>2</kbd><kbd>3</kbd> | guided demos | <kbd>?</kbd> | all shortcuts |
|
||
|
||
Camera panning is deliberately disabled: it would move the orbit target without
|
||
moving the camera, quietly breaking the invariant that the camera *is* the ear.
|
||
|
||
## The signal path
|
||
|
||
The bottom-left panel is the whole point. It lists every stage between the
|
||
source and your ear with its loss in dB on one shared −60…0 dB scale, so the
|
||
bars are directly comparable and the stage losses sum exactly to the total:
|
||
|
||
```
|
||
Source Sawtooth −3.1 dB ██████████
|
||
Distance 12.24 m −21.8 dB ███████
|
||
Directivity 0.0° off-axis 0.0 dB ██████████
|
||
Occlusion clear 0.0 dB ██████████
|
||
Air tone only · LP 20.5 kHz —
|
||
──────────────────────────────────────────────
|
||
At your ear 35.7 ms flight −21.8 dB ████
|
||
Room (wet) t60 1.63 s −9.1 dB ████████
|
||
Direct / room room-dominant −12.7 dB
|
||
```
|
||
|
||
Air absorption has no broadband dB — it only shapes tone — so it deliberately
|
||
reports no number rather than an invented one.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
src/
|
||
physics/ Pure functions, no Web Audio, no Three.js. Fully unit-tested.
|
||
vector · attenuation · cone · doppler · occlusion
|
||
audio/
|
||
AudioEngine Node graph + per-frame spatial update + telemetry
|
||
presets Band-limited, seam-free looping source material
|
||
reverb Sabine acoustics and impulse-response synthesis
|
||
userAudio Decoding and mono-summing files from disk
|
||
scene/
|
||
Stage Renderer, camera-as-listener, orbit + gizmo, occlusion rays
|
||
Room · SoundSource · Annotations · motion
|
||
ui/
|
||
panel · signalPath · readouts · meters · onboarding · controls · format
|
||
main.ts One rAF loop: stage → engine → readouts
|
||
```
|
||
|
||
The audio graph:
|
||
|
||
```
|
||
source ▶ sourceGain ▶ delay ─┬─▶ air ▶ occlusion ▶ direct ▶ panner(HRTF) ─┐
|
||
│ ├▶ master ▶ limiter ▶ analyser ▶ out
|
||
└─▶ reverbSend ▶ convolver ──────────────────┘
|
||
```
|
||
|
||
`AudioEngine.updateSpatial()` computes the full acoustic state whether or not an
|
||
`AudioContext` exists, so every readout is live and correct before the user has
|
||
unlocked audio.
|
||
|
||
Source presets are built additively from a bounded harmonic series rather than
|
||
sampled from ideal waveforms. That leaves an octave of clean headroom, so
|
||
Doppler can pitch them up without folding harmonics back down as aliasing.
|
||
|
||
## Tests
|
||
|
||
```bash
|
||
npm test # 116 unit tests: physics, room acoustics, audio graph, presets, file loading
|
||
npm run test:e2e # 47 checks driving real Chrome (starts its own dev server)
|
||
npm run build # tsc + production bundle
|
||
```
|
||
|
||
The unit suite runs in Node against a mock `AudioContext` that records the graph
|
||
topology, so it can assert things like "the reverb send is tapped before the
|
||
distance gain" and "the listener's orientation is published, not just its
|
||
position" — both of which were silently wrong before.
|
||
|
||
The browser suite covers what a mock cannot: that the view is actually sized,
|
||
that orbiting swings the sound between the ears, that walking away attenuates
|
||
the direct path while the reverberant field holds steady, that each demo does
|
||
what it claims, and that the panel controls follow the demos rather than
|
||
silently reverting them.
|
||
|
||
## Known limitations
|
||
|
||
- Reverb is a single static impulse response per room. It does not update with
|
||
listener position, so you cannot hear yourself walk into a corner.
|
||
- No precedence effect. Beyond the critical distance real ears still localise
|
||
well because the first wavefront wins; here localisation degrades with the
|
||
direct-to-reverberant ratio, so a very live room smears the image more than
|
||
it would in life.
|
||
- Occlusion tests one obstacle, and models diffraction as a fixed broadband loss
|
||
rather than as a frequency-dependent function of the obstacle's size.
|
||
- HRTF quality is whatever the browser's `PannerNode` provides.
|
||
- Dark theme only: the viewport is a lit 3D room, and a light chrome around it
|
||
would need a second set of scene materials to not look like a picture frame.
|