How the iPhone recording array works

September 19, 2026 · OdiMeet development notes

Interviews, meetings, podcast field recordings, consultations — when one of those ends, what you actually want is not an audio file but a summary that answers questions: who said what, what the key points were, which action items came out of it, what was stated as fact, which moments are worth replaying. And that material is often sensitive enough that it should never leave your phone, so the tool has to be privacy-first.

"Who said what" needs speaker diarization first. The mainstream way is algorithmic: slice the audio into short segments, extract a voice-print embedding for each with a model such as an x-vector or ECAPA-TDNN, then cluster similar segments into the same speaker — pyannote.audio, NVIDIA NeMo and WhisperX all work this way; the newer EEND family trains one network to output "who is talking now" end to end. All of them are clever, and all of them share a weakness: at bottom they are guessing. Similar voices, two people talking over each other, someone leaning into the microphone — the guess breaks, and every vendor's documentation says so.

OdiMeet wants to explore a different route: several iPhones as a microphone array, one in front of each speaker, deciding who is talking by physics rather than judgement — whoever is closest is loudest on their own phone. This note is the story of getting that from an idea to something that works on two real recordings, including the two times my own measurements lied to me. So how is it actually done?

Where the usual gadgets put your privacy

The products in this space follow the same recipe. One: you buy a small recording device for a hundred and something, sometimes two hundred dollars — some are badges you clip to your chest. Two: after the meeting, the device or its app turns the audio into text locally. Three: the content goes up to the vendor's servers, where the cloud does the speaker diarization and generates the summary, the mind map, the to-do list. Step three is what makes it convenient, and it is also the price: everything you said, everything the other side said, all of it has been read in full by that company and now lives on their machines.

OdiMeet is built the other way round. Who is talking is decided by the iPhone array; capture, recording, transcription, alignment and merging all happen inside those few iPhones. Only when you want a summary do you choose to send the already speaker-labelled transcript to Google Gemini — with your own key, straight to Google, nothing of ours in between. We have no servers; strictly speaking, apart from you, only Gemini ever sees that text, and only at the moment you ask for a summary. If you'd rather skip even that, export the labelled transcript and hand it to a local model on your own computer — then not a single word leaves your devices.

Physics instead of a model

Distance doesn't fail the way voice-prints do. A phone 40 cm from its speaker and 1.5–2 m from everyone else hears its own person ten to twenty decibels louder — the inverse-square law, doing the work a neural network can't do reliably.

So the pipeline is deliberately dumb: align the tracks, keep only the voice band (300–3400 Hz), measure each track's energy every 200 ms, and hand the moment to the loudest track. If two tracks are both loud and close, both people are talking, and both are kept. Then each speaker is transcribed from their own microphone — the closest, cleanest one — and the pieces are merged by time.

Two phones, one start

The phones pair over Multipeer Connectivity — Wi‑Fi and Bluetooth, no network, no account — gated by a six-digit code. They ping each other to estimate the clock offset, the host counts down 3‑2‑1, and both start recording 48 kHz WAV at the agreed instant. When the host stops, the guest sends its track back automatically, so the host holds both and can merge them with one tap.

The first real take was 14 seconds: two phones about 1.5 m apart on a table, each 40 cm from its speaker, two people alternating. I pulled the two WAV files off and looked at the numbers before writing another line of app code.

Trap one: I found a bug that didn't exist

The energy numbers were good news: on windows with speech, the two tracks differed by a median 10.6 dB. But the alignment step said the tracks were 2.83 seconds apart. A synchronized start that misses by three seconds is broken, so I went hunting in the clock code — rewrote the calibration, switched to absolute timestamps, added a spin-wait to the exact instant. The next take came back "3.90 seconds off". Worse.

The clock was fine. My aligner was wrong. I was cross-correlating the two tracks' loudness envelopes — and when two people alternate, the envelopes are anti-correlated at the true offset (A loud while B quiet), so the search locks onto some other lag where A's speech happens to line up with B's. The method that works is GCC‑PHAT on the waveforms: the same words reach both microphones a few milliseconds apart, so the phase correlation peaks at the true offset regardless of who is talking. The real number was 88 ms. The synchronized start had been good all along.

Trap two: a suspiciously perfect 0.00 s

Next take, 20 seconds, with names entered this time. The app reported an alignment offset of 0.00 s. Two independent phones do not land within one sample of each other; the previous take had been 88 ms. I looked at every segment separately: 0.000 ms, every time, with a correlation peak 530 times its background. And the per-frequency coherence between the tracks was 0.004. The "perfect" peak had no acoustic content at all.

The cause was a step I'd added for good reasons. Before correlating, I band-passed both tracks by zeroing FFT bins outside 300–3400 Hz — a brick-wall filter. That stamps identical Gibbs ringing onto both tracks at identical sample positions, and the correlator found it and called it a match. Band-limiting inside the cross-spectrum instead, with no filter on the time signals, gave the real offset: 106 ms, with a clean 61× peak.

That one now has a regression test with the structure of the real failure: two independent alternating talkers, only −16 dB of bleed between them, one started 106 ms late. The old code returns 0. The new code returns 106.

Once the alignment was right

The merged transcript labels each line with the name entered before pairing, and the tester's verdict on the second take was "separated, quite good". Two smaller traps on the way there, for completeness: a merged conversation borrows track A's audio file and session id, so the next "combine" paired A with itself (offset 0.00, separation 0.0 dB — the same tell); and iOS returns the literal string "iPhone" as the device name to apps, so both speakers were called iPhone until we asked for a name up front.

So how good is the algorithm, really? We ran it too — after being schooled by our own data

Earlier I called the mainstream approach a guess, citing other people's documentation. I took a recording, one track only, and ran the route a single phone can run: pyannote segmentation‑3.0, 3D‑Speaker voice-print embeddings, clustering — the whole chain on device through sherpa‑onnx, about 40 MB of models. My first comparison used the array as the answer key, produced a set of numbers, and went live; hours later I learned that the 20‑second take had been one person taking turns toward each phone. The array was fine — it measures position, and all seven turns were right — but for a voice-print algorithm the correct answer was "one speaker", and my conclusion was backwards. One thing learned in passing: distance leaks into voice-prints; the same person at 40 cm and at 1.5 m looks like two people to the algorithm.

So we recorded again: two different people, 2 metres apart, 41 seconds, strict turn-taking with a few deliberate interruptions at the end — this time the answer key is the known order, not either method's output.

The array: alignment error 53 ms; across 28 seconds of speech the two tracks differ by a median 17.2 dB, with 96% of windows decided by a ≥ 6 dB margin; 18 utterances, strictly alternating, 18 of 18; three 200 ms windows flagged as both speaking, all between 30.8 and 31.6 s — exactly where the interruptions were. Moving from 1.5 to 2 metres made separation better, not worse.

The algorithm depends on which track it gets and on whether someone tells it how many people there are:

Told "two speakers" and given the track where both voices are reasonably audible (B, the less sensitive microphone, where the far person is relatively less faint), it reaches 90% — 19 utterances with a single extra cut, an honestly good result. On the other track the far speaker is so faint that the segmentation model drops nearly half the speech, and what remains is clustered by time rather than by person: 37%. And in the room you don't know which kind of track you are holding. More practically: nobody types in a speaker count before a real meeting — left to guess, the algorithm found 1 person on one track and 8 on the other.

The array needs no speaker count and doesn't care how clear anyone's voice is; it only looks at distance. So OdiMeet ships both, in a clear order: Precision Mode with two iPhones is measured; single-phone speaker separation is an estimate for when there is no second phone, off by default and labelled that way.

So, can a three-iPhone array reconstruct the room?

By this point a question suggests itself: if the array knows who is talking, and measures when each phone hears it, can we run it backwards and draw the room — who sits where, next to whom, across from whom?

Physically it is the same data. Sound travels at 343 m/s, so two metres is 5.8 ms, and the phase correlation we already use for alignment resolves well under a millisecond. And a sentence comes out of one mouth: when A talks, A's phone hears it first and B's a little later; when B talks, the other way round. The average of the two lags is the clock offset; half the difference is the extra distance the sound travelled. So I re-measured all three takes utterance by utterance, tapering each span with a Hann window (so the cut edges don't stamp identical clicks on both tracks — a cousin of the earlier trap):

The first two rows are good news: the measurement is stable to three centimetres, and the first take's distance comes out right. The third row is an open puzzle: every one of B's utterances arrives at A's phone exactly 27.7 ms late — far too consistent for noise, far too long for the direct path. Most likely A's phone never got B's direct sound (blocked, or too faint) and the correlation locked onto a strong reflection with a round trip of about 9.5 m. It is also a reminder that geometry may only trust physically possible lags; a two-metre table does not produce 27 ms.

A useful by-product fell out: the single whole-file alignment leans toward whoever talked more — take 3's "53 ms" was really A's value, and the true clock offset is about 66 ms. Harmless for 200 ms attribution windows, but geometry needs the average of both speakers.

And three phones? Two phones yield one distance. With three, each speaker's words give three pairwise lags, which is the distance between each pair of phones — a triangle, and with it who is adjacent to whom and who sits opposite. What distances alone cannot give is left versus right: a mirrored triangle has the same sides. Breaking the mirror needs one directional reference, and an iPhone carries two ready-made: each phone's compass heading (read once at pairing; 5–15° of error is plenty to tell left from right), or stereo recording, available since the iPhone XS.

None of this is in the app yet. It is an exploration — but the raw material is already inside every array recording, with nothing extra to capture. If it holds up, a meeting would end with a transcript and a small top-down sketch: who sat where, who spoke to whom and how much. A product with one microphone cannot draw that picture.

What I'm keeping from this

My aligner passed a synthetic test — one signal delayed by 1.234 s — perfectly, then failed on real audio twice. The synthetic case shared none of the real case's structure: two independent talkers, faint bleed, alternation. Validate an estimator on the case it will actually see.

And an answer that is too good is the instrument talking about itself. Two phones aligned to 0.02 ms, a peak 500 times its background, an offset landing exactly on the search boundary — each of those was the estimator, not the room. The check that settled it was asking what was coherent, and finding nothing was.

OdiMeet is free for 90 days, then $2.99 a month, $29.99 a year, or $39.99 once. Follow odimeet.app for the App Store link.