Benn Jordan of The Flashbulb fame (who showed up here on HN recently for saving a PNG to a bird and then retrieving it) has a YT video where he shows off several interesting techniques for capturing audio using only video capture, which seem similar to this technique. https://youtu.be/mEC6PM97IRI
You can do something similar with a laser. The laser is reflected off of an object and into an receiver where it is converted into audio. The two techniques differ in methods, but they both rely on measuring the movement of secondary materials affected by sound waves which can then be used to infer speech.
It's interesting that measuring speech vibration is possible, but on the whole I don't see this as viable attack. One would need to get real close to a target, the cited 60% accuracy is at 50 cm distance. At 300 cm they report 2% which seems on the level of guessing words randomly.
The "remotely" in the title is no warranted in my view. Of course there may be ways to improve on their results. But so far it seems to me that a highly directional microphone would outperform the radar at any distance. One interesting aspect here is how sensitive the attack is to ambient noise. So maybe sitting close to a target with a radar hidden in headphones, one could glean some words even in a noisy environment like a subway. But I'd still put my money on a microphone performing better in that environment.
Now I wonder whether the attack would work better on headphones. They are smaller and should vibrate more. On the other hand the surface is smaller, so on the whole they might not provide more signal to a radar.
Most humans would assume that giving an app permission to use the accelerometer is not the same as giving the app permission to use the microphone. It's yet another form of permission confusion that allows for unintended privacy violations.
Researchers have managed to produce photos from the theoretically 1-bit photosensor on your phone (which is similarly permissionless), so don't make that proclamation so hastily. Even if it can't decipher a conversation, it may be able to detect things about your environment that leak bits of entropy about you.
Single pixel imaging is a well known method that depends on injecting known information (light patterns) and capturing the same scene multiple times in order to reconstruct.
Audio is by definition a time varying signal. If you cannot sample it fast enough, the information is gone. The fundamental Nyquist-Shannon sampling theorem has proven the concept of the critical (Nyquist) frequency. There are ways around this in special cases: for band limited signals undersampling is a method to reconstruct them using sampling lower than the critical frequency, and for sparse signals compressed sensing can be used. Real life audio is neither band limited (as sampled by any device on the phone), nor sparse. I think it is a physical impossibility with the current sensors.
To reiterate, I'm not suggesting it should be able to listen in on conversations, but rather that it may plausibly be able to detect recurring environmental vibrations. For example, let's say your phone is left on a table unnattended, then if your house is near a train then it may be able to sense the regularity of the train's passing (whose schedule itself varies throughout the day, providing a unique fingerprint for both determining which train it might be and your position along that train's route).
A microphone could work together with a radar; correlating the two signals may help tell the sound from the noise.
I suppose that at certain radio frequencies human bodies are mostly translucent, while the small metal membrane is still resolvable. When a head of a person covers well the acoustic signal (the phone is at the opposite ear), a microphone is helpless, but a radar us still fairly usable.
Of course Penn State. See Geschickter Fund (an CIA cover for MKULTRA research) and the School of International Affairs (SIA) (another CIA recruitment front).
That was likely Gyrophone (2014) from Stanford/Rafael researchers who demonstrated accelerometers and gyroscopes could capture speech frequencies at 80-250Hz - sufficient for speaker identification and some speech recognition without requiring microphone permissions.
Benn Jordan of The Flashbulb fame (who showed up here on HN recently for saving a PNG to a bird and then retrieving it) has a YT video where he shows off several interesting techniques for capturing audio using only video capture, which seem similar to this technique. https://youtu.be/mEC6PM97IRI
...to a bird?
He got the bird to repeat the sound he encoded the png into.
Appropriate username.
You can do something similar with a laser. The laser is reflected off of an object and into an receiver where it is converted into audio. The two techniques differ in methods, but they both rely on measuring the movement of secondary materials affected by sound waves which can then be used to infer speech.
* https://hackaday.com/2010/09/25/laser-mic-makes-eavesdroppin...
Benn Jordan has a cool video about a laser microphone he built:
https://youtube.com/watch?v=mEC6PM97IRI
Formerly TLA-level technology that you can now build yourself.
(three letter agency, eg NSA)
It's interesting that measuring speech vibration is possible, but on the whole I don't see this as viable attack. One would need to get real close to a target, the cited 60% accuracy is at 50 cm distance. At 300 cm they report 2% which seems on the level of guessing words randomly.
The "remotely" in the title is no warranted in my view. Of course there may be ways to improve on their results. But so far it seems to me that a highly directional microphone would outperform the radar at any distance. One interesting aspect here is how sensitive the attack is to ambient noise. So maybe sitting close to a target with a radar hidden in headphones, one could glean some words even in a noisy environment like a subway. But I'd still put my money on a microphone performing better in that environment.
Now I wonder whether the attack would work better on headphones. They are smaller and should vibrate more. On the other hand the surface is smaller, so on the whole they might not provide more signal to a radar.
Most humans would assume that giving an app permission to use the accelerometer is not the same as giving the app permission to use the microphone. It's yet another form of permission confusion that allows for unintended privacy violations.
MEMS accelerometers found in phones don't have nearly enough sampling frequency to encode meaningful audio
Researchers have managed to produce photos from the theoretically 1-bit photosensor on your phone (which is similarly permissionless), so don't make that proclamation so hastily. Even if it can't decipher a conversation, it may be able to detect things about your environment that leak bits of entropy about you.
https://news.mit.edu/2024/study-smart-devices-ambient-light-...
Single pixel imaging is a well known method that depends on injecting known information (light patterns) and capturing the same scene multiple times in order to reconstruct.
Audio is by definition a time varying signal. If you cannot sample it fast enough, the information is gone. The fundamental Nyquist-Shannon sampling theorem has proven the concept of the critical (Nyquist) frequency. There are ways around this in special cases: for band limited signals undersampling is a method to reconstruct them using sampling lower than the critical frequency, and for sparse signals compressed sensing can be used. Real life audio is neither band limited (as sampled by any device on the phone), nor sparse. I think it is a physical impossibility with the current sensors.
https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampli...
To reiterate, I'm not suggesting it should be able to listen in on conversations, but rather that it may plausibly be able to detect recurring environmental vibrations. For example, let's say your phone is left on a table unnattended, then if your house is near a train then it may be able to sense the regularity of the train's passing (whose schedule itself varies throughout the day, providing a unique fingerprint for both determining which train it might be and your position along that train's route).
Yes, for sure you can sample and detect lower frequency vibrations. That's the whole point of an accelerometer :)
A microphone could work together with a radar; correlating the two signals may help tell the sound from the noise.
I suppose that at certain radio frequencies human bodies are mostly translucent, while the small metal membrane is still resolvable. When a head of a person covers well the acoustic signal (the phone is at the opposite ear), a microphone is helpless, but a radar us still fairly usable.
Note that in this study the radar targeted the whole phone, not the membrane.
As I understand it, this technology would pick up the remote end of the conversation, are you sure your directed microphone could accomplish that?
Yes why not? I often hear the other side when somebody is on the phone.
Of course Penn State. See Geschickter Fund (an CIA cover for MKULTRA research) and the School of International Affairs (SIA) (another CIA recruitment front).
I think the thing with a camera, and detecting the movment of plants and chipsbag around to recreate the sound was much more impressive.
'chipsbag'??
https://news.mit.edu/2014/algorithm-recovers-speech-from-vib...
'potato-chip bag'
Scary tech.
i remember a poc that showed using the phones accelerometer as a microphone and bypassing app permissions.
if i remember correctly it was a defcon demo and the guy ended up getting hired by facebook.
That was likely Gyrophone (2014) from Stanford/Rafael researchers who demonstrated accelerometers and gyroscopes could capture speech frequencies at 80-250Hz - sufficient for speaker identification and some speech recognition without requiring microphone permissions.
Thanks, do you know if the claim about members of that team winding up at metabook is true?
Feels more like an advertisement for the radar than a practical application.
We already have highly accurate sensors for detecting vibrations a few feet away — EARs!
Seems like laser bounce on the phone’s glassy screen/back would be more effective…