I discovered a sound that leaks your exact location (and it's everywhere) — Transcript
Full transcript
- 0:00I have found awkward structures in the
- 0:01middle of a rainforest, things 600 m
- 0:04deep in the ocean, and some stuff I
- 0:06can't even share on YouTube. But, what
- 0:08I'm going to share today is, no
- 0:09exaggeration, the single most powerful
- 0:12geolocation clue I've come across. And
- 0:15to illustrate this, I'm going to
- 0:16geolocate this.
- 0:19It's a video, but nothing really changes
- 0:21visually. If you want to pause and try
- 0:22it yourself, it's linked in the
- 0:24description. The only information you
- 0:26have is [music] that it was filmed
- 0:27September 8th somewhere in the
- 0:29continental US.
- 0:31That's all you get. But, with that
- 0:33information,
- 0:34>> [music]
- 0:34>> it is possible.
- 0:35And I'm going to show exactly how.
- 0:41So, I did a video geolocating with
- 0:43thunder sounds. The benefit is that they
- 0:44travel really far, and they penetrate
- 0:46structures extremely well. The downside
- 0:48is that we were only able to get about 5
- 0:50km precision.
- 0:51>> [music]
- 0:51>> That could tell you basically what city
- 0:53it was filmed in, and maybe what side of
- 0:55town. What we're using today has those
- 0:57same benefits,
- 0:58but here's the precision.
- 1:06So, I'm going to play you two sections
- 1:08of audio. One has the information
- 1:09required to find this location, the
- 1:11other does not. See if you can tell
- 1:13which one does.
- 1:25You probably couldn't tell because it's
- 1:27completely inaudible. But, if I open
- 1:29this in a spectrogram, here we can see
- 1:31all the frequencies. So, if you're not
- 1:33familiar with a spectrogram, the X axis
- 1:35is time, and the Y axis is frequency.
- 1:38Now, even with this, you still really
- 1:40can't see anything. It's only when I
- 1:41zoom into the right frequency, and then
- 1:44increase the resolution, that you start
- 1:46to see it.
- 1:47I'll do one more.
- 1:50This is a Doppler curve from an
- 1:52aircraft, and it's what we're going to
- 1:54use to locate this video.
- 1:57Now, fair warning, this is insanely
- 1:58powerful, but it also requires a lot of
- 2:01complex stuff. This took me months to
- 2:03build, over 5,000 lines of code. I was
- 2:05working with 50 GB parquet files, 3D
- 2:08physics modeling, country wide DEMs, and
- 2:10so much more.
- 2:12That's all to say, this is going to be a
- 2:13long video, but I'm going to explain
- 2:15every step in a way that anyone can
- 2:17understand it, even if you know nothing
- 2:19about any of this stuff. So, with that
- 2:20out of the way, let's look at the rest
- 2:21of this audio. We have one Doppler curve
- 2:24here. If I keep going to the right,
- 2:26you can see another one. It's a lot
- 2:28louder, and you can also see the
- 2:29harmonics on top.
- 2:31I'll zoom out a bit.
- 2:33I think we have one more here.
- 2:35So, these two are what we'll be using.
- 2:37But, let's take a step back. Why do we
- 2:38even care about Doppler curves? What are
- 2:40we going to be using these for? So,
- 2:42there's something called ADSB data, and
- 2:44it's data on pretty much every plane in
- 2:46the sky. It's position, speed, altitude,
- 2:49and a bunch of other things. So, my
- 2:51first thought was, could I narrow this
- 2:52down to only places that heard a plane,
- 2:55and then 920 seconds later, heard
- 2:57another one?
- 2:58To do that, we need all this ADSB data
- 3:01first.
- 3:02They have an API you can pay for.
- 3:04There's also a Python package just to
- 3:06scrape the data, but it's massive. So, I
- 3:08ran this for a few hours to get all the
- 3:10flights in the continental US on
- 3:12September 8th. Now, here's what that
- 3:14data looks like. By default, it
- 3:16downloads the entire flight path. That's
- 3:17why you see things going to Hawaii over
- 3:19here or other countries. But, anything
- 3:21that flew over the US on that day is in
- 3:24this data set, including the two planes
- 3:27in our recording. So, now we just need
- 3:29to find them. So, like I said, I want to
- 3:31narrow down to the locations that heard
- 3:32a plane, and then 920 seconds later,
- 3:35heard another one. Now, these Doppler
- 3:36curves are made by a plane, of course,
- 3:39but specifically, a propeller plane. I
- 3:41boosted some frequencies so you can hear
- 3:42it better. Let me play you a bit of the
- 3:43louder one, and you can hear pretty
- 3:45intuitively that it's a propeller plane.
- 3:55One of the fields in the ADSB data is
- 3:57type code. Now, this doesn't say
- 3:59propeller versus jet plane, but I did
- 4:01some research, found this lookup table,
- 4:03and this does. Now, there's some missing
- 4:05data in that lookup table, so instead of
- 4:06filtering to the propeller planes, I
- 4:09just filtered out all the known jet
- 4:10types. But, after doing that, the map
- 4:13looks like this.
- 4:16But now, time for this time offset
- 4:18filter. So, I'll do this on a small area
- 4:20to illustrate first. This pink line is
- 4:22the ADSB data for one flight. You can
- 4:25see the individual flight points here,
- 4:27where we have coordinates, altitude,
- 4:29timestamp, things like that. What we do
- 4:31is lay a 200-m grid of points, and then
- 4:34for each point, note the time that it
- 4:36would have heard this flight. Now, it's
- 4:37time for our first new terminology, CPA.
- 4:41Not that one.
- 4:43Closest point of arrival. So, if we're
- 4:45an observer, say, here, CPA is where the
- 4:47plane is closest to that observer. The
- 4:50reason that's important is when we say
- 4:52heard a plane 920 seconds later, hearing
- 4:55a plane is not really a single point in
- 4:57time. If you're standing at this point,
- 4:59you hear the plane here, and here, and
- 5:02here. This whole Doppler curve is almost
- 5:042 minutes. So, what we need to use is
- 5:05the CPA time, which is roughly the
- 5:07middle of this. It's when the plane was
- 5:09closest and loudest. So, for this
- 5:11flight, lay a 200-m grid of points. I
- 5:13did a pretty conservative 10-km radius.
- 5:16Like, beyond that, you probably wouldn't
- 5:17hear a plane even in like ideal
- 5:19conditions. So, that's just make sure we
- 5:20don't miss anything. And then for each
- 5:22one of these points, we note the CPA
- 5:24time for this flight. Two important
- 5:27things here. First is that CPA time
- 5:29actually means the time when the plane
- 5:31was closest. For this example, that
- 5:33would be probably this point, so
- 5:3522:46:36.
- 5:38That's actually not what we need,
- 5:39though. Now, for the second terminology,
- 5:41what we need is the delayed CPA time.
- 5:44It's the time that the CPA would have
- 5:46been heard by an observer. And that's
- 5:49cuz the sound has to travel from here
- 5:51all the way 3-plus kilometers to a
- 5:53listener. That was the first important
- 5:55thing. The second is that planes
- 5:58are in the sky. Meaning, the distance
- 6:00that we use in this calc needs to be the
- 6:02slant distance, not just the horizontal
- 6:04distance. Luckily, the ADSB data gives
- 6:07the plane's altitude. However, altitude
- 6:10is not the height above the ground.
- 6:12Altitude is the height above sea level.
- 6:14So, to get the slant distance, we
- 6:16actually need to know the elevation for
- 6:18each grid point. And to do that, we use
- 6:20a DEM. That'll tell us at any point the
- 6:23elevation.
- 6:25So, now with that, we can actually say
- 6:27for our point here, for this flight, the
- 6:29CPA time of 22:46:34,
- 6:32but the delayed CPA time of 22:46:46.
- 6:37So, we do this for every point here, and
- 6:40that's one flight done. Then we just do
- 6:42that for all flights in the area, and
- 6:43we're left with this grid of points. Cuz
- 6:45basically, every point heard at least
- 6:47one flight, especially since I gave the
- 6:4910-km radius. But, for each of these,
- 6:52here, let me zoom in on one. For each of
- 6:53these, what I have is a list of flight
- 6:55IDs and the associated delayed CPA
- 6:57times. Basically, if you were standing
- 6:59here on this day, at each of these
- 7:01times, you could look up and could hear
- 7:02a plane. So, back to our goal. What we
- 7:05want is somewhere that heard a plane,
- 7:07and then 920 seconds later, heard
- 7:09another one. So, for each of these
- 7:10points, we then just check for that
- 7:12offset. Here's what that actually looks
- 7:13like. So, here's an example point and
- 7:15each of those delayed CPA times. And
- 7:17then we have a 920-second offset window,
- 7:19and we just look here, nothing 920
- 7:22seconds later.
- 7:26This one again, not quite.
- 7:27And then this one over here.
- 7:29There we go. Now, when we say 920
- 7:32seconds apart, it matters where we're
- 7:33measuring from. Like, are you calling
- 7:35the center here, or here, or here? I
- 7:38mean, that's like a plus or minus
- 7:395-second variance. So, that's why if you
- 7:41look closely on this, there's a bit of
- 7:43wiggle room that we're allowing for in
- 7:45that 920-second offset. But, anyway,
- 7:47that's the methodology. So, how many of
- 7:49our points survived that filter?
- 7:52It's about 35,000, and that's for this
- 7:54small test area, which is less than
- 7:560.05%
- 7:58of the continental US land area that we
- 8:00need to search. Like, just a side
- 8:01tangent, I don't think people understand
- 8:03how much land area there is in the US.
- 8:05If you divided the continental US into a
- 8:07bunch of football fields and laid them
- 8:09end to end, it would wrap all the way
- 8:11around the earth and to the moon
- 8:14and back.
- 8:16Almost 200 times.
- 8:18That's what we're searching
- 8:19for this.
- 8:21So we need to do a lot better and we
- 8:23can, but we need to know some things
- 8:25about
- 8:28All jokes aside, this is actually so
- 8:30cool. Doppler curves will give us a ton
- 8:32of additional information and actually
- 8:34don't need to teach you anything because
- 8:35you already know this intuitively. Let
- 8:37me play you two clips.
- 8:46You can already tell that clip two was
- 8:48going faster. The reason is because the
- 8:50drop in frequency was much larger and
- 8:53that drop in frequency can actually tell
- 8:54us the exact speed that an object was
- 8:56going. So assuming linear constant
- 8:59velocity, there are only two things that
- 9:00determine what this Doppler curve looks
- 9:02like. The velocity determines how big of
- 9:05a frequency drop and our CPA distance,
- 9:07remember that one, determines how
- 9:09quickly that drop occurs.
- 9:11So for the Doppler curve in our audio,
- 9:13we can fit a curve to that and know how
- 9:15fast the aircraft is going and how far
- 9:18away it was from the listener.
- 9:20So now instead of filtering to points
- 9:22that heard an aircraft and then 920
- 9:24seconds later heard another one, we can
- 9:26filter to points that heard an aircraft
- 9:28going a certain speed at a certain CPA
- 9:30distance and then 920 seconds later
- 9:33heard another plane going a certain
- 9:35speed at a certain CPA distance. That's
- 9:38a lot more constraining. So let's
- 9:40measure one of the Doppler curves. You
- 9:41can see that the velocity makes the
- 9:43frequency drop larger and then the CPA
- 9:45slant distance just changes how quickly
- 9:47that drop occurs. What you'll notice on
- 9:49this one is that it doesn't fit a
- 9:51perfect Doppler curve though, like this
- 9:53right hand side does. Left hand side,
- 9:56not so much. That'll be important later.
- 9:58But what we do in the meantime is we
- 9:59just take a pretty generous range for
- 10:01velocity so that we don't accidentally
- 10:03filter anything out. So here's kind of a
- 10:05decent fit on the right side at 100 m/s.
- 10:08I can't confidently say this plane was
- 10:10going exactly 100, but we can be pretty
- 10:12sure it wasn't going 70. Similarly for
- 10:15the time offset, there's still a couple
- 10:16seconds of like potential measurement
- 10:18error here. Like okay, best match
- 10:20151.15. I'll move to 151.13, slide that
- 10:24fundamental frequency up a little bit,
- 10:26and like yeah, that you know, that could
- 10:27work. That's just to say we still can't
- 10:29pin down the offset time precisely to
- 10:31the second.
- 10:32But we do this for event one and event
- 10:34two, and we get the measurement ranges
- 10:36for each.
- 10:37So the first thing is the speed filters.
- 10:39Here are all of the prop flights, but
- 10:41here are all the ones matching the event
- 10:43one velocity range. And then in pink,
- 10:46here are the same ones before event two.
- 10:48So now what we're looking for is a point
- 10:50that heard a green line, and then 920
- 10:53seconds later heard a pink line. But
- 10:55remember the other thing we know about
- 10:56this too, in addition to the velocity,
- 10:58is the distance away.
- 11:00So on this green line for event one, we
- 11:02don't need to lay that entire grid from
- 11:04zero to 10 km away. We know the pretty
- 11:07precise slant distance that the observer
- 11:09was from each of these.
- 11:11So here are those points.
- 11:13They're about 73,000.
- 11:15But if I zoom into any of these
- 11:16clusters, what do you notice?
- 11:20Airport.
- 11:22Another airport.
- 11:24Most of those 73,000 points are just
- 11:26near airports. And it makes sense cuz
- 11:29that's where the planes are. More air
- 11:30traffic just means higher chance that
- 11:32you happen to have two flights that meet
- 11:34those criteria. But the signal in our
- 11:36video is not only what we hear, it's
- 11:39also what we don't hear.
- 11:41There's no Doppler curve between these
- 11:43two. That's almost 15 minutes of not
- 11:45hearing a plane. You're not going to get
- 11:47that right outside of a major airport.
- 11:49And that's part of why this is so
- 11:50useful, cuz if you're near an airport,
- 11:53you hear a lot of flights, and you get a
- 11:54lot of signal. If you're not, then the
- 11:56silence itself is also a signal. So it
- 11:58really works anywhere on that spectrum.
- 12:01So what that means for our search, I'll
- 12:03take all of these 73,000 points, and
- 12:06I'll apply a silence filter. Feel free
- 12:08to pause here if you're interested in
- 12:10the specific thresholds and logic I
- 12:12used, but overall I just disqualified
- 12:13any point that would have heard a flight
- 12:15between our two events. I was pretty
- 12:17conservative with the thresholds. So, to
- 12:19recap, we took our velocity filtered
- 12:21flight segments for event one and event
- 12:23two, laid candidate points based on the
- 12:26slant distance range from the Doppler
- 12:28curve measurements. Then, for all those
- 12:30candidate points, did the time offset
- 12:32filter where that point would hear a
- 12:33flight of the right speed at the right
- 12:36distance away, and then 920 seconds
- 12:38later hear another flight of the right
- 12:40speed at the right distance away. That
- 12:42gave us this roughly 70,000 points. Now,
- 12:45adding on to all that logic, the silence
- 12:47filter where that point doesn't hear
- 12:49another flight between the two events.
- 12:51That drops the 70,000
- 12:53to
- 12:5515,000.
- 12:57Even with all those constraints, there's
- 12:59just so much land area and so many
- 13:02flights that 15,000 points still meet
- 13:05all those criteria. And usually at this
- 13:07point in a search, I would just start
- 13:08looking at some of the points, seeing
- 13:10why they don't work, and then encoding
- 13:12that logic as another filter.
- 13:14The problem here is we just have this
- 13:16picture. There's nothing else we can get
- 13:19from this. And we used pretty much
- 13:20everything we could from the
- 13:21spectrogram. The velocity, the slant
- 13:23distance, the time offsets, even the
- 13:26absence of the planes between the two.
- 13:28There's just not much more we can do.
- 13:31So, thanks for watching.
- 13:33I'm just kidding. Of course we're going
- 13:34to find it. Remember when I said this.
- 13:37So, assuming linear constant velocity,
- 13:39there are only two things that determine
- 13:41what this Doppler curve looks like. And
- 13:43this. What you'll notice on this one is
- 13:45that it doesn't fit a perfect Doppler
- 13:47curve, though.
- 13:49The reason they don't fit an ideal
- 13:51Doppler curve is because planes don't
- 13:53fly in steady, perfectly linear motion.
- 13:56They change heading, they speed up, they
- 13:58slow down, and all of that info is
- 14:00encoded in this Doppler curve. But, to
- 14:03get it out, we're going to need to go a
- 14:05layer even deeper.
- 14:09First, we looked at whether a flight
- 14:11occurred at this offset. Then, we looked
- 14:13at whether a flight of a certain speed
- 14:16occurred at this offset at a certain
- 14:17distance. Now what we're going to be
- 14:19asking is how exactly did that flight
- 14:22pass over? Now you can't go from
- 14:24spectrogram to knowing the exact flight
- 14:26geometry. Multiple things can cause
- 14:28variations in this. It could be a change
- 14:30in speed, heading, altitude, any
- 14:32combination of those. So instead of
- 14:34trying to reconstruct what flight
- 14:36geometry would have produced this, what
- 14:39we can do instead is model what the
- 14:40exact Doppler curve would have been for
- 14:43the actual flight geometry of each
- 14:45flight. We then do this for each
- 14:47candidate point on each flight path, and
- 14:49then we find the one that matches.
- 14:52So, how do we model a Doppler curve?
- 14:57The Doppler curve is a function of the
- 14:59radial velocity of the plane with
- 15:01respect to the observer. That sounds
- 15:03complicated, all it means is how fast
- 15:05was the plane moving directly toward or
- 15:08away from the observer. So if this is
- 15:10us, and we're right in front of the
- 15:11plane, plane velocity is 90, radial
- 15:13velocity is 90. If we're directly to the
- 15:15side, plane velocity is 90, radial is
- 15:18basically zero. It's not going toward or
- 15:21away from us. So at this point, there's
- 15:23no Doppler effect. And anywhere in
- 15:24between, we can see that red line
- 15:26representing the radial component of the
- 15:28plane's velocity. You can see here how
- 15:30the Doppler curve would change depending
- 15:33on the actual flight geometry. So how
- 15:35we'd actually create this Doppler curve.
- 15:37What we want to know is what frequency
- 15:39did the observer hear over time? So
- 15:42we'll start at T equals zero or CPA. In
- 15:44this specific example, from the flight
- 15:46data, that's 7:12:30. We don't know the
- 15:49actual time from the recording, this is
- 15:50just an example to illustrate. So for
- 15:52that time, we go look at the flight data
- 15:54at 7:12:30, and we get the plane's
- 15:56radial velocity. In this case, since
- 15:59it's at CPA, the radial component is
- 16:00zero. But we then take that radial
- 16:02component and run it through the Doppler
- 16:04equation, and it outputs a frequency.
- 16:06The Doppler equation looks complicated,
- 16:08it's not really. C is just a constant
- 16:10for the speed of sound. V sub R, that's
- 16:12the radial component that we just
- 16:14calculated. The only other term in here
- 16:16is F sub zero, that's the fundamental
- 16:18frequency. And that just comes from here
- 16:19where we measure the curve. Just
- 16:21remember how we slide this up and down.
- 16:23So that's all the components. It outputs
- 16:25130 hertz and then we plot that for 7 12
- 16:2830. Then we just do the same thing again
- 16:30for 7 12 35. Check where the plane is,
- 16:33get its radial velocity, 62.7, run that
- 16:36through the Doppler equation, and get
- 16:38109.9, and then plot that there. If you
- 16:40do that for all these points, you end up
- 16:42with the Doppler curve for this
- 16:44observer.
- 16:45However,
- 16:46that's actually wrong and it would make
- 16:48our search fail entirely. I'll give you
- 16:50a second to pause and think about why.
- 16:52The reason is we want to model the
- 16:53frequency heard by the observer at each
- 16:56of these times. So the frequency heard
- 16:58by the observer at 7 12 30 is not based
- 17:01on the plane's radial velocity at 7 12
- 17:0330.
- 17:05It's actually based on the plane's
- 17:06velocity when it produced the sound that
- 17:09we then hear at 7 12 30. So the correct
- 17:11flow is for t equals zero, find where
- 17:13the plane was when it produced that
- 17:15sound, take that radial velocity, run it
- 17:17through the Doppler equation, and then
- 17:19plot that frequency. And so here's what
- 17:21that looks like.
- 17:23Now if we just do that to the end, we're
- 17:24left with two curves. One is the wrong
- 17:26way, one is the right way. And you might
- 17:28just think, "Okay, sound takes time to
- 17:29travel, so just delay the curve. What's
- 17:31the difference?" It's not just the same
- 17:34curve delayed in time. See if I match up
- 17:36like this point, very different curve.
- 17:39Same thing over here, match up that
- 17:41point. So that first methodology didn't
- 17:43just make the right curve, but a few
- 17:45seconds late. It made an entirely wrong
- 17:47curve. Here's an actual example. So this
- 17:49is our spectrogram. This is what that
- 17:51wrong curve would have plotted versus
- 17:53this is the right methodology.
- 17:54So that's how we model this Doppler
- 17:56curve for a given observer. We then just
- 17:58repeat this process for all the
- 18:00candidate points and all of the flights.
- 18:03But what we're left with from that is
- 18:05the exact Doppler curve that each point
- 18:08would have heard based on real flight
- 18:10geometry. So then we have all these
- 18:12curves. The question is, how do we score
- 18:14them? So we take the Doppler curve, we
- 18:16plot it on a spectrogram, and then based
- 18:18on how close it looks, I raise up a
- 18:20scorecard, kind of like gymnastics. I'm
- 18:22just kidding. We do plot it on a
- 18:23spectrogram, but we just sum the
- 18:24brightness values that each curve
- 18:26covers. There's two things we have to
- 18:28account for, though. One is that the
- 18:30fundamental frequency that we used to
- 18:31model this curve was based on a
- 18:33measurement here. So, that could
- 18:35introduce some source of error. The
- 18:37second is, similarly, the CPA time is as
- 18:40well.
- 18:41So, the first thing, we have this
- 18:42modeled Doppler curve. How do we know
- 18:44where to put it on the spectrogram for
- 18:46scoring? Well, for the modeled curve, we
- 18:48have a delayed CPA point. That's just
- 18:50from flight geometry. But, remember, we
- 18:52don't know the time of filming. So, we
- 18:54could know the delayed CPA time for this
- 18:56flight, but on our spectrogram, we have
- 18:58no idea where that is. But, we do have
- 19:00the CPA time in terms of just the
- 19:01timestamp of the audio. So, that lets us
- 19:04say, "Put the delayed CPA time right at
- 19:06this timestamp." But, back to what I was
- 19:08saying, that is a measurement, and it's
- 19:10a source of error. So, the way we
- 19:11account for this is when we place the
- 19:13curve on the spectrogram, we do a sweep
- 19:16of a few seconds on both of those
- 19:18variables that we measured. So, the
- 19:20slight time offset, and then same thing
- 19:21for the fundamental frequency. And then
- 19:23we just take the highest scoring
- 19:25combination of those. That's how we
- 19:27score event one here. Event two is a
- 19:29little bit different. We still do the
- 19:30fundamental frequency sweep, cuz that is
- 19:32still a measurement. However, event
- 19:34two's temporal placement relative to
- 19:36event one is entirely determined by the
- 19:39flight geometry. Meaning, if we correct
- 19:41for that error, when we place event one,
- 19:43and event two is still placed at a bad
- 19:46offset time, that genuinely just means
- 19:48that this is not a good candidate. And
- 19:50we want to capture that, too. So, that's
- 19:52the combined locked scoring methodology,
- 19:55where the offset between these two,
- 19:57because it's determined only by flight
- 19:59geometry, is entirely locked. We correct
- 20:01for measurement error on event one, and
- 20:03then event two falls wherever it does.
- 20:06That's how we eliminate any potential
- 20:08measurement error, but still get really
- 20:09precise temporal offset scoring. The one
- 20:12downside of this is that 200-m grid
- 20:15resolution that we've been using is just
- 20:17a little bit too coarse for this. Like,
- 20:19the actual correct candidate could be up
- 20:21to 141 m off, and like kind of score
- 20:24pretty bad. That's good because it means
- 20:26it's really precise, but it also just
- 20:28means we have to take our 200-m grid
- 20:30points and expand them into 100-m
- 20:33resolution, and just score more points.
- 20:35All of this is done for one candidate
- 20:38point. So, we need to do this for the
- 20:3915,000 times whatever to make them 100-m
- 20:42resolution, so probably like 30, 40,000.
- 20:45So, super computationally expensive, and
- 20:48that's why you see all the steps we did
- 20:49prior to this to narrow down are still
- 20:51extremely important. Like you just could
- 20:53not do this on the full US for the full
- 20:56day. So, that's the entire methodology.
- 20:58I ran this for all the points.
- 21:01Let's take a look at the results.
- 21:03So, each candidate point has an event
- 21:05one score and event two score. So, I
- 21:07placed them on the scatter plot here.
- 21:08Higher scores are better, so we're
- 21:10looking for things in the top right
- 21:11quadrant. So, there's some high-scoring
- 21:13ones here, but there's not really one
- 21:14that stands out a ton. But, for each
- 21:17combination of flight IDs, there's kind
- 21:19of a cluster of points, and really
- 21:20what's in each cluster, what we care
- 21:22about is the highest-scoring point. So,
- 21:24I can do basically that on the scatter
- 21:25plot, group them by flight pair, and
- 21:27then just take the highest-scoring point
- 21:29for each flight pair. And now you can
- 21:31see there is one that scores a lot
- 21:33higher. And I'll show you the actual
- 21:35spectrograms and modeled curves for a
- 21:37couple of these top ones. So, this
- 21:39highest-scoring point with 20.48,
- 21:41that's this one here. For event one, I
- 21:44mean, you can't even see the Doppler
- 21:45curve because it's perfectly covered.
- 21:47And then for event two, very similar.
- 21:50But, let's also look at the second
- 21:51best-scoring one so you kind of see the
- 21:53difference between a 20 versus a 15
- 21:55score. So, for event one, it's pretty
- 21:57close.
- 21:58It misses this top bit right here, as
- 22:00well as being a little bit off in the
- 22:02center. But, I mean, as far as event
- 22:03one, this is a 9.5. This is a 10.6. What
- 22:07really brings down the score is its
- 22:08event two. And you can see its event two
- 22:10is pretty badly off. But, this is the
- 22:13second best-scoring point. It's kind of
- 22:15similar for three.
- 22:16Worse on event one, little bit better on
- 22:18two. Like none of these are even close.
- 22:20And then first place is spot on. So,
- 22:23let's Let's look at this point in QGIS.
- 22:25So, here are the scored points. Now that
- 22:26I know the flight IDs though, what I can
- 22:28do is run much finer resolution with
- 22:31just that flight. So, like here's the
- 22:32heat map when you just score event one.
- 22:35And then for event two, you can see it's
- 22:36flight path and it's heat map. And then
- 22:38combined we have this.
- 22:41But all of this has just been dots and
- 22:42curves and scores. We still haven't
- 22:45found this location.
- 22:47I want to do one more thing before we
- 22:48hop in a street view because this is
- 22:49probably the most insane part of the
- 22:52entire project. Do you remember this
- 22:54Doppler curve? The one that you couldn't
- 22:55hear even when you were listening for
- 22:57it. What if you just had this one-minute
- 23:00clip, but you knew the time that it was
- 23:01recorded? Could you still find this?
- 23:04So, say I told you this was 8:37 p.m.
- 23:07UTC. We could do the same spectrogram
- 23:09measurement, get the velocity bounds for
- 23:11the flight, but then also filter down to
- 23:14only the segments at that time. So,
- 23:16that's the view I have here.
- 23:18So, here're the scores for that. Since
- 23:19it's only one flight, there's no scatter
- 23:21plot, it's just a histogram. But
- 23:23right-hand side is better. And here's
- 23:25the view if you only take the highest
- 23:26scoring point per flight. So, we have a
- 23:28couple high scoring points here. And
- 23:30here's the spectrograms.
- 23:32I did one thing slightly different on
- 23:33this, which is because this Doppler
- 23:36curve is so faint, I just kind of traced
- 23:38it with a line and then scored each
- 23:40candidate's curve based on how close it
- 23:41was to that line rather than based on
- 23:44the actual brightness. So, the top
- 23:45scoring point spot on. Second place is
- 23:47close, but I mean you can definitely see
- 23:49where it's off. And beyond that it gets
- 23:51pretty bad. But what you might notice
- 23:53about this coordinate, if we go back to
- 23:55our original two event scoring, the
- 23:56highest scoring point 40.87
- 23:5940.87
- 24:01-73.5 -73.5
- 24:05The same location scoring first by a
- 24:07significant margin across two different
- 24:09methodologies with entirely different
- 24:11flights. And if I were to generalize
- 24:13this, you can kind of pick two things
- 24:15here that are bad. So, in our first
- 24:16example, we didn't know what time the
- 24:18video was filmed or where in the US, but
- 24:20we had two flights. In the second
- 24:22example, we only had one flight. We
- 24:23still did the full US search, but we had
- 24:25a much tighter temporal constraint. So,
- 24:27you kind of just need one of these
- 24:28things to be good. The thing is though,
- 24:30I've never had a search where all I knew
- 24:33was that it was just somewhere in the
- 24:34US. You typically know a city or a state
- 24:37or even okay, I'm looking north at some
- 24:39body of water. Like I chose this picture
- 24:41just to demonstrate the extremes of like
- 24:43even knowing absolutely nothing, you can
- 24:45still find the location. If you knew the
- 24:47city, a city is like 0.01% of the land
- 24:50area. So, when you constrain the spatial
- 24:52bounds that much, it allows you to open
- 24:54up the temporal bounds and you don't
- 24:56need to know the minute or the hour or
- 24:57even the day of filming. I just want you
- 24:59to grasp how powerful this is. We got
- 25:02the location of that recording within a
- 25:04couple hundred feet from one minute of
- 25:06audio where you cannot hear the plane.
- 25:09And I mean, these flyovers happen all
- 25:11the time and those frequencies go
- 25:13through walls. Like any video you take
- 25:16could have that in there. But anyway, we
- 25:18still have not found this exact photo.
- 25:20So, let's go back here in QGIS. I'll
- 25:22zoom into kind of the center of this. I
- 25:24mean, it's honestly in water, so it'll
- 25:25be like right on this bank. Grab this
- 25:27coordinate and let's take a look in
- 25:28Google Maps. And let's see what we got
- 25:30for street view.
- 25:33All right, let's zoom in here. So, what
- 25:34we're looking at is a super pixelated
- 25:35tree line. And we kind of want this
- 25:37V-shaped tree followed by a point and
- 25:40then this big rounded part. So, from
- 25:42this location, nothing really that way.
- 25:45I'm not seeing any V-shaped trees here.
- 25:50Here's kind of one, but we need it to go
- 25:51up into the right.
- 25:54There's one more here where we have this
- 25:56V-shape with a point and then this
- 25:58rounded part. Here they are side by
- 26:00side.
- 26:01And if I zoom out and look at the top of
- 26:03this pole, here is the exact security
- 26:06camera
- 26:07that took this footage.
- 26:09So, how do you protect yourself? Well,
- 26:10the first thing people might think is a
- 26:12high pass filter. There are two reasons
- 26:14that this doesn't work. First, the
- 26:16frequencies of these Doppler curves are
- 26:18pretty much the same frequencies of the
- 26:19human voice. You'd be removing a good
- 26:21chunk of what you actually want to
- 26:23record.
- 26:24Second, depending on how you high pass,
- 26:26it might not even work anyway. So, say
- 26:28this is my recording. I have the Doppler
- 26:30curve here. I'll apply the high pass and
- 26:32cut off the Doppler curve.
- 26:33I'll export this as an MP3, open that
- 26:36MP3,
- 26:37and the Doppler curve is completely
- 26:38gone.
- 26:39Now, let me just boost all the
- 26:41frequencies, and the curve is back.
- 26:45The next thing you might think, "Well, I
- 26:46would be talking during my video.
- 26:47There'd be other loud things going on,
- 26:49so the mic would pick up on that, not
- 26:51background plane sounds."
- 26:53What if I told you for the past 15
- 26:55seconds, I've had a JBL speaker across
- 26:57the room playing a Doppler curve?
- 27:03So, high pass doesn't work. You can't
- 27:04drown it out. One thing that works is
- 27:06just making cuts. That way you can't use
- 27:08the time offset between these.
- 27:11But, remember when we found the location
- 27:12with just 1 minute of audio? If it's a
- 27:14large search area, you might need to
- 27:16know the pretty precise time of
- 27:17recording. And you might think, "When do
- 27:19you ever get to know the exact time of a
- 27:21recording?"
- 27:22How about any phone call you've ever
- 27:24had? But, phone calls compress audio.
- 27:27They apply noise reduction. Surely a
- 27:29Doppler curve won't survive that.
- 27:32>> But, now that I'm listening to another
- 27:34phone speaker under a bunch of squish
- 27:36balls,
- 27:37I'm in a different room with that same
- 27:38JBL speaker playing Doppler curve.
- 27:41Just to verify there's no leakage, I'm
- 27:43going to go on mute.
- 27:48So, with the Doppler curve is going
- 27:49through the phone and only through the
- 27:51phone, what you should have seen there
- 27:52was a gap in Doppler curve. And then,
- 27:55when I started talking again,
- 27:56it came back.
- 27:59>> So, high pass doesn't work. You can't
- 28:00drown it out, and it can survive phone
- 28:02compression. So, what can you actually
- 28:04do?
- 28:05Like I said, you could make cuts in your
- 28:07videos or apply some additional noise
- 28:09cancellation. But, look, I'm not trying
- 28:10to fearmonger with this video. The odds
- 28:12that someone can and does actually do
- 28:14this is pretty slim. In reality, there's
- 28:16probably a hundred easier ways to find
- 28:19you than this. The one real use case I
- 28:21could see, I mean, the audio we used is
- 28:23just from some random security camera.
- 28:25But, it could have just as well been
- 28:27footage from a
- 28:28investigation or a human
- 28:30video online. Something with literally
- 28:33zero visual clues could be located using
- 28:36this.
- 28:37And maybe that makes you feel
- 28:38optimistic, maybe a bit uneasy, or maybe
- 28:42like me you just find all this
- 28:43fascinating.
- 28:44That when you hear a plane fly over,
- 28:46you're probably in the only place on
- 28:48Earth hearing that exact curve.
- 28:51And all the information needed to
- 28:52recreate that
- 28:54exists.
- 28:55And 30 minutes ago
- 28:57all of this
- 29:00was just background noise.
About this transcript
This page contains the full transcript of I discovered a sound that leaks your exact location (and it's everywhere) by colsto, generated from the public captions YouTube serves with the video. The transcript has 5,635 words across 840 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.