Watch This BEFORE Apple AirPods Pro 3 Reviews
Just like with the AirPods Pro 2, most of the AirPods Pro 3 measurements you're seeing online are wrong. The flawed testing methodology of reviewers and the lack of context on the part of consumers is fueling an endless spiral of confusion, so Listener's here to give you a way out.
When the first AirPods Pro 3 measurement graphs began circulating online, the audiophile Discord servers lit up with hot takes: “These look so much worse!” or “Apple really dropped the ball this time.”
But… are these measurements even correct?
The unfortunate truth when it comes to the AirPods Pro 3 is that most people measuring—and most people interpreting—the data we see propagated about these devices are, simply put, doing it wrong.
And even worse, this isn’t a new problem. We saw the exact same confusion when the AirPods Pro 2 launched, and this confusion persists to this day. Unless the conversation shifts, history is about to repeat itself.
The Complexity Behind AirPods Pro 2 (and 3)
To talk about why the AirPods Pro 3 measurements we’ve recently gotten probably shouldn’t be taken at face value, we first need to talk about the AirPods Pro 2.
Under the hood, the AirPods Pro 2 is one of the most advanced audio playback systems ever made. The first novel aspect happening behind the scenes is volume-dependent EQ, which boosts bass & treble at low listening levels and reduces them at higher levels—all to account for the fact that we hear more bass and treble at higher volumes, and less at low volumes.
The other, probably more important, impressive, and confusing process, is Apple’s Adaptive EQ. AirPods Pro 2 uses microphones inside the nozzle of the earpiece to monitor the sound pressure in your ear canal, and adjusts (in real time) the sound in your ear so that it matches Apple’s intended sound signature.
This adaptive system means that the AirPods Pro don’t behave at all like traditional wired in-ears. While the sound of wired IEMs can vary dramatically depending on your ear canal’s idiosyncrasies, the AirPods Pro 2 (and now 3) are actively correcting those differences on the fly, so that every single person receives essentially the same sound—under 2 kHz or so, at least.
Why Most Measurements Look “Wrong”
So if Apple is making sure the AirPods Pro 2 (and 3) measure the same regardless of the ear it’s placed in… why do AirPods Pro 2 (and 3) measurements look so inconsistent?
Well, it’s for two reasons. One, most people doing measurements aren’t actually test engineers who know or care about getting it right. But the other reason is that the measurement process itself is actually unusually tricky, especially compared to simple wired IEMs. To get reliable data, you need to account for several factors
The Wear Sensor
The AirPods Pro 2 only fully engages all of the active systems when they detect they’re in an ear. For example, if you have the wear sensor enabled, you won’t be able to engage ANC without placing the AirPods Pro 2 in a sufficiently human-like ear. Silicone test ears will not be sufficient to spoof this sensor, and simply dropping them into a coupler and pressing go won’t work either.
Reviewers have used everything from deli meats to keyboard cleaning gel to simulate the reflectivity of human skin that the earpieces expect to see. An easier solution is to simply disable the wear sensor in the AirPods Pro settings (Settings > AirPods Pro > Automatic Ear Detection), which then allows you to engage the Active Noise Cancellation mode without having to have one of the earpieces in your ear.
Priming the Adaptive EQ
Thanks to insights from community member MayaTL, we now know you have to “prime” the AirPods with a full bandwidth, music-like stimulus — such as wideband music, pink noise, or M-noise — before taking a measurement. This prepares the earpieces to output the correct frequency response that it would ordinarily be outputting with typical program material.
This is probably the most important missing link in this chain for most people measuring this device, and it’s likely the biggest contributor to people’s measurements of the AirPods Pro 2 (and 3) being wrong. Skipping this step means an incorrect result that looks wildly different from proper measurements.
The Measurement System Itself
Older rigs like the IEC 60318-4 (“711”) coupler simply aren’t as accurate when it comes to simulating the acoustic impedance of the human ear, which is why the B&K 5128 is such an important development in our space.
While yes, when measured properly the AirPods Pro 2 (and 3) completely normalize the response under 2 kHz regardless of the ear canal it’s placed in, the accuracy of the ear still matters for everything above 2 kHz as well, and this is still where we’ll need the 5128 to make heads or tails of any of the treble features the AirPods Pro 3 will have.
New Targets
Most people measuring on the old 711 systems will be familiar with Delta (∆) targets, which proliferated in late 2023-early 2024 as a means to make measurements taken on the 711 systems more compatible with the new 5128.
While the ∆ targets do a reasonable job approximating what a passive IEM may look like when measured on the 5128, these targets are entirely inappropriate for use with the AirPods Pro 2 and 3, because these incredibly clever devices from Apple already compensate for the errors of the older 711 systems, so no ∆ target is necessary.
Because so few reviewers follow all these steps, it’s safe to say that the vast majority of published AirPods Pro measurements, even from otherwise reputable sources, are flawed.
The Misinterpretation Problem
Even if the measurement itself is correct, interpreting it is another challenge. Without context, it’s easy to look at a peak on a graph and assume it will sound harsh to everyone. But that isn’t necessarily true. IEMs currently represent a much bigger unknown than headphones do when it comes to the variance factors we’ve identified and the degree to which they vary; what looks like a problem in one measurement rig might not exist at all in your ear.
This is why the AirPods Pro 2 (and likely the 3) continue to get misrepresented online. People share graphs stripped of context, jump to conclusions, and the narrative spirals.
The Solution?
Wait for the right sources: Don’t trust the first graphs you see floating around YouTube, Reddit, Head-Fi, or Discord. Look for data from experienced reviewers using state-of-the-art gear like the 5128. For reference, the only accurate measurements I’ve seen of the AirPods Pro 2 are on my own website, REALAB, RTINGS, and EarphonesArchive.
The AirPods Pro 3 may already be one of the most misunderstood headphones on the market, and it hasn’t even been released yet.
Measurement errors and misinterpretations aren’t just a possibility, they’re already happening, and they’re almost guaranteed to continue unless people start fully reckoning with the complexity of these devices.
For deeper discussion or any questions about all of this, you can hit me up on our Discord server or forum. Thanks for reading.
Full Video Transcript Below:
Okay, so today I woke up to a gazillion Discord pings all saying some version of the same thing. Oh my god, we got AirPods Pro 3 measurements. Oh my god, they're so much worse. And this is indicative of the same problem we had with the AirPods Pro 2, which we've not really talked about on this channel, but it's time to talk about it because I don't want the whole discourse around the measurements of AirPods Pro 3 being the same hotbed of confusion they were for the AirPods Pro 2. The problem is that all of you were doing it wrong. What I mean when I say all of you are doing it wrong is that both the people measuring and presenting this data as well as the people interpreting that data are making significant enough errors and lack enough context that even today even some of the nerdier places on the internet most of the people don't know how to collect or interpret these measurements. So to talk about this issue we're going to have to talk about the AirPods Pro 2 specifically because this is where a lot of this started. So, as we've said before, the AirPods Pro 2 is one of the most advanced audio playback systems on the planet. What we mean by that is that under the hood of the AirPods Pro 2, there are a whole bunch of active processes that are adaptively tuning the AirPods Pro 2 to adapt well in a certain given circumstance. The one that most people will probably be familiar with is the volume dependent equalization. So, the AirPods Pro 2 when used with an Apple device at least, if you're listening to something at low volume, it will boost bass and treble uh because we don't hear as much bass and treble at low volumes. Whereas at high volumes, it's cutting bass and cutting treble relative to that because at higher volumes, we don't need as much of the energy in those bands to actually hear those frequencies. But the other active process going on under the hood here is arguably way cooler, but it's also way more opaque and something that a lot of people don't know about or know how it works. And that process is what they call the adaptive EQ. I think in their marketing they say something like it delivers consistently excellent sound to all listeners or something like that. And yeah, it does, but that's not easy. So, let's talk about how they do that. So, the first thing people need to know about devices like this and the AirPods Pro 2 especially is that they have a microphone in the nozzle that constantly monitors the sound pressure in the ear canal that makes adjustments for a few of the processes. So, for example, like the active noise cancelling and transparency mode, they both take some of the information from this microphone pointing in towards your eardrum and use that to tune the ANC and transparency to be as effective as possible. But this microphone is also listening to the sound in your ear canal for music, movies, podcasts, whatever you're listening to and adjusts that in real time so that it hits Apple's intended frequency response or their intended sound target. For those who aren't aware, in-ear headphones with wires and stuff don't do this at all. So, when you put a typical wired in-ear headphone into two very different ear canals, say like the IEC60318-4 coupler that's been used for years and years as a simulator for testing how IM sound and the new 5128, which is a lot more advanced, a lot more humanlike, you'll see that a wired IM differs extremely widely under a certain frequency around 2 kHz, let's say, whereas the Apple AirPods Pro 2 basically doesn't deviate at all. Well, okay. If Apple is tuning it to be the same frequency response under a certain frequency regardless of the ear canal it's placed in, why do so many measurements of the AirPods Pro 2 look so weird? Here's the problem. Most of the people measuring these devices either don't know that they do this or don't know how to manipulate this system to get Apple's intended frequency response. What it has functionally led to is most of the measurements out there, and I mean like 98% of the measurements out there for the Apple AirPods Pro 2 just being flatout wrong. So, let's talk about how you measure one of these devices properly, cuz there are a few things you need to account for. The first is the wear sensor. So, Apple has a uh setting in their AirPods Pro settings that is automatically detect when placed in ear or something to that effect. It basically has a sensor on the back of it that measures the reflectivity of the surface that it's rested against. So, if you just place the AirPods Pro 2 in a silicone ear like what I have here, you're not going to get the active system triggered properly. So, you actually need to put something in the ear that simulates humanlike reflectivity. Um, a lot of people used ham. We have used deli meats of various types in the past. Or you can use uh certain types of keyboard cleaner gel. I think some people also use tin foil. you need to either trigger the wear sensor or disable it. Now, the second thing is something that wasn't really known for a while until a user by the name of Maya TL came into spaces like Discord and Audio Science Review and educated people on what the active system was actually doing. What Maya would say is you need to do what's called priming. So, you need to prime the system with a music-like stimulus to prepare the AirPods Pro 2 with the right frequency response for music playback. So, you need to either feed it wide bandwidth music, pink noise, or M noise, which is a more esoteric test that kind of splits the difference. You need to feed it one of these three stimuluses to get the intended frequency response of the AirPods Pro 2. Most people haven't done this. So, most of the measurements you'll see, even from incredibly nerdy sites like Squiglink, you're not going to see the correct frequency response because most people aren't priming the device. So, because most people measuring these devices aren't considering the wear sensor and aren't accounting for the adaptive EQ's effect on the frequency response and aren't priming it to make sure that they get the correct frequency response, most of the frequency responses you're going to see out there are wrong. And this doesn't even touch on the fact that especially for the 7-Eleven equipped systems, so the older, less accurate systems, we now have a target methodology that compensates for the error of these systems. But a lot of people will evaluate the AirPods Pro 2 against these new targets, which is wrong because the AirPods Pro 2 is compensating for the deficiencies of that system. Which brings us to the second half of the problem, which is most people when happening upon a measurement of the AirPods Pro 2 don't have all of the context necessary brought to this measurement to understand how to unpack it properly. Which is why you see a bunch of people either sharing the wrong measurements, sharing the right measurements but against the wrong targets or are posting the right measurements and don't have the correct context to unpack it properly. So they say things like this 6K peak is going to be gnarly for everyone when it actually isn't. So how do we fix this? When it comes to the AirPods Pro 3, we need to wait until reputable sources who know how to measure all of this stuff and have the most up-to-date measurement gear, like the 5128, have measurements of these devices to share because frankly, that's the only way we're going to get reasonably close to how this responds in the human ear. And the last thing we need to do is just be a whole lot less certain when it comes to measurements of inear headphones. The fact is we don't know how these devices are responding in our ears like we kind of do with overear headphones. We don't know that what you see as a peak on a given measurement is actually going to sound like a peak. So just be aware that when trying to unpack a measurement from a system that's this complex, you have to be a lot more considerate and bring a lot more grains of salt to the table than you otherwise would. If you want to discuss any of the information I've talked about here or ask any further questions, reach out to me on our Discord, which will be at discord.gg. gg/adphones. You can just ping me in there. My name is listener. And you can also find me on the forum at forum.adphones.com with the same username. And lastly, I just want to give a shout out to headphones.com who makes educational content like this possible in the first place. So, if you really like educational content like this and want to see more of it, definitely make sure to consider headphones.com for your next headphone, DAC, speaker, cable, whatever purchase. With that, I think that's it. So, I'll let you guys go later.
