So just after I wrote about qualia in last week's Philosophy Monday, Sabine Hossenfelder comes out with a video claiming the exact opposite citing a recent study.

In the video she says the following about the claim that qualia are a hard limit to science:

I’ve always thought this is bullshit. You can measure what’s going on in the brain, therefore you can study what experience means, in scientific terms. You cannot of course yourself experience the same as some other person, or bat, or carrot, because you don’t have the same brain. But give it some time and we can find the next closest approximation to someone else’s experience that works on your circuits.

The study Sabine cites is provocatively titled "Is my 'red' your 'red'?" which goes right at the subjective experience of color. The rest of the title is more informative though "Evaluating structural correspondences between color similarity judgments using unsupervised alignment." Now in the introduction they already soften their claim a bit by writing "Although we do not argue here whether or not there might be a purely intrinsic and incommunicable aspect to qualia, in this paradigm we assume that at least some phenomenal properties of qualia can be empirically measured." Late in the paper they go even further in walking back what the results show by writing "As we emphasized in the introduction, the structural correspondences evaluated by our unsupervised approach are not direct evidence that different people have the same subjective experiences. Rather, it should be considered as one of the necessary conditions for different people to have the same subjective experiences. For sufficiency, other conditions may also need to be met."

But when you actually read the paper the claim that is actually substantiated is something more akin to "people's perception of color is pretty similar" and also "color blind people's perception of color differs." They apply some pretty fancy methods to establish this by using an unsupervised approach that uses pairwise comparisons to establish an embedding space instead of asking the observers for color labels. This is elegant and makes use of some new insights that we are getting from machine learning techniques. But it says little about the subjective experience associated with seeing these colors.

Now there is actually a study that could be performed using this approach that might get us a bit further. Instead of color pairs, the participants would rate sentence pairs along with seeing a color where the question is "which of these sentences better captures your experience of this color." The sentences themselves would be designed to contain various emotional valences. This would come closer to creating an embedding space over subjective experiences than simply over the perception of color.

Of course I wouldn't at all be surprised if we found clusters of people based on such an approach. We have lots of reasons to suspect that our subjective experiences have some similarity to those of other people. That should be our prior based on everything we know. Our biological blueprint (DNA) is extremely similar. Our physical environment is quite similar. Even though our languages appear quite different we are finding that under the hood they share a lot of structure (again unsurprisingly). So the real surprise would be if there was no clustering in an embedding space of color comparisons or of emotions elicited by colors.

And yet despite all of this my claim (along others such as Chalmers and Nagel) still stands. Just because we can learn that there is similarity between subjective experiences among some people tells us very little. Even the extremely limited study cited above shows drastic perception differences between color aware and color blind people. So even in otherwise very similar groups we immediately run into important limits on what can be learned about subjective experience.