To me this points to there being a lazy trick for AI Models that precludes a more nuanced expression of reality that the AI models all find themselves at from optimizing more revealing but potentially esoteric encodings of reality away.

Researchers began to explore representational similarity among AI models with this approach in the mid-2010s and found that different models’ representations of the same concepts were often similar, though far from identical. Intriguingly, a few studies found that more powerful models seemed to have more similarities in their representations than weaker ones. One 2021 paper dubbed this the “Anna Karenina scenario(opens a new tab),” a nod to the opening line of the classic Tolstoy novel(opens a new tab). Perhaps successful AI models are all alike, and every unsuccessful model is unsuccessful in its own way.

I think it is far more likely and logical that the more you cook an AI models the more they end up looking the same, but not in a good way, rather the more you experiment with mixing too many nice paint colors together the more you end up with the same brown muddy color.

“The endeavor of science is to find the universals,” Isola said. “We could study the ways in which models are different or disagree, but that somehow has less explanatory power than identifying the commonalities.”

This is the most revealing quote of all for me, there is nothing inherent to the pursuit of science about universals, that is something that people project onto science because it is emotionally satisfying. Yes, some scientists study universal aspects, some scientists look for large general shared similarities, but the vast vast vast vast vast vast majority of scientists study how each part of the universe is unique unto itself.

This isn’t science…

Efros noted that in the Wikipedia dataset that Huh used, the images and text contained very similar information by design. But most data we encounter in the world has features that resist translation. “There is a reason why you go to an art museum instead of just reading the catalog,” he said.

Computer/AI people really seem to have a hard time taking this lesson to heart and it undermines absolutely everything they try to do that affects the real world because of it.