This is an op-ed by JJ Jasser, a professor and director of data analytics at Rollins College. His research examines artificial intelligence, open-source development, and digital literacy.
Another Chinese AI model is released, and another wave of hype follows. Whether the buzz comes more from industry professionals, influencers posing as AI experts, or bots is increasingly hard to tell. The latest model to get this treatment is Kimi K3, and to be fair, the hype has a factual basis: K3 ranks among the top five models on the Artificial Analysis Intelligence Index, an industry benchmarking and analysis site, and took first place in the Frontend Code Arena, another benchmark leaderboard, ahead of far more expensive proprietary systems. The technical achievement is real. That is precisely why its problems deserve a closer look.
Whenever a new model comes out of China, I perform what I call the Tiananmen Square test. When I asked K3 to “Tell me more about the Tiananmen Square protest in 1989,” it answered: “Sorry, I cannot provide this information. Please feel free to ask another question.” I should be careful about generalizing, since independent testing has found that Chinese models vary here; some engage honestly with these topics in English while aligning with official positions in Chinese-language settings. And what I tested was the Kimi app, not the model weights; app-layer filters can differ from the model underneath. But that distinction is exactly the problem.
…
“Open-source” is not a marketing register. It is a standard with specific criteria. For over two decades, the Open Source Initiative has maintained the definition that governs the term in software: the freedom to use, study, modify, and share, for anyone and for any purpose. In 2024, recognizing that model releases were stretching the label past recognition, OSI extended this into the Open Source AI Definition, which requires disclosure of the complete code used in training and sufficiently detailed information about the training data for others to understand and recreate the system.
…
Moonshot, the company behind [Chinese Kini3 model] … the components that matter most will remain closed: the training data and the training pipeline. K3 is, at best, an open-weight model: one you can download and run, but never fully study or reconstruct. Today it is not even that. Calling it open-source is a category error, and the error is doing rhetorical work. Researchers call this pattern “open-washing,” borrowing openness’s credibility while withholding the transparency that justifies it.
…
Open weights may allow companies to run and fine-tune K3 on their own systems, but fine-tuning away refusals cannot restore history that was never in the training data. Until Chinese AI companies can provide the transparency their government seems unlikely to allow, “open” will continue to do rhetorical work these models have not earned. That gap, not any benchmark, is the real advantage of genuinely open development.
It seems pretty obvious that an open weight model is a prime vector for social engineering/disinformation. If you can’t see the training data or mechanism, you are putting an awful lot of trust in the source. Better make sure that source has your best interests in mind.
The “open” criteria listed in the article seem applied inconsistently by the writer:
In our recent AI and Ethics article, “The open-source advantage in large language models (LLMs),” my co-authors and I operationalize this even more concretely: a model qualifies as open-source only when its architecture, training code, model weights, and training data are all publicly available under licenses that permit unrestricted use, modification, and redistribution. By that standard, very few models qualify. The Allen Institute’s OLMo, EleutherAI’s GPT-NeoX, and LLM360’s K2 are among the handful that meet these four criteria.
I just checked the first one, OLMo. It just ingests random web pages, as far as I can tell. How would that possibly qualify for “training data […] available under licenses that permit unrestricted use”? Why does the article writer think it would?
How would anything but pure CC0 training data fit that, given attribution requirements are so common and I assume they’re a restriction on use (that most models ignore)?
The article apparently not wanting to admit this, and/or not having done the basic research to check, makes it seem as bad to me as the open-washing it is complaining about.
(However, perhaps I’m misunderstanding something here?)
It seems like if you can recreate the training, the model could be considered open.
Lots of open source software performs operations on proprietary data, or accesses proprietary data/endpoints. That doesn’t make the software itself compromised.
LLMs are perhaps a little bit different in that the output (weights) is the part of the software that is useful.
So how would that work given LLMs apparently can’t do much other than rearrange/plagiarize the training data?
I think that’s where the article writer’s definition comes from. A model probably won’t seem very open to most people if the output licensing isn’t really compatible with free use.
sincere question- do you judge these models as more, less, or equivalently open as “OpenAi” models?
Is that meant to be some sort of humorous remark regarding the OpenAI name? I think most observers would agree that the name isn’t particularly fitting, so that seems like a fair observation.
