It’s not that complicated. Recreating training data via distillation is basically asking structured questions and recording the responses and reformatting that to use as cleaned “good” training data. Much less energy and compute intensive than creating the training data on your own.
I think I remeber reading somewhere how Chinese research’s do this by basically using bots and spreading out the distillation to many source queries.
The process takes time because even when you’re distilling answers, you still need to actually do reinforcement training on the model. And given that Fable and GPT 5.6 just came out there simply hasn’t been much time to do that. However, models like Kimi also do better than Fable or GPT on a lot of tasks, which means it’s not just distillation but also difference in architecture. You can watch this talk from Kimi founder to see how Kimi was actually trained and why it performs well.
It’s also absolutely hilarious that people think only Chinese companies use distillation, as if Anthropic or OpenAI are above that or something. Not to mention that they basically ignored copyrights on all the data the siphoned and are now crying that people aren’t respecting their terms of use.
I gotta know what you mean by “copying” ai models. Same training data? Amount of training? How would you go about copying a closed-source model. I think china is just…. Making good models.
Distillation is a technique where an older “teacher” AI model is used to train a newer, “student,” model that replicates the capabilities of the earlier system — often at a much lower cost than producing an original model from scratch. Some forms of distillation are widely accepted and even encouraged by AI labs, such as when companies create smaller, more efficient versions of their own models, or allow outside developers to use distillation to build non-competitive technologies.
Basically one AI asks questions and records the answers to recreate training data without massive compute for training data.
the Chinese would know they fucking copied it.
You can’t distill models that fast lmfao. People just keep parroting this without having any clue how any of this works.
It’s not that complicated. Recreating training data via distillation is basically asking structured questions and recording the responses and reformatting that to use as cleaned “good” training data. Much less energy and compute intensive than creating the training data on your own.
I think I remeber reading somewhere how Chinese research’s do this by basically using bots and spreading out the distillation to many source queries.
The process takes time because even when you’re distilling answers, you still need to actually do reinforcement training on the model. And given that Fable and GPT 5.6 just came out there simply hasn’t been much time to do that. However, models like Kimi also do better than Fable or GPT on a lot of tasks, which means it’s not just distillation but also difference in architecture. You can watch this talk from Kimi founder to see how Kimi was actually trained and why it performs well.
It’s also absolutely hilarious that people think only Chinese companies use distillation, as if Anthropic or OpenAI are above that or something. Not to mention that they basically ignored copyrights on all the data the siphoned and are now crying that people aren’t respecting their terms of use.
Yeah I know they’re not the only companies doing distillation, it’s just currently in the news and on peoples minds.
Cope
Not really. I’m glad that China is copying AI models and providing them for cheaper usage or open sourcing the training.
I don’t believe any US company should monopolize the entirety of human knowledge.
I gotta know what you mean by “copying” ai models. Same training data? Amount of training? How would you go about copying a closed-source model. I think china is just…. Making good models.
https://www.latimes.com/business/story/2026-04-07/china-is-copying-u-s-ai-models-american-companies-say-it-is-costing-them-billions-of-dollars
Basically one AI asks questions and records the answers to recreate training data without massive compute for training data.
I see and agree with you then. I’m glad it gets us better cheap models but more traditional innovation would definitely be preferred.