Well… I’d rather the dataset be public and there, with an ostensible centralized opt-out, instead of every AI startup frantically rescraping the same things their predecessors did.
True, I understand why this is better than all those companies scraping it individually (for both ability to opt out and site load), but the way they handled opt out is still quite silly.
If they only include repositories that come with a proper open source license technically they don’t even need to provide an opt-out option. So, good thing that at least it exists.
GPL is dodgy to be included in training data for AIs that are then used to generate non-openaource code tbh. Even MIT loses the attribution it’s supposed to have once laundered through AI…
Speaking as someone that’s been into FOSS for a long time, it does piss me off how quickly copyright got thrown under the bus the moment it became inconvenient for people with money.
Yes, but what HuggingFace is doing here is the distribution of a data set. And so long the data set itself is open that doesn’t conflict with GPL. HuggingFace is not responsible about how others use that data set.
Whether LLMs themselves violate copyright for being trained on MIT or GPL code is an entirely different discussion.
Well… I’d rather the dataset be public and there, with an ostensible centralized opt-out, instead of every AI startup frantically rescraping the same things their predecessors did.
True, I understand why this is better than all those companies scraping it individually (for both ability to opt out and site load), but the way they handled opt out is still quite silly.
If they only include repositories that come with a proper open source license technically they don’t even need to provide an opt-out option. So, good thing that at least it exists.
EDIT: oops replied to wrong comment
Second that. After all all my open repositories are all either licenced under GPL or MIT, so complaining would be a kind of hypocritical.
GPL is dodgy to be included in training data for AIs that are then used to generate non-openaource code tbh. Even MIT loses the attribution it’s supposed to have once laundered through AI…
Speaking as someone that’s been into FOSS for a long time, it does piss me off how quickly copyright got thrown under the bus the moment it became inconvenient for people with money.
Yes, but what HuggingFace is doing here is the distribution of a data set. And so long the data set itself is open that doesn’t conflict with GPL. HuggingFace is not responsible about how others use that data set.
Whether LLMs themselves violate copyright for being trained on MIT or GPL code is an entirely different discussion.