Skip to main content

The problem with publication 'On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?'

Publication On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? caused some stir. I understand frustrations, etc. I understand the publication is trying to point out what can go wrong. I just do not like the way it was done.

From the paper:

"In the case of US and UK English, this means that white supremacist and misogynistic, ageist, etc. views are overrepresented in the training data, not only exceeding their prevalence in the general population but also setting up models trained on these datasets to further amplify biases and harms."

This sentence is not supported by any data (is it really overrepresented, or is it underrepresented, or about the same?). Research publications should stick to facts or close to facts as much as possible and there is no way for me to know if this sentence is correct or not. It might be dependent also on dataset we create (as you are pointing out) and I can imagine this percentage being underrepresented as well.

If I was a reviewer of this paper, I would ask that the problematic sentence to be either modified to "this might mean" or that it would be supported by some experiment and some results showing that statement is actually correct (or direct reference for this claim would do as well). Otherwise, you are just stating something that might be not correct.

You don't have to stick to facts if you are doing theoretical physics, but even in that case you state your assumptions. In your case, you are making assumption without stating that it is only assumption (and what's worst - it might be wrong assumption). Maybe introducing your own bias? Something you probably don't want to happen (based on your own article). If someone trains language model on research papers, it might now include your own, possibly biased, sentence... Emotions shouldn't get into research papers and it just happened. Your "research" is politics, not research.

Full disclosure: I did not read whole paper due to already mentioned sentence.