Friday, 2 June 2023

ai, 'perplexity' and non-native speakers

AI is being used more and more by students: Half of College Students Say Using AI Is Cheating | BestColleges

The problem is that the systems to detect this 'cheating' are throwing out work done by non-native speakers:

Non-native English speakers are being unfairly discriminated against by AI-detectors; this is according to a study by Stanford University. James Zou, professor of biomedical data science at Stanford, has said “[Detectors] typically score based on a metric known as ‘perplexity,’ which correlates with the sophistication of the writing.” Non-native speakers are likely to score lower on perplexity, which measures lexical richness, lexical diversity, syntactic complexity, and grammatical complexity. As a result, students could face discrimination and be unfairly accused of cheating.

AI biased against TOEFL students | E L Gazette

Essays in English written by people from China were branded by text-analysis tools as being generated by artificial intelligence 61 per cent of the time

Tools to spot AI essays show bias against non-native English speakers | New Scientist

It's very difficult anyway to work out if someone's cheating:

AI detectors falling short in the battle against cheating



.

.

.

No comments: