Article via https://www.vice.com on 02 May 2023
In a new paper, Stanford researchers say they have shown that so-called “emergent abilities” in AI models—when a large model suddenly displays an ability it ostensibly was not designed to possess—are actually a “mirage” produced by researchers.
60 Minutes claimed that AI models are “teaching themselves skills that they weren’t expected to have.”
The researchers found that AI abilities only appeared to suddenly emerge when people used specific metrics to measure the task.
A discontinuous metric is something like a “Multiple Choice Grade,” which is the metric that produced the most supposed emergent abilities.
This understanding can guide researchers in making more informed decisions about the metrics they choose to evaluate emergent abilities.”