Imagine an artificial intelligence designed to create truly original concepts – new drug molecules, innovative engineering solutions, even fresh artistic styles. Alternatively, envision an AI specifically designed to assess whether those concepts are truly new, genuinely novel. It's a critical task, because without a reliable way to judge originality, we cannot know if our creative AIs are making real progress. This is precisely where we hit a significant snag. For some time now, large language models, or LLMs, have increasingly been pressed into service as these novelty judges. They are asked to sort through ideas and decide which ones stand out as original. The underlying assumption is that these powerful models can somehow grasp the essence of what makes an idea new. But a recent, systematic study reveals a profound instability in these AI judges, exposing a serious challenge for the entire field of automated idea generation.
Noy Sternlicht and colleagues found that even tiny adjustments in the instructions given to an AI judge could drastically change its mind. Think of it this way: simply rephrasing a question to the AI, perhaps telling it that human reviewers had found one idea novel and another not, was enough to make it flip its verdict on more than half of identical idea pairs. This isn't a small margin of error; it's a fundamental unpredictability. The researchers detail these findings in "Old Ideas, Novel Problems," a preprint appearing on arXiv.
The key thing to understand here is the sheer scale of the shift. These minor changes to the wording of instructions could cause the correctness of the AI's comparisons to swing by over 50 points. Sometimes, this meant the AI's judgment became worse than simply guessing. Imagine a judge at a science fair whose opinion could be swayed so easily by how you worded your explanation. It quickly undermines any trust in their judgment.
This instability has huge implications. If the very tools we use to assess the originality of AI-generated ideas are so unreliable, then any claims about AIs making "novelty gains become highly questionable. It means we might be celebrating breakthroughs based on a shaky foundation, unaware that a slightly different set of instructions would have yielded a completely different evaluation.
To investigate this, Sternlicht’s team conducted a controlled study. They looked at six different AI judges. To create a reliable test, they constructed a unique evaluation set. They did this by looking through OpenReview, a public platform where researchers review scientific papers, and specifically identified passages where human reviewers explicitly talked about the originality of submitted papers. They then paired these real human judgments with ideas generated by a standard large language model. This allowed them to rigorously test how small tweaks to the instructions given to the AI judges affected their assessments.
The challenges don't stop at instruction wording. The researchers also explored whether giving the AI judges more resources would help. They tried providing more information for the AI to retrieve and larger "reasoning budgets—essentially, more computational power to process information and make a decision. Neither of these approaches significantly improved the performance of the novelty judges. Even more surprisingly, two specialized AI systems, specifically built to evaluate novelty, were actually outperformed by a much simpler, more basic AI judge given straightforward instructions. This suggests that simply throwing more computing power or complex designs at the problem isn't the solution.
This task of judging novelty has a history. Years ago, in fields like computational linguistics, "novelty detection often meant something simpler: identifying if a sentence was redundant compared to previous sentences in a list. It was about finding factual overlap. Today's challenge is much harder. We're asking AIs to assess the abstract, conceptual originality of complex ideas. This requires a deeper understanding than merely spotting repeated words or phrases.
So, where do we go from here? The findings make it clear that we need much more robust and transparent ways to evaluate AI-generated novelty. Some researchers are exploring structured approaches, for example, systems that break down new ideas into key components like their purpose, mechanism, or how they might be evaluated. By comparing these structured "facets to existing ideas, such systems aim to provide a more consistent and understandable basis for judging originality. This approach moves beyond simply asking an AI to give a "yes or "no on novelty and instead asks it to reason about why an idea might be new.
Until methods like these become widespread and proven reliable, we should look at claims of AI systems generating truly novel ideas with a healthy dose of skepticism. For now, the yardstick we’re using to measure AI’s creativity is far too wobbly. ◆
Sourced from: "Old Ideas, Novel Problems: The Instability of LLM-Based Novelty Evaluation" by Noy Sternlicht et al.