Why scientists should not “contribute to” AI benchmarks

More and more AI benchmarks are now built by recruiting scientists under the banner of an open community. The process is much the same everywhere. Domain experts write questions, check that frontier models cannot answer them, revise them after each rejection, and resubmit until they are accepted. In return, they get their name on a paper. Quite a few projects go further and, from the first day of recruitment, use “this paper will go to a top journal” as their selling point.

First, to be clear: I am not telling scientists to stay away from benchmarks. Quite the opposite. I think that in the age of AI, scientists' work will increasingly shift towards designing systematic evaluation methodology and towards problem formulation. Good benchmarks need domain experts, and much good benchmark work really does take many people. I have been involved in several agent-evaluation projects myself, both as an organiser and as a participant.

What I want to discuss is where some of today's collaboration mechanisms fall short on fairness and transparency. I put “contribute” in quotation marks in the title because I have never much liked the word. It sounds like one side handing something over to the other: the project belongs to the organisers, and the participants are just there to help out. But a research collaboration should be between equals. Each person brings different expertise, and together they define the problem and do the work. No one is simply contributing to someone else's project. Once collaboration is described as “contributing”, it becomes easy to undervalue what participants put in. They have no say and no clear reward, and what they get in the end may be nothing more than a cheque that never clears.

Paid work vs. research collaboration: where is the line?

The same expert work has a market price. Look at Mercor's standing listings for physics and chemistry PhDs. The first item on the physics page reads: “Writing a problem from your own area, with the system, the assumptions, and the regime, plus the reference answer a model's attempt gets graded against.” The next two are grading model-written derivations, and flagging where a model reaches the right answer through reasoning that would not hold up. That is writing and reviewing benchmark questions, almost word for word. And the page says it plainly: past rates have run $70 to $100 an hour. The chemistry page is the same, at $65 to $105 an hour.

Handshake's AI Fellowship is also aimed mainly at master's students, PhDs and postdocs. Its FAQ gives hourly rates of $30 to $125, based on education level, expertise and project, followed directly by: “All compensation is transparent before project acceptance.” You know what you will be paid before you take the work on.

Academic benchmarks have paid at this level too. GPQA hired 61 contractors through Upwork to write and validate its questions. Each question earned a $10 base payment, plus bonuses set out in advance: one for every expert validator who answered correctly, and one for every non-expert who did not. The authors estimate an average of about $95 an hour. This is what paid work should look like: before anyone starts, the pay and the rules are clear.

So what if a project does not pay? That is not wrong in itself. In many academic collaborations, participants were never paid in money to begin with. But then the only thing the project has to offer in exchange is the value of the research itself. And “academic” cannot be just a label and a selling point. It should include at least three things.

The first is transparent rules. Who did what, what earns authorship, how authors are ordered, and who owns and uses the data should all be settled before the work begins, not decided behind closed doors by a few people once the paper is nearly written. The second is real participation. Contributors should be able to help define the problem and design the evaluation criteria, and to challenge those criteria, rather than submit questions to someone else's template. The third is growth. In a normal research collaboration, participants learn from design discussions how to define a good problem. They talk directly with senior researchers, rather than merely appearing on the same author list as them. And they come away understanding the problem more deeply. That experience is theirs to take into their next project.

Without these three, participants are simply being distilled and exploited for nothing. By “distilled” I mean this: what participants hand over is domain knowledge and judgement built up over years. That knowledge is written into questions and reference answers, becomes someone else's dataset and someone else's benchmark, and ends up as a signal for training and evaluating models. It is extracted from people and settles into other people's work, while the people who provided it are left with nothing but a paper that may never be published. The organisers, meanwhile, gain a dataset, influence, and possibly commercial value. With effort and reward this unequal, “exploitation” is not too strong a word.

Why is this kind of collaboration so fragile?

In a collaboration of several hundred people, nothing directly binds the organisers to most of the people they recruit. There is no employment relationship, so the chain of trust is fragile. Yet the criteria for judging contributions and deciding author order sit firmly in the hands of a few organisers.

Scale itself is not the problem. Collaborations in particle physics and gravitational-wave physics often have thousands of members, most of whom have no employment relationship with one another, and they settled on mature, public authorship rules long ago. LIGO's publication policy spells it out: members earn author status by devoting at least half of their research effort to the collaboration for about a year, and engineers and technicians who contributed in an important way to designing, building and operating the detectors are included too. The author list is published twice a year, on 15 February and 15 August, by a dedicated Elections and Membership Committee, and it is always alphabetical. CERN's CMS works in a similar way: after a year in the collaboration and a set period of EPR service work, members become authors automatically. As early as 2008, the IUPAP Commission on Particles and Fields stated in a report that collaborations should have clear internal rules on who is an eligible author, and that “the rules shall be public and transparent”.

The same holds in XENON, the collaboration I belong to. An Editorial Board decides on papers and authorship. Infrastructure work is counted separately from the work on a specific paper: everyone has worked on the infrastructure, so everyone signs, while the corresponding author is decided by the dedicated data analysis done for that paper. Authorship is settled before writing starts, and it is open and transparent within the collaboration.

Large research collaborations, in other words, long predate crowdsourced benchmarks, and many of them realised long ago how much open, transparent collaboration standards matter.

Red flags and thumbs up: how to tell whether a collaboration is fair

So, back to crowdsourced benchmarks. If scale is not the problem, what is? Whether a project is fair and transparent is not hard to judge, as long as you look at a few things before joining. Below are four criteria I have drawn from my own observations and from conversations with friends. They may not be entirely accurate, and they cannot cover everything, but I think they are a good starting point. They follow the life of a project: recruitment, participation, authorship, and what happens after publication. Each starts with a red flag, a warning sign to watch for, followed by a thumbs up: what a project that gets it right looks like, with a real example.

1 · Recruitment: a reward you can count on, or an empty promise?

Red flag: recruitment promises that the paper will appear in a top journal, when it has not been submitted, let alone accepted. Nobody can keep that promise, because the outcome of peer review is never in anyone's control. What makes it worse is who it is aimed at: the people who most need a top-journal paper, PhD students and postdocs. And when the promise falls through, contributors are left empty-handed, with nowhere to turn.

Thumbs up: a well-run project attracts people only with what it can actually deliver. If there is a plan to submit somewhere, it can say so honestly, but it should be clear that this is a plan, not a promise, and it should never be used as a selling point. Humanity's Last Exam is an example. It offered contributors two things: anyone with an accepted question could choose to be a co-author, and fixed prizes of $5,000 for each of the top 50 questions and $500 for each of the next 500. Authorship followed from acceptance, and the prizes had stated amounts. Both were in the organisers' own hands; neither depended on a reviewer's mood.

2 · Participation: collaborators, or labour?

Red flag: contributors have no channel into the design of the evaluation and can only complete tasks against a fixed standard. That standard is often vague and set by a handful of core authors alone. On top of their domain knowledge and time, contributors even pay for their own tokens: to show that a question defeats frontier models, they call the API or run agents again and again, and after each rejection they revise and try again.

Worse, they learn very little from it. A normal research collaboration gives participants at least some design thinking, direct discussion with senior researchers, and a deeper understanding of the problem. Here, they rarely get any of it.

Thumbs up: a well-run project lets contributors take part in design discussions, or at least challenge the standard. A submission that does not yet meet the bar gets specific suggestions for improvement and feedback on its scientific validity, rather than being closed with a line or two. And the senior domain experts who review submissions meet clear, public qualifications, such as a PhD in the field or a professorship. CritPt, in physics, does participation and feedback well: each problem is refined back and forth between the expert who wrote it and the coordinators, usually for three or more rounds and up to ten, before peer review by researchers in closely related areas. Better still, the paper says openly that over a seven-month collaboration, the experts' first-hand observations of model performance deeply transformed the design of the benchmark itself. That is the kind of influence a collaborator should have.

3 · Authorship: public rules, or decisions behind closed doors?

Red flag: authorship on these projects usually comes in tiers. Some projects divide authors into categories such as core author, contributor and data contributor. Others put everyone on a single list but mark out the core authors, and the markings themselves come in several levels. Tiers are not a problem in themselves; people really do take part to different degrees. The problem is that how the tiers are drawn is often decided among the core authors, and participants have no way of knowing. The stated criterion is usually the number of questions contributed, and sometimes there is none at all. When the rules are not public, the outcome may come down to personal ties or shared interests with the organisers, or to the prestige of one's institution. Author order may even be changed at will, without consulting the people affected.

Thumbs up: a well-run project publishes its roles, criteria and responsibilities on its website from the start. Most important of all is how core authors themselves are defined and chosen. Terminal-Bench Science is an example. Its contributing guide sets out the whole structure and the path upwards at recruitment. Each merged task earns one co-authorship point, one point is enough for co-authorship, and author order follows the number and impact of accepted tasks. Top contributors are invited to become reviewers and maintainers, with a vote on approving proposals and merging PRs. Reviewers in good standing can become area chairs and lead a scientific area. Faculty who actively bring in contributors or review tasks as domain experts are eligible for senior co-authorship. Contributors, reviewers and advisors are all listed publicly on its contributors page. Everyone knows where they stand, and how to move up.

4 · After publication: your own work, or someone else's asset?

Red flag: recruitment states no licence and says nothing about how the data will be used. Once the questions are in, they may be packaged and sold as a commercial product, or provided exclusively to an AI lab or a company for internal evaluation. These decisions determine where contributors' work ends up, and contributors should be consulted beforehand. In practice the organisers often decide alone, and contributors may not even be told. Meanwhile, what rights contributors still have over their own questions is anyone's guess. Can they write a separate paper around a task they designed? Use it in teaching? Put it in another benchmark of their own? Publish it? Nobody can say.

Thumbs up: a well-run project spells all of this out at recruitment: the licence the data is released under, who will receive it, and whether commercial use is allowed. If the data may later be sold or licensed exclusively, that should be stated in advance and contributors' consent sought; contributors should share in the revenue, or at the very least be informed. Contributors also keep clear rights over their own questions: to publish their own papers around them, to teach with them, and to use them in later research. Keeping some test questions private to prevent contamination (models having “seen” a question in training once it is public) is entirely reasonable, but the project should then say under what conditions, and from when, contributors may use and publish their own questions. Agents' Last Exam does fairly well here. Its dataset is released under CC BY 4.0, and under its contributor terms, contributors grant the project a non-exclusive licence while keeping ownership of their work. Non-exclusive means the project gets the right to use the work, but the contributor does not lose their own right to use it or license it to others.

What should we do?

So what can we do? I think participants, organisers and the wider community each have something within their reach.

First, participants. Ask the organisers whether this is purely academic research, and the answer will almost always be yes. Look instead at what they have put in writing. If the project pays, the pay should match what the same work earns elsewhere. If it does not pay, check it against the four criteria above. Does it promise only rewards it can deliver? Do you have a real say in the design? Are the authorship rules public? Are the licence and the destination of the data stated? If none of this is written down anywhere, you are not “contributing”. You are working for free.

Second, organisers. Write the rules down at the start of recruitment, and make them public. Start with the core team: who is in it, how they were chosen, and what each of them is responsible for. Then what contributors get: the rewards, the roles, how to move up, and the authorship rules. And finally the data: what licence it is released under, and where it will go. While the project runs, submissions that do not yet meet the bar should get clear and appropriate feedback. Rules written down in the open are a mark of respect for contributors, and part of what makes a project credible.

Third, the community. Building benchmarks through large-scale collaboration is still a new way of doing research, and some rough edges are entirely normal. I am not analysing these patterns to point fingers. I organise benchmarks myself, and these criteria come from problems I have run into and reflected on in my own work. I hope we can face these problems together, look for better norms of collaboration, and improve today's very uneven landscape. Projects with clear rules and fair collaboration deserve to attract more scientists.

In the age of AI, deciding which questions are worth asking, and what counts as a right answer, still comes down to scientists' judgement. And that judgement is being turned, one question at a time, into benchmarks. Because it matters this much, it deserves to be taken seriously.

But evaluation is far more than designing a task and checking whether an agent completes it. Piling up questions without regard to their quality will not tell us where a system is strong, where it is weak, or how it should improve. If scientists are confined to writing questions, and never study systematic evaluation methodology or how to use it to improve systems, we are being penny-wise and pound-foolish. The loss is not only the scientists' own, but the whole field's.

Scientists' work on benchmarks should not be an unpaid “contribution”, and it should certainly not be free data labelling. It should be a collaboration between equals, where everyone brings their own strengths, and together we define the problems, design the evaluation and drive the improvement.

That is the future I hope to see: scientists are no longer just the people who supply data, and AI researchers are no longer just the people who train models. AI will become a tool in everyone's hands. Only when people from every field and background can build broader skill trees as this wave unfolds will our exploration of science and technology reach further than ever before.