Grant Panels Fund Data Sharing Less Than Their Own Scoring Rubrics Assume

Grant panels routinely score data sharing as a high priority, yet the awards they fund rarely require it in practice. This piece settles a narrow question: what procedural choices in review and oversight produce that gap, and what would it take to close it.

The Rubric Versus the Ledger

On paper, data sharing looks like a settled norm. Funding agencies such as the National Institutes of Health and the National Science Foundation have public policies encouraging or requiring data management plans. Review rubrics often list data sharing as a scored sub-criterion, sometimes weighted alongside broader impacts or innovation. The signal to applicants is clear: sharing matters.

The ledger tells a different story. A scan of public award databases suggests that only a minority of funded proposals include explicit, enforceable data-sharing commitments. Exact rates vary by field, but the pattern holds across several directorates. Reviewers score the box, then fund proposals that leave sharing vague or absent.

Some science-policy analysts call this symbolic compliance. Others argue the gap is a predictable result of how panels are composed and what they are asked to judge. The disagreement is not about whether sharing is good; it is about whether current review structures can deliver it.

How Panel Scores Get Built

Most panels use a 1–5 or 1–9 scale for overall merit, with sub-scores for specific criteria. Data sharing typically appears as one sub-criterion among many, often worth a small fraction of the total. Reviewers are asked to rate it based on the proposal text, which may include a data management plan of one or two pages.

Scores are averaged across panel members. A single low score on data sharing rarely sinks a proposal if the overall science is strong. Reviewers may lack expertise in data curation, so they default to generous scores or skip the sub-criterion entirely. No audit follows the award to check whether promised sharing actually happens.

The result is a scoring system that rewards the appearance of sharing without creating a mechanism to enforce it. This is not unique to data sharing; similar gaps exist for mentoring plans and broader impacts. But data sharing is unusually easy to promise and hard to verify.

Evidence From Funding Records

Public award databases at NIH and NSF allow keyword searches of abstracts and project descriptions. A rough count suggests that around 10–20% of awards in some fields mention data sharing explicitly, though this varies widely. Many proposals use boilerplate language that commits to sharing "upon request" or "in accordance with funder policy."

These phrases are difficult to enforce. They do not specify a repository, a timeline, or a license. When researchers move institutions or lose funding, the data often becomes inaccessible. A related piece on this site argued that brain imaging findings shrink when pipelines share preprocessing code, a reminder that sharing practices shape reproducibility.

The evidence is not definitive. Database searches capture only what is written in public abstracts, not what happens after award. Some funders require separate data management plans that are not posted publicly. The true rate of sharing may be higher or lower, but the visibility gap is real.

Why Reviewers Skip the Box

Three procedural factors push data sharing to the margins. First, most grants have no budget line for data curation. Reviewers see a promise to share but no money to do it, so they treat it as aspirational. Second, panels rarely include a data specialist who can judge whether a plan is credible.

Third, sharing is not tied to renewal or future funding. A researcher who never shares data faces no consequence at review time. Career incentives favor generating new data over curating old data, which is time-consuming and rarely rewarded in promotion. This site has argued that computational science's staunchest advocates fund it from software budgets, a similar workaround when formal lines are missing.

Reviewers are not villains here. They are asked to judge too many criteria with too little time. When a proposal is strong on innovation and weak on sharing, the path of least resistance is to fund it and hope for the best.

The Cost of Unchecked Promises

Unchecked promises waste public money. Data collected with grant funds may never be used again, even when it could answer new questions. Replication becomes harder because independent researchers cannot access the original data. This site has covered how priming effects shrink when labs pre-register the word list, showing how procedural transparency changes results.

Trust in science policy erodes when funders claim to value sharing but do not enforce it. Critics point out that panels look hypocritical: they score sharing highly, then fund proposals that ignore it. The objection from some program officers is that strict enforcement would penalize early-career researchers who lack institutional support for data curation.

That trade-off is real. A small lab may not have the infrastructure to archive data properly. But without any enforcement, the norm never develops. The cost of inaction falls on the broader research community, which loses access to data it paid for.

What Funders Should Do Next

Concrete changes could align scoring with outcomes. Each step below targets a specific procedural weak point.

• Add a dedicated data-sharing reviewer to every panel, with expertise in repository standards and metadata, so plans are judged by someone who knows what a credible commitment looks like.
• Require a two-page data management plan as a separate, scored attachment, not buried in the proposal narrative, so reviewers can evaluate it directly.
• Tie 5–10% of the budget to sharing milestones, released only after data is deposited in a public repository with a persistent identifier.
• Audit data-sharing plans at grant closeout, with a simple check: did the data appear in the promised repository, and is it usable?
• Publish panel scoring rubrics and average sub-scores openly, so applicants and analysts can see how sharing is actually weighted.

These steps cost money and time. A funder that adopts them will fund fewer proposals or spend more on administration. That is the trade-off. The alternative is to keep scoring a criterion that the system does not enforce, and to keep wondering why the ledger never matches the rubric.

Case Study: The Human Genome Project's Bermuda Principles

One historical example shows what enforcement can look like. The Human Genome Project, a large public effort to map the human genome, adopted the Bermuda Principles in 1996. These rules required that primary genomic sequence data be released publicly within 24 hours of generation. Funders enforced this by making continued funding conditional on compliance. The result was a norm of rapid data sharing that transformed genomics.

That success did not come from a scored sub-criterion alone. It came from a clear, auditable requirement backed by a credible threat: lose funding if you do not share. The Bermuda Principles were not without costs. Some labs complained that immediate release undercut their ability to publish first. But the policy held because funders treated sharing as a condition of the grant, not an aspirational goal.

Not every field can adopt a 24-hour rule. Clinical trials, for instance, involve privacy concerns that require careful de-identification. But the principle is transferable: if sharing matters, make it a condition of funding, not a box to check.

What a Realistic Enforcement Model Looks Like

Enforcement need not be draconian. A funder could start by requiring that data management plans name a specific repository, such as Zenodo, Dryad, or a domain-specific archive. The plan should specify a timeline: data will be deposited within 12 months of collection or upon publication, whichever comes first. It should also specify a license, such as CC0 or CC BY, so users know what they can do with the data.

To make this work, funders must provide resources. A dedicated budget line for data curation, even a modest one, signals that sharing is expected and supported. Some funders already allow such costs; making them mandatory would help. Panels should include at least one reviewer with data expertise, even if that means recruiting from outside the usual pool.

Finally, funders should audit compliance. A simple check at grant closeout—did the data appear in the promised repository?—would create accountability. Non-compliance could trigger a requirement to submit a corrective plan before future funding. This is not about punishment; it is about making the norm real.

The Path Forward

The gap between rubric and ledger is not inevitable. It is a product of procedural choices: how panels are composed, how criteria are scored, whether budgets include curation costs, and whether anyone checks. Each of these can be changed.

Funders that adopt the steps outlined here will likely see more data shared. They will also spend more on administration and may fund fewer proposals. That is a trade-off worth making if the goal is to maximize the value of public research investment. The alternative—continuing to score sharing highly while funding proposals that ignore it—erodes trust and wastes data. The choice is theirs, but the evidence points one way.