Grade Whitepaper Claims on Evidence Quality and Bias

Audits specific empirical claims in whitepapers across four objective criteria and assigns an aggregate confidence tier.

You getA Markdown table with columns and under the table

Prompt

Prompt
Act as a research methodology auditor. Review the excerpt or whitepaper text provided directly below this prompt. Extract up to 4 major factual claims made by the authors and evaluate each claim against an evidence quality rubric.

Rules:
1. For each claim, evaluate four factors on a 1 to 5 scale: Sample Size (adequacy and statistical power), Methodology (controls, blinding, or design validity), Funder Bias (commercial interest or conflict disclosures), and Replication Status (independent confirmation or precedent).
2. Calculate the average score for each claim and assign an Aggregate Tier: High Confidence (4.0-5.0), Moderate Confidence (2.5-3.9), or Weak / Speculative (1.0-2.4).
3. State the exact weakness or unverified assumption for any metric scored below 3.
4. Never extrapolate or assume methodology details not present in the text; if details are missing, score that factor 1 and note that data was omitted.

Output format:
- A Markdown table with columns: Claim, Sample Size (1-5), Methodology (1-5), Funder Bias (1-5), Replication (1-5), Tier.
- Under the table, an 'Audit Notes' section providing 2 bullet points per claim detailing specific red flags, missing disclosures, or corroboration needs.

Paste your whitepaper text or summary below:

Example output

Example for: an enterprise software vendor whitepaper evaluating workflow automation ROI.

ClaimSample Size (1-5)Methodology (1-5)Funder Bias (1-5)Replication (1-5)Tier
Agentic routing reduces enterprise triage overhead by 68%2212Weak / Speculative (1.75)
Engineers save an average of 4.2 hours weekly on code reviews3323Moderate Confidence (2.75)
Automated triage lowers ticket escalation churn to under 4%1211Weak / Speculative (1.25)
Audit Notes

Claim 1: Agentic routing reduces enterprise triage overhead by 68%

  • Sample size is limited to 14 mid-market pilot customers over a 3-week window, lacking seasonal normalization and statistical controls.
  • Funder bias is severe: the study was designed and analyzed internally by the vendor's marketing operations team with no third-party audit.

Claim 2: Engineers save an average of 4.2 hours weekly on code reviews

  • Measurement relied on self-reported developer surveys across 220 respondents rather than objective git telemetry or timestamped activity logs.
  • Replication is partially corroborated by independent industry benchmarks, though baseline review volume variations were ignored.

Claim 3: Automated triage lowers ticket escalation churn to under 4%

  • Omitted methodology: the whitepaper cites zero sample figures, timeframe markers, or baseline churn rates prior to deployment.
  • Heavy commercial interest with zero independent validation; the 4% metric appears derived from an isolated, best-case customer testimonial.
Written by AI from this prompt. Your results will vary.

Tips and common edits

Why this works

  • Explicit four-factor scoring prevents superficial acceptance of marketing-driven empirical claims.
  • Strict rule penalizing omitted data prevents the model from hallucinating plausible research methodologies.
  • Direct calculation of aggregate tiers makes it easy to compare findings across multiple vendor reports.

Paste your whitepaper excerpts or methodology sections directly after the prompt in Claude, Gemini, ChatGPT, or Perplexity to spot unsubstantiated claims.

Common edits

  • Change the four rubric categories to match clinical research standards, such as PRISMA or Cochrane risk-of-bias criteria.
  • Add an extra column in the table specifically to capture whether the original data is publicly available in a repository.

Make it repeatable

Keep using this prompt without pasting it in every time.

Keep exploring

More in AI Research