Noam Levi

@noam_levi • 1 weeks ago

Fact-Check a Draft and Label Each Claim by Confidence

Extract every checkable claim from a draft, label it likely accurate, wrong, unsure or misleading, and get the primary source to verify each.

At a glance

Best for: Newsletter writers, marketers and small firms publishing factual content who need a first-pass check and a list of what to verify properly.
Tested on: Claude · Opus 5.5
You fill in: textcontext
You get: Claims and labels "MTD for Income Tax became mandatory in April 2026 for all sole traders." Likely wrong. My understanding is that from April 2026… (full result below)

Prompt

Fact-check the factual claims in the text below. You may not have access to current sources, so be explicit about the limits of what you know rather than sounding certain.

Text: {{text}}
Where it will be published and who reads it: {{context}}

1. Extract every checkable factual claim (numbers, dates, names, causal statements). Ignore opinions and advice.

1. Give each claim one label:

- Likely accurate: consistent with what you know, with no reason to doubt it.
- Likely wrong: say what you believe is correct instead.
- Unsure: you don't know, it may have changed since your training data, or it's too specific to know.
- Misleading: technically defensible but framed to mislead.

1. For every label, give one line of reasoning and the primary source I should check (the type of source or organisation, not a made-up URL).

1. Rank the 3 claims that would be most embarrassing to get wrong for this audience.

Never invent a citation. If a claim needs a live check (rates, thresholds, prices, current office holders, recent statistics), say so even if you think you know the answer.

Fill in

{{text}}: The draft text to checke.g. Running a small business in 2026? Here's what's changed...
{{context}}: Where it's published and who reads ite.g. Monthly newsletter from an accountancy practice to 2,000 small-business clients

Example input

text: Running a small business in 2026? Here's what's changed. Making Tax Digital for Income Tax became mandatory in April 2026 for all sole traders. The VAT registration threshold is now £90,000, so fewer small firms need to register. The National Living Wage is £12.21 an hour for workers aged 21 and over. Small businesses make up 99% of all UK businesses, yet late payment costs each of them £22,000 a year on average. And remember what Einstein said: compound interest is the eighth wonder of the world, so pay down your overdraft first.
context: Monthly email newsletter from a UK accountancy practice to about 2,000 small-business clients.

1.2k0

Generated Outputs (1)

2 hours ago

Claude

Opus 5.5

2 hours ago

Generated Output

Claims and labels

1. "MTD for Income Tax became mandatory in April 2026 for all sole traders." Likely wrong. My understanding is that from April 2026 it applies to sole traders and landlords with qualifying income over £50,000, dropping to £30,000 in April 2027 and £20,000 in April 2028. It doesn't apply to all sole traders. Check: HMRC's Making Tax Digital for Income Tax guidance on GOV.UK. Live check.

1. "The VAT registration threshold is now £90,000." Likely accurate. It rose from £85,000 to £90,000 in April 2024. "Now" implies it's new, which may mislead. Check: HMRC VAT registration guidance. Live check: thresholds can change at a Budget.

1. "So fewer small firms need to register." Likely accurate as a direction: a higher threshold means fewer firms must register, though the effect is small. Check: HMRC's impact note for the change.

1. "National Living Wage is £12.21 an hour." Likely wrong (out of date). £12.21 was the April 2025 rate. I believe it rose to £12.71 from April 2026, but I'm not certain of the figure. Check: GOV.UK National Minimum Wage rates. Live check.

1. "Small businesses make up 99% of all UK businesses." Likely accurate, roughly. Official figures say small businesses (0-49 employees) are over 99% of the business population, and SMEs about 99.9%. Check: Department for Business and Trade, Business Population Estimates.

1. "Late payment costs each of them £22,000 a year on average." Unsure about the figure, and Misleading in framing. I don't know this statistic's source, and "each of them" turns an average into a claim about every business. Many sole traders have little invoicing at all. Check: find the original survey and its sample.

1. "Einstein said compound interest is the eighth wonder of the world." Likely wrong. There's no documented evidence Einstein said it; it's widely treated as a misattribution. Check: quotation research sources such as Quote Investigator.

Most embarrassing to get wrong

1. National Living Wage. Clients may underpay staff, which is a legal breach, on their accountant's advice.

1. MTD "all sole traders". It would worry clients who aren't yet affected, and it's your core area of expertise.

1. Einstein. It's minor but easy to spot, and it undermines trust in the rest of the email.

Why it works

The four labels, especially "Unsure" and "Misleading", give the model an honest option other than sounding certain, which reduces confident errors. "Never invent a citation" plus asking for the type of source rather than a URL prevents made-up links. The live-check rule matters for anything that changes by date, such as rates and thresholds, where a model's knowledge can be out of date. Ranking by embarrassment for this audience tells you where to spend your checking time.

When not to use it

This is a first pass, not a substitute for checking the primary sources it points you to, especially for tax, legal or medical content. The model can be wrong about its own confidence. For breaking news or anything from the last few months, use a search-enabled tool or check sources directly.

Comments (0)

No comments yet. Be the first to share your thoughts!