Choosing a tool

Best tool for conflicting sources: six tests any candidate must pass

When sources disagree, a summary is not enough: a summary picks a side without telling you it did. A tool that weighs sources must show its work. Here are six requirements, with a test for each, that apply to any candidate.

1. Each claim is tied to a named source

Test: pick one conclusion and ask the tool who said it. If it cannot name the source, it is summarising, not weighing.

Arbiter: every claim belongs to a named source, and every piece of evidence to the claim it supports or opposes.

2. Sources carry different weights, and the weights are published

Test: ask why one source counted for more than another. "The model decided" is not an answer.

Arbiter: each source type starts in a published reliability band with a stated reason, from regulators at 0.85–0.95 down to known fabricators at 0.01–0.15. The full list is the source-reliability table.

3. Repetition does not count as agreement

Test: give the tool ten articles that repeat one press release. It should not report ten independent confirmations.

Arbiter: meets it when you say which sources repeat which. Each source can name the sources it depends on, such as the wire service a newspaper reprints; sources in one dependency group count once. Without that marking, Arbiter treats every source as independent. The same claim submitted twice by the same source is always dropped.

4. Old claims lose weight

Test: add a claim from years ago that newer sources contradict. The older claim should count for less.

Arbiter: claims lose weight with age. How fast depends on the field (news goes stale faster than science) and on the kind of source. A later independent confirmation restores a claim's weight.

5. Disagreement stays visible

Test: give the tool two credible sources that contradict each other. The answer should say "contested", not split the difference.

Arbiter: reports a contested claim as contested, and lists the different readings of the evidence side by side.

6. It admits what it cannot know

Test: ask whether the tool can tell you what is true. A tool that says yes is promising more than any tool can deliver.

Arbiter: no. We measure the sources you give us against what they say. A verdict of "not supported" means your sources do not carry the claim, not that the claim is false in the world. If every source is wrong, Arbiter is wrong with them.

Where Arbiter falls short

It does not find sources for you. You, or your AI assistant, supply them. Through an assistant, see Arbiter with a Model Context Protocol (MCP) connection.
The assistant or the extraction prompt that turns a document into claims can misread the document. Check the extracted claims against the original on any decision that matters.

Start

See the requirements met on a real case: a full fraud investigation, worked end to end. It is free to start, no card required; current plans are on the pricing page.