What does “not a jot of evidence” mean?

The “not a jot” comment in the BCA v Simon Singh libel case has re-reared it’s head now that it’s beginning to look like “bogus” and “happily” weren’t so libellous after all. What on earth is a “jot” of evidence anyway?

The philosopher John Watkins held that some people treated evidence as some sort of “juice”. These people treat evidence as a substance which you go about collecting and, as soon as you have enough, you have proved your point. Substances are easy to measure or, at least, easy to compare. We may not have a measuring jug handy to say that a has a gallon of x whereas b has only a pint but it’s pretty clear who has more. Substances also persist. You may only have a cupful but it is a cupful and will remain a cupful. If you have some evidence you may add to it to get lots of evidence but you will never go back to having no evidence at all. Finally substances are additive. If you have one pint of x and you add another pint of x then you have two pints of x.

Such an attitude is mistaken. Take ravens, as an example. It will be readily agreed that we have huge amounts of evidence for:

1. All ravens are black

Whilst we have no evidence for:

2. All ravens are male

Think about just exactly what that evidence is and, if we think in “juice” terms, we are bound to take it that we have a very good deal of evidence that, in fact, every single raven is male. Part of our evidence for ravens being a uniformly black species is that we have collectively seen huge numbers of ravens that are black. Let’s mentally take that evidence-juice and place it in the huge vat needed to accommodate it. Naturally with that much evidence we would have a pretty hard time denying that all ravens are black.

But we have also seen huge numbers of ravens that are male. In fact approximately half of all the huge numbers of ravens seen have been male. Mentally take that evidence-juice and place it in the huge vat next to the ravens-are-black vat. The juice will reach a level half that of the other vat. We could argue about whether it was enough, whether it had reached the level of “iffy” or “proven” or “certain”. We could not argue that it was nothing: which, of course it is.

The thing missed in this analysis is that sightings of black ravens on their own is no evidence at all for concluding that all ravens are black. Sightings of black ravens and no sighting of ravens of any other colour is. We could preserve the juice/substance by considering sightings of ravens of any other colour as a sort of “anti-juice”. If you see a black raven then you get a whisky measure of evidence-juice. If you see a raven of another colour then you get a whisky measure of anti-evidence-juice. The more black ravens predominate over other colours the more net-evidence-juice we have. At a cost of giving up persistence we save the juice concept of evidence.

This would work for “all ravens are male”. Each male raven seen would contribute a measure of evidence-juice, each female raven seen a measure of anti-evidence-juice. As half the ravens seen would be female the sum total of evidence would be zero.

It wouldn’t work for “all swans are white”. The existence of black swans does not reduce the amount of positive evidence for “all swans are white” but (no matter how few of them there are) is fatal for the hypothesis. Negative evidence in other areas is not so powerful: the odd bad game played by Everton reduces the evidence for, but is not fatal to, “Everton are the greatest team on earth”. It seems that we have to give up the idea of a nice, neat, additive nature of evidence.

Bayesian analysis seeks to replace a nice, neat, but unworkable, additive concept of evidence with a calculus of belief. If I test positive for disease a (Bayesians like to talk about diseases) then my degree of belief in “I have disease a” should increase. How much depends on how likely I am to test positive if I have the disease, how likely I am to test positive if I don’t have the disease and just how likely I am to have the disease (ignoring any evidence). But here we’ve gone another stage on: we have given up quantifying evidence and are now quantifying belief.

Yet we do speak of an “amount” of evidence. The evidence for “all ravens are black” is “huge”, the evidence for “all ravens are male” is “not a jot”. This is simply loose, colloquial, talk. It is not the evidence that is quantifiable (or, at least, rankable) but belief. The total evidence we have informs belief and, taking the degree of belief caused, we use this quantity to label all the evidence. The totality of evidence for “all ravens are black” is such that no reasonable person would doubt that all ravens are black. We are really sure that all ravens are black and, thus, label the totality of evidence that got us there “huge”. No reasonable and informed person thinks all ravens are male. Our degree of belief in “all ravens are male” is zero and so we use this to quantify the totality of evidence for “all ravens are male”. This is “zero”, despite the fact that there is substantially more than a plethora of male ravens.

So when someone says, for example, that there is “not a jot” of evidence for chiropractic as a remedy for asthma it does not mean that no trial has ever seen anything even vaguely compatible with chiropractic. What is meant is that, given the evidence, a reasonable and informed person would have “not a jot” of belief that chiropractic is a remedy for asthma.