Statistical analysis reveals truth about data. Academic papers based on experimental data normally require a formal, statistical analysis of said data. This is true whether the topic is investing or the mating habits of guinea pigs. It is thus surprising just how little use is made of statistical analysis in published investment research reports. Part of the problem is statistical analysis is hard to understand.
"Pattern must be strong enough to profit from"
There is also a mistrust of statistics in general; a feeling that it gets used to derive the desired answer rather than the correct one. Whoever said, “There are three kinds of lies: lies, damned lies, and statistics” – anyone who says it was Benjamin Disraeli is lying – has a lot to answer for.
The beauty, and indeed the importance, of statistical analysis can be appreciated when one considers how it is applied – or should be applied – in the justice system. In statistics, the null hypothesis is the central hypothesis being scrutinised, and it gets either rejected – found to be false – or not rejected. A Type I error is made if the null hypothesis is rejected when it is in fact true. A Type II error if not rejected when false.
The ‘golden thread’ running through criminal law is that a person accused of a crime is presumed innocent until proven guilty. That a person is innocent can be thought of as the null hypothesis, with Type I and Type II errors being equivalent, respectively, to finding the innocent guilty and the guilty not guilty.
Type I errors in this context are also known as miscarriages of justice, and there are many examples of such having occurred in the UK and elsewhere – that they can and do occur must in my opinion be the central argument against capital punishment.
Furthermore, the principle of ‘beyond reasonable doubt’ means, in theory if not in practice, that the significance level of the hypothesis test is set close to 100% rather than the typical 95% found in many other areas. In other words, the bar for a jury to find an accused guilty is set extremely high.
A particularly tragic case of a miscarriage of justice is that of Sally Clark. In 1999, she was found guilty of murdering her two infant sons. The prosecution countered the defence’s argument that sudden infant death syndrome (SIDS) was to blame by citing the evidence of paediatrician Professor Sir Roy Meadow, which stated that the probability of such an occurrence was 1 in 73 million. In other words, there could be no reasonable doubt.
Meadow arrived at his figure by taking his estimate of the probability of one occurrence of SIDS in a family – 1 in 8,500 – and squaring it. This calculation, however, was deeply flawed. Squaring probabilities is only appropriate for independent events, whereas it is not at all clear that multiple occurrences of SIDS in the same family satisfy that criteria.
Indeed, a 2004 study found that “after a first cot death the chances of a second become greatly increased”. The study also found that Meadow “conveniently ignored factors such as both the Clark babies being boys, which make cot death more likely”.
Clark was released from prison after her conviction was overturned in 2003 but she died in 2007, having developed psychiatric problems because of her experience.
Following the advice in an investment research report – to buy or to sell – only makes sense if four criteria are satisfied. First, it must be clear that pattern exists in the price history of the instrument that the recommendation relates to. ‘Pattern’ in this context means that future price movements are influenced by past movements; if they are not, it’s just random noise.
Second, there must be reason to believe that any pattern identified will persist.
Third, the timing must be right. Mean reversion is an example of a pattern in which many price movements in one direction tend to be followed by movements in the other, but it is only invoked when a price has veered a long way from its trend or mean – think about the bounce in markets that began last March.
Fourth, the pattern must be strong enough to profit from. In other words, potential profit must exceed trading costs.
While the second criterion is more a matter of judgement, the other three can be answered with statistical analysis. It is a shame such analysis is rarely done and, more importantly, that the field of statistics is not better understood.
Published in What Investment
The views expressed in this communication are those of Peter Elston at the time of writing and are subject to change without notice. They do not constitute investment advice and whilst all reasonable efforts have been used to ensure the accuracy of the information contained in this communication, the reliability, completeness or accuracy of the content cannot be guaranteed. This communication provides information for professional use only and should not be relied upon by retail investors as the sole basis for investment.