February 26, 2012 § 1 Comment
Financial statisticians should study the following charts of change in some important metric over time. Before giving away what the metric is I’d like you to try to guess what it is after studying the figures for a while. For example, it could be the log scale of a particular stock price through 50 years, aggregate earnings of the S&P500 through some 50-year period, book value of some corporation through 50 years or some other metric. Even if you cannot guess the metric, the important thing is to try to imagine whether the subsequent values will have a greater chance of being falling or rising. Click on one of the charts to view them as a slide show.
While studying each chart, observe carefully how the blue values interact with the orange trend line. For each chart imagine the blue data extending for five more years and write down whether the blue line has (1) a linear trend or whether it is will (2) continually decline, (3) temporarily decline, (4) continually ascend or (5) temporarily ascend. Important: Don’t read the next paragraph below the charts until you have applied your best intuition to make sense of the data in each charts and written down your estimates.
Well to give the answer, the metric is actually a purely random function with slight upwards bias. I placed the orange trend line through the series because this demonstrates four very important things. The first thing is that the slope of the line rarely matches the true upwards trend in the data – thus the slope of the line itself tends to give false confidence as to the long-term trend. Even extending the data series into the future with twice as much data could result in the slope remaining quite incorrect as to the actual bias in the random function, which was identical for all the charts: y’ = y (0.7 + random[0 to 0.7]).
Secondly the orange line encourages the person interpreting the data to imagine that there would likely be some reversion to the mean in the short-term if the data was extended, while there is no such reversion to the mean – extending the chart further to the right has no regard for what occurred before.
The third thing of great interest is that as the data is extended into the future the slope of the new trend line changes so that this extra data, which might have values that were previously unexpected (such as continual upwards movement when recent values were already above trend), becomes expected after it arises owing to the original trend-line being ignored and the new trend line replacing it. It is this phenomenon that makes events appear more reasonable, even predictable, in hindsight – even if they were the opposite of what was originally expected. To illustrate, if the blue data has recent (right-most) values above the trend-line then it might appear likely to fall back to the trend line. If it continues to rise however, the slope of the new orange trend line also rises substantially making these new values appear not so abnormal after the events take place.
The fourth thing relates to the third but is worth distinguishing as follows. With so many charts having exactly the same random function we have a reasonable idea what the true trend line is – some orange slopes are steeper than others but by averaging all of them we have an idea what the true slope of the random function is. However, consider just one chart and observe the slope. From this chart alone, covering 50 years, you will deduce the slope and thus expect that if the chart was extended for another 50 years then it would be likely to have a slope that is similar. Yet look across the various charts at how different the slopes are. Knowing that US corporate earnings over the last 100 years increased 1.6% per year after inflation (a multiple of five times) we are inclined to expect that the next 100 years will provide a similar multiple however the result could be quite different, and our latest position above or below the trend only a mirage.
The act of studying the graphs repeatedly knowing that they are random will gradually help to negate some of these biases and helpfully reduce confidence to allow better decision making in all fields.