“You can win any argument in the markets by simply changing your start and end dates.”
So wrote Ben Carlson in a recent post on his “A Wealth of Common Sense” blog. Carlson is director of institutional asset management at investment adviser Ritholtz Wealth Management. And he’s right: We need to be on our guard against financial analysts who pick and choose time periods to support their predetermined conclusion.
Or, as Adlai Stevenson, the Democratic candidate for President in 1952 and 1956, used to say when he mocked his opponents: “Here’s the conclusion on which I will base my facts.”
The recent occasion for Carlson’s comment was how much better U.S. stocks have performed since 2009 than their long-run average. From 1928-2017, he noted, stocks have produced a 9.7% annualized return. From 2009 to 2017, in contrast, their annualized return was nearly double, at 15.1%.
This contrast prompted me to slice-and-dice U.S. stock market history to show just how intractable the problem is that Carlson identified. Believe it or not, you can extend your measurement to over a century and still reach dramatically different conclusions.
Read: All this volatility is following one bear’s script for a 60% tumble in the stock market
Consider first the conclusion you reach by focusing on data back to the beginning of the 19th century, courtesy of Jeremy Siegel, the Wharton School finance professor and author of “Stocks for the Long Run.” Notably, his data take both dividends and inflation into account. Stocks today are 7.3% below a best-fit trendline drawn from then until now. (See chart; the trendline, for the statisticians among you, equalizes the amount by which the actual data is above and below.)
That will be welcome news to beleaguered bulls who for years have had to brace themselves against the onslaught of academic-sounding arguments that stocks are wildly overvalued.
And it would seem to be a compelling conclusion. After all, no other database extends back as far as the early part of the 19th century, and both dividends and inflation are included in the analysis — as indeed they should be.
But, unfortunately, there’s more to the story. Some, such as Jason Zweig of the Wall Street Journal, have raised serious objections about Siegel’s data for the early part of the 19th century. It is partly for these reasons that Robert Shiller, the Yale Univeristy finance professor and recent Nobel laureate, extends his historical database no further back than 1871.
But guess what: When you draw a trendline from 1871, you reach a different conclusion. Using the same database as plotted in the accompanying chart back to 1801, but instead beginning in 1871, you discover that stocks currently are 5.3% above trend — not 7.3% below.
You might want to hold off on your “buy” orders.
Which analysis is right? For the record, I should stress that I am not taking a position, having not studied the situation myself. The more important point is that this can’t be resolved quantitatively, since the statistical analyses involved in both are both unimpeachable. We have to squarely face the more qualitative issues involved in determining whether the data on which we are basing an analysis are trustworthy.
In making those qualitative judgments, we also need to ask which data are relevant to today. Some argue, for example, that stock market returns from the 19th- and early 20th centuries are not particularly applicable to our modern era, since they were produced when the U.S. economy was largely agrarian and before the country had become the dominant world geopolitical power and the U.S. dollar DXY, -0.10% had become the world’s reserve currency.
Unfortunately, the stock market currently stands far above trend if we focus on recent decades. Take a look at the accompanying chart that focuses on the five decades since the mid 1960s. Once again I relied on the same dataset that Siegel uses back to 1801, but began the trendline in the mid-1960s rather than early 1800s. The stock market today is 47% above the trendline that best fits the data since then.
Does this discussion mean that historical data don’t matter? Of course not. What it does mean is that we can’t avoid the hard non-statistical questions about what parts of market history are relevant to today. And it’s difficult not to let your preconceived beliefs influence your decision. We’re guilty of this, for example, when we continually mine the historical database in search of that particular historical period that most supports our preconceived beliefs.
That’s improper, of course. To guard against it, financial analysts should report how many different time periods they have studied before reaching the result they are presenting. One who makes this argument strongly is Campbell Harvey, a finance professor at Duke University and the recent president of the American Finance Association, one of the leading organizations of finance academics. He has specific recommendations for how statisticians can adjust their results to take into account the number of hypotheses they tested along the way.
Harvey’s broader goal, which he articulated in his presidential address to the AFA last year, is to cultivate a “robust, transparent research culture in finance economics.” Financial analysts who are truly committed to having their advice be data driven should be part of this effort too.
For more information, including descriptions of the Hulbert Sentiment Indices, go to The Hulbert Financial Digest or email mark@hulbertratings.com .
Now read: Jeff Reeves has 7 reasons to stop worrying and instead stick with stocks