Tag Archives: georgia

HARKing Back: Lessons In Investing From Science

Confirm Ye Not Here’s what ought to be a really boring idea – we need scientists in general and psychologists and economists in particular to stop hypothesising after results are known (HARKing, geddit?). Instead, they need to state what they’re looking for before they conduct their experiments because otherwise they cherry pick the results they find to confirm hypotheses they never previously had. The underlying problem is our old foe, confirmation bias . And the solution for scientists and social scientists alike is known as pre-registration. It would be no bad thing for investors to demand a similar process for fund managers and financial experts. Or, for that matter, to apply some of the ideas to their own investing strategies. No No Negatives It’s been known for years that a lot of scientific research isn’t very reliable. There are numerous problems, chief amongst them being the non-publication of negative results: an issue known as publication bias . There’s no kudos in showing that your hypotheses were wrong, so researchers and corporations tend to bury the data, but it’s still valuable information that should be shared: scientists see further by standing on the shoulders of others, we shouldn’t be encouraging them to shrug them off because they’ve got bored. Worse still, though, is the fact that many studies turn out not to be replicable. The ability to re-run an experiment and produce the same result is an absolute cornerstone of the scientific method : science works because it’s not built on faith, it’s constructed out of evidence. If it turns out that the evidence is unreliable then what’s being done isn’t science, it’s more like religious studies with instruments. Or economics. Repeat, Again Once we move to the social sciences then the problems are even worse. Human beings are terrible things to experiment on , being inclined to change their minds, develop opinions about the experiments and to second-guess what the researchers would like them to do, just to be nice. All too many experiments in the social sciences turn out to be flawed because of social or situational factors that didn’t seem important at the time. Given this, you’d think that repeating experiments to make sure the results held would be even more important for psychologists than it is for researchers in the hard sciences. Well, guess again. According to research by Matthew Makel, Jonathan Plucker and Boyd Hegarty , only a little over 1% of psychology studies have ever been replicated. Everything else is simply a matter of faith in the integrity and lack of bias of the original researchers. Which is not science: in the words of John Tukey, quoted at the head of their paper: “Confirmation comes from repetition. Any attempt to avoid this statement leads to failure and more probably to destruction.” Pre-Register The best solution to this we’ve yet found is known as pre-registration: studies have to be registered in advance, and the hypotheses under investigation stated up front before the research is done. This prevents the experimenters from looking at their data after the event and picking out interesting positive correlations which they didn’t control for, but which are likely to get published. Where pre-registration has happened the proportion of studies giving positive results has fallen dramatically: analysis of studies into treatments for heart disease have shown a frightening drop in positive results since pre-registration was mandated: “17 out of 30 studies (57%) published prior to 2000 showed a significant benefit of intervention on the primary outcome in comparison to only 2 among the 25 (8%) trials published after 2000”. Some of this may be because the low-hanging fruit on the subject was picked earlier, but it’s a scary result all the same. It seems likely that because the researchers can no longer consciously or unconsciously pick the results, they prefer they remove the possibility of confirmation bias – and the fall is so dramatic it places the previous results in question. And, of course, it’s not clear how many of those have been replicated. Creative Scientists Pre-registration isn’t universally popular: there is much rending of white coats and grinding of molars over the issue. Opponents argue that it risks putting scientists in a creative straight-jacket. Although when respectable peer reviewed journals start publishing papers alleging the existence of extra-sensory perception based on … “Anomalous processes of information or energy transfer that are currently unexplained in terms of known physical or biological mechanisms” … then you have to wonder whether the creative juices maybe need a touch of reduction – oh, and the results of this experiment don’t seem to be replicable, bet they never saw that coming. So, what other group of people do we know who are given to making ad-hoc hypotheses, investing loads of money in them, and then ignoring the results while cherry picking specific successes in order to publicly claim that they were successful? OK, apart from politicians. Investing Feedback Investors have all of these faults, and a few more. If we truly wanted to become better investors, then we’d pre-register our hypotheses – including our expected timescales – and then measure our results against the results. Doubtless the outcome would frequently be embarrassing, but the evidence that we do have suggests that getting real feedback about our performance is the only way to improve predictive capability in complex systems like the stock market (see: Depressed Investors Don’t Need Feedback. Everyone Else Does ). The other thing this would do would be to force us to face up to the reality that we can be successful by luck and can fail through no fault of our own. In complex adaptive systems, we simply cannot predict every possible situation; we can only hope to be able to predict a little better than average. But a little better is enough to make a turn, so every percentage point improvement we can make is worth it. Commit and Document So I wonder if some enterprising developer out there fancies setting up a pre-registration website for investors keen to improve their returns, rather their personal status? Public commitment backed up by a positive rewards system has been shown to produce powerful results in a whole variety of situations. For example, in Tying Odysseus to the Mast: Evidence from a Commitment Savings Product in the Philippines , Nava Ashraf, Dean Karlan and Wesley Yin showed: “Commitment-treatment group participants have a 12.3 (9.6) percent higher probability of increasing their savings by more than 20 percent after six (twelve) months, relative to the control group participants, and an 11 (6.4) percent higher probability of increasing their savings by more than 20 percent, relative to the marketing group participants. The increase in savings over the twelve months suggests that the savings response to the commitment treatment is a lasting change, not merely a short-term response to the new product” I suspect that even a non-financial reward system based on peer support would facilitate uptake. HARK, hear… Avoiding HARKing is the future of the hard and the soft sciences. And, by analogy, as investors, if we don’t have hypotheses about what we’re investing in, then we’re simply the modern equivalents of astrologers. And, if we have hypotheses, we should write them down and test whether they’re right, not simply crow about the random successes and ignore the equally random failures. It’s worrying, of course, that this isn’t already the basic investing process. But to be honest it’s even more worrying that it doesn’t seem to be the basic scientific process. Genius and creativity has its place in all human activity – Kepler came up with his third law of planetary motion by mapping orbits to harmonic ratios , believing these to be a sign of heavenly perfection. But Kepler was a mad genius who happened to be correct, so here’s my hypothesis: relying on mad geniuses for humanity’s future and your family’s well-being is probably not prudent.

The Altman Z-Score After 50 Years: Use And Misuse

By Larry Cao, CFA This is the second installment in my interview series with Edward Altman in which we discuss the most advisable and problematic applications of the Altman Z-score. For additional details of our conversation, check out the first installment. Larry Cao, CFA: It’s been almost 50 years since the Z-score was first developed. Would you suggest doing anything differently today? Edward Altman: Over the years, the so-called cutoff scores in the model has been retained by the people who applied the model. But in my opinion, that is not the best thing to do. Over time, I began to observe that the average Z-score of American companies mainly, but even global companies, began to get lower and lower. [The bond market] became more available for both investment grade and non-investment grade companies and companies periodically took advantage of low interest rates to raise their leverage. As a result, the financial risk of companies began to increase. Also with global competition, companies’ profitability began to diminish. And so the average Z-score became lower and lower, which meant that more firms would have been classified as likely bankrupt using the Z model if we kept the original cutoff scores. In order to modernize the model, we needed bond-rating equivalence of the scores, which changes constantly and adds on an updated nature to the interpretations of the scores. We now think the most important attribute of the Z model is the probability of default (PD), not the zone classification – safe, grey, or distress. We do it in a two-step process. We get the PD from the score of the company, whether it be from Z, or Z prime, or Z double prime. And then we look at the bond rating equivalent as of that point in time. For example, 2015 – the average B-rate company has a Z-score [of] about 1.6. That would be in a distress zone back in 1968. But today, B is a very common bond rating for many companies. In fact, globally it’s probably [the] most dominant junk rating category. If you rated all companies in the world, the average would probably be about B if they had a rating. And so we ascribe a probability of default based on a bond rating equivalent by looking at the historic incidence of default given a B rating at birth. Cumulatively, I can tell you, from one to 10 years, what the likelihood of default is given a bond rating equivalent. So no longer do we only look at the cutoff scores for the three zones of credit worthiness. Okay, bond rating equivalent is in and cutoff scores are out. What mistakes do you see practitioners making in using the Z-score today? To this date, I would say the vast majority of people are misusing the Z-score because they are applying it across the board regardless of the sector, the industry. And what we found over the years is that non-manufacturers, especially in certain industries like services or retail, have on average higher Z-scores than manufacturing companies. My advice for users is if you are outside the United States, and particularly if you are not a manufacturer, you should look at Z” and its bond rating equivalence approach for ascertaining a PD. Would you say the value of the Z-score is more in its methodology or the score itself? That’s a great question, Larry. Yes, I’ve always argued it’s better to use a local model rather than the original US model. And I’ve done it myself. I’ve personally built models in Brazil, Australia, France, Italy, and Canada. And you will find references to models almost anywhere in the world in the literature. It’s a pretty easy methodology for Ph.D students and practitioners to adapt to a different environment. But then again, even if it isn’t the best model that could be built for service companies or energy companies in 2016, it’s still a good benchmark and has retained its accuracy. If I had the time, I would build the model for Malaysian companies or Indonesian companies or Hong Kong companies or Asia all together. I suppose that there are good researchers there who might just attempt that! Will there be a data issue? For a lot of these countries, the history may not be there. They don’t have bond rating equivalence. That’s exactly right. That’s a very good point. The bond rating equivalence in almost all cases has to be derived from data from the United States. We have lots of defaults, lots of bankruptcies in the US, so you can get probability distributions based on ratings that have a fairly large sample. You can’t do that in emerging markets or countries like Australia, where they haven’t had a recession since the early 1990s. So yes, people said I should have continually updated the Z model but that means you have to keep publishing the updates. People have to find it. People have to use it and test it. It’s much easier to just periodically test the model, and to even build new models that incorporate the lot of data from the relevant countries and industries and combine this firm data with market value measures and possibly even macroeconomic data. What advice do you have for practitioners who want to build their own version of the Z-score model? For example, what’s your secret sauce for putting together the sample? Although the methodology is pretty straightforward, there are subtleties to it. You need a sample of healthy companies and unhealthy companies. There are issues such as sample size. Should [there] be [an] equal number in the two groups or should there be more representatives of the population – 99% non-default, 1%, 2%, or 3% default, depending on the time period? Should they all be manufacturers? Should they be a cross section of industries? Disclaimer: Please note that the content of this site should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute.

The Altman Z-Score In Edward Altman’s Own Words

By Larry Cao, CFA The Altman Z-score is a famous formula for measuring a company’s financial worthiness devised by Edward Altman . I sat down with Altman in Hong Kong recently to discuss the Z-score, its original inspiration, evolution over the years, use and misuse, as well as the current credit situation around the world. In this first installment, Altman discusses how the model was initially developed and what has changed since then. For the rest of our conversation, please stay tuned for additional installments in the weeks ahead. Larry Cao, CFA: Can you start by giving us some background on how you came across the problem and how you developed the formula as a solution? Edward Altman: When I was a graduate student at UCLA in the mid-1960s, one of my mentors, Professor J. Fred Weston, knew that I was looking for a topic for research, and he wrote me a one-word note one day: “bankruptcy.” In those days, bankruptcy was not a very popular research area, although there had been some work done using individual measures to look at the financial risk of companies. I decided I had to look at the subject of predicting financial distress of companies using a multivariate approach. You know, sometimes breakthroughs are not so much a function of the brilliance of the people but the timing and the luck. And I was very lucky to be a Ph.D student at the right time in the right place. If I had thought about this subject two years earlier, I would not have had the computer firepower that was just beginning to come on campuses in the United States. If I had been on the scene two years later, someone else would have already done the work. I combined a number of financial indicators with a technique for statistical classification known as discriminant analysis to predict bankruptcy. That was written in 1967, published in 1968, [and] known as the Z-score model or the Altman Z-score. And this model originally was built and still is mainly relevant for manufacturing companies. I had no idea that, almost 50 years later, people would still be using it and, indeed, using it more than ever. In your paper, you used five categories of variables – liquidity, profitability, leverage, solvency, and activity – to predict insolvency. How did you end up choosing the specific variables in the model? At that time, there were a lot of variables in the literature that you could choose to predict insolvency. But I decided there are two variables that were potentially very powerful but had not been used yet. One was the retained earnings: The argument there being a firm that has grown its assets mainly by reinvesting earnings is healthier than a firm that has grown the assets by using “other people’s money.” Retained earnings is also a measure of the age of the company and leverage. So that one measure combined leverage, profitability over the life of the company minus dividends, and also the age or experience of the company. You would think it makes a lot of sense because it does go back to the history of the companies and says, “Hey, how much money have you made and how much of that have you reinvested rather than paid out to your owners?” Yet you don’t come across models that use retained earnings very much these days. That’s true. It’s funny. Retained earnings/total assets is so powerful in my model, but you don’t find them very much taught in the classroom or found in the literature. I found it extremely important and helpful in almost every model I built over the years, for different industries and countries. What’s the other new variable you identified? The other new variable then – even though now it’s quite commonly understood – was the market value of the equity relative to the book value of the debt, as opposed to the book value of equity. It was the first study that – even before the Merton model, which was 1973, 1974 on risky debt – anticipated the importance of market equity relative to book debt as a very important indicator where it represents the ability of the company to raise money from the capital markets to pay down the debt or to expand the company. So market equity is now a fundamental part of many so-called structural models provided by Merton, KMV, and a number of other providers. So Z-score is a statistical model, with all the parameters driven by the particular sample. [Exactly] For a different sample, should users get new estimates for the parameters? The original sample was manufacturers. Rather than updating the original model for, say, more recent bankruptcies, which we can do, what we prefer to do is build new models. I developed the Z”-score model in 1995 mainly for emerging market and non-manufacturing industrial companies. We also decided to take out the fifth variable, sales to assets. And we re-estimated the coefficients. So you took out an activity ratio? Exactly. It was very sensitive to the industry and, to some extent, the country. There was a new breed of corporate debt coming from emerging markets in the mid-90s, such as from Mexican, Brazilian, and Argentinian companies. And we tried to get a model which was more appropriate for that segment of the world and for manufacturers and non-manufacturers. We find that Z” is far more robust across sector and countries than Z-score, although both do a good job in classifying companies as to their bankruptcy potential with the same further modifications. How did the five variables rank in terms of importance? We look at the relative contribution and its statistical test. It turns out return on assets is number one. Retained earnings to total assets is number two. Market equity to total liabilities, three. Sales to assets, four. And the least important one, surprisingly, is the liquidity ratio, net working capital to total sales. Has the ranking changed from Z to Z”? No, it has not changed, except that sales/total assets is no longer a factor in the revised Z-score model. Fascinating. Disclaimer: Please note that the content of this site should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute.