Tag Archives: james-picerno

A (Partial) Solution For Narrative Risk: Probit Modeling

The search for objective analysis in the cause of making informed investment decisions is the Holy Grail of finance. Unfortunately, narrative risk continually threatens to derail us on our crucial quest for perspective. Everyone loves a good story, and it’s no different when it comes to finance and economics. The problem: there’s an excess of interesting narratives that too often are bereft of useful information. Genuine insight that’s earned by way of a clear-eyed review of the numbers, in other words, is the exception in a world that’s overflowing with story lines that appeal to emotion rather than intellect. Unfortunately, we’re bombarded with distraction. The self-proclaimed seer on TV who spins a good yarn about what’s really driving prices or the business cycle can draw a crowd by dispensing entertaining narratives about gloom and glory. Bigger is always better from a media vantage, even if the facts don’t easily fit the narrative. Meanwhile, a sober reading of the numbers is a yawn if you’re trying to maximize eyeballs pinned to the tube. Making reasonable decisions and producing compelling media content, in short, are often at cross purposes. What’s the solution? There are many paths to quantitative enlightenment, including the powerful analytics that arise from probit and logit regressions. There are no silver bullets here, but the ability to translate raw data into specific probability estimates for a particular condition offers a valuable resource for cutting through the noise. The flexibility and power that probit and logit models bring to the table suggest that running the numbers through these filters should be on everyone’s short list of analytical tools. As a simple review of how these models work, let’s run through the basics using R code as the quantitative lingua franca, although you could easily do the same in Python or even Excel. But first a warning: the illustration below is a toy example and not a very useful one as presented in terms of making real-world decisions. But the general outline is valid and so the process will offer a flavor of how to deploy this modeling framework. ( Here’s the R code to replicate the data discussed below.) Let’s say that you think that the VIX index, which tracks the implied volatility of the US stock market (S&P 500), offers useful information for monitoring and measuring equity risk. After eyeballing a chart of the two indexes (as shown below) you decide that a VIX reading above 20 is a warning sign for the market. But how threatening is a 20-plus reading? One way to quantify the danger is by analyzing the S&P 500 in context with the VIX by way of a probit model. The first step is creating a data set of binary signals that reflect your definition of “risk.” The sky’s the limit for customizing this definition, but in the interests of keeping this short review from becoming a consulting project let’s use one simple metric: rolling one-year return for the S&P 500. The research agenda is estimating the probability of a negative one-year return based on the current VIX reading. (Yes, this is a bit naïve not to mention superfluous, but it’s good enough to profile probit modeling.) With our research map in hand, it’s a simple matter of getting the data in shape. The first step is creating a set of binary signals to indicate the state of the market that we’re trying to model. Remember, a probit model is designed to estimate probabilities for one of two states, which is considerably easier and therefore more practical in the real world vs. trying to model a spectrum of conditions. In keeping with our simplistic example. any negative one-year return for the S&P is labeled as “1” and a positive return as “0”. The next step is instructing the probit model to estimate the probability that the S&P is in negative territory by way of analyzing the historical relationship between the VIX and the signal data as defined above. Right about now you’re probably complaining that we already know the state of S&P one-year return by looking at real-time market data without referring to the VIX. Agreed, and so creating a probit model to tell us what’s already obvious is a redundant exercise. True, at least in this case, but the point of all this is to outline a basic probit procedure. Keep in mind that a genuine effort in this corner would probably focus on modeling a state that’s unclear in real time, such as the start of a recession or some other aspect of market risk that’s not readily available. As for our toy example with the S&P, here’s the result of the probit model estimates for the probability that the S&P’s trailing return is below zero. Visual inspection suggests there’s some meat on this bone. The rising probability that eventually reached 100% in late-2008, for instance, tells us that there’s a relatively robust relationship between the S&P and the VIX. Well, of course there is! We already knew that. The probit model is simply quantifying the relationship per our specifications. The question is how or if such a model should be adjusted. Is modeling trailing 6-month return preferable to 1-year performance? Should we raise or lower the 20-plus VIX trigger? What about adding in additional variables – the 10-year Treasury yield, for instance. There’s a wide array of possibilities here, which is a benefit and a curse. A benefit because probit modeling (and its close cousin logit modeling) can be customized in an endless variety to extract estimates of a particular state from raw data. But that’s a curse if you’re unsure of how to proceed. In other words, doing preliminary research to map out a reasonable strategy is essential before you dive into the numbers. But with a bit of advance planning, deploying a probit model can offer deep insight into market and macro analysis. There are no guarantees, of course-probit models can lead us astray in some cases, particularly when we’re sloppy with assumptions about relevant variables. But compared with listening to someone’s interpretations of what the latest market moves suggest, probit modeling offers objective context without the baggage of behavioral biases. It’s not a complete solution to narrative risk, but it’s a good start.

Revisiting 10 Asset Allocation Funds Amid Market Turmoil

Earlier this year, I reviewed ten asset allocation mutual funds with a range of strategic designs as an academic exercise for exploring how multi-asset strategies stack up in the real world. Seven of the ten funds post losses for the trailing one-year period through yesterday (Sept. 22), along with one flat performance and two modest gains. One lesson in all of this is that investment success (or failure) is usually driven by two key factors: asset allocation and the rebalancing methodology. For the elite who beat the odds, the source of their success is almost certainly bound up with superior rebalancing methodologies that shine when beta generally takes a beating. Earlier this year, I reviewed ten asset allocation mutual funds with a range of strategic designs as an academic exercise for exploring how multi-asset strategies stack up in the real world. Not surprisingly, the results varied, albeit largely by dispensing a variety of gains as of late-February. But that was then. Thanks to the recent spike in market volatility (and the slide in prices), a hefty dose of red ink now weights on these funds. Seven of the ten funds post losses for the trailing one-year period through yesterday (Sept. 22), along with one flat performance and two modest gains. This isn’t surprising considering the setbacks in risky assets over the last month or so. But the latest run of weak numbers is also a reminder that asset allocation comes in a variety of flavors and the results can and do vary dramatically at times. One lesson in all of this is that investment success (or failure) is usually driven by two key factors: asset allocation and the rebalancing methodology. Of the two, rebalancing is destined to be a far more influential force through time. Assuming reasonable choices on the initial asset mix, results across portfolios – even with identical allocation designs at the start – can and will vary by more than trivial degrees based on how the rebalancing process is executed. And let’s be clear: it’s no great challenge to select a prudent mix of asset classes to match a given investor’s risk profile, investment expectations, etc. Tapping into a solid rebalancing strategy (tactical or otherwise) is a much bigger hurdle. But at least there’s a solid way to begin. For most folks, holding some variation of Mr. Market’s asset allocation strategy – the Global Market Index, for instance – will do just fine as an initial game plan. The choices for tweaking this benchmark’s design will cast a long shadow over results if the weights are relatively extreme – heavily overweighting or underweighting certain markets, for instance. Otherwise, the details on rebalancing eventually do most of the heavy lifting, for good or ill as time rolls by. With that in mind, we can see that most of our ten funds have had a rough ride recently. The reversal of fortune has been especially stark for the Permanent Portfolio (MUTF: PRPFX ) this year. After leading the pack on the upside in April and May (based on a Sept. 23, 2014 starting point), the fund has since tumbled and suffers the third-worst slide among the ten funds for the trailing one-year return. (click to enlarge) At the opposite extreme, we have the Bruce Fund (MUTF: BRUFX ) and the Leuthold Core Investment Fund (MUTF: LCORX ), which are ahead by around 3.5% for the past 12 months. Those are impressive results vs. the rest of the field. Note the relative stability for BRUFX and LCORX over the past month or so. Is that due to superior rebalancing strategies? Or perhaps the funds beat the odds by concentrating on asset classes that fared well (or suffered less) in the recent and perhaps ongoing correction? We can ask the same questions for the other funds in search of reasons why performance suffered. In any case, the answers require diving into the details. A good start would be to run a factor-analysis report on the funds to see how the risk allocations compare. Another useful angle for analysis: deciding how much of the performance variations are due to what might be considered asset allocation beta vs. alpha. A possible clue: BRUFX’s longer-run results are also impressive while LCORX’s returns are relatively mediocre in context with all of the ten funds, as shown in the next chart below. Is that a hint for thinking that BRUFX’s managers have the golden touch in adding value over a relevant benchmark? Maybe, although the alternative possibility is that the fund is simply taking hefty risks to earn bigger returns. In that case, the risk-adjusted performance may not look as attractive. Perhaps, although several risk metrics (Sharpe ratio and Sortino ratio, for instance) look encouraging and give BRUFX an edge over LCORX, according to trailing 10-year numbers via Morningstar. (click to enlarge) Meanwhile, keep in mind that an investable version of the Global Market Index – a passive, unmanaged and market-weighted mix of all the major asset classes – is off by roughly 5% for the trailing one-year period. That’s a middling result relative to the ten funds, which isn’t surprising. In theory, a market-weighted mix of a given asset pool will tend to deliver average to modestly above-average results vs. all the competing strategies that are fishing in the same waters. In other words, most of what appears to be skill (or the lack thereof) is just beta – even for asset allocation strategies. But there are exceptions. That doesn’t mean that we shouldn’t customize portfolios or study what appear to be genuine advances in generating alpha in a multi-asset context. But as recent history reminds once again, beating Mr. Market at his own game isn’t easy. But for the elite who beat the odds, the source of their success is almost certainly bound up with superior rebalancing methodologies that shine when beta generally takes a beating.

Backtesting With Synthetic And Resampled Market Histories

We’re all backtesters in some degree, but not all backtested strategies are created equal. One of the more common (and dangerous) mistakes is 1) backtesting a strategy based on the historical record; 2) documenting an encouraging performance record; and 3) assuming that you’re done. Rigorous testing, however, requires more. Why? Because relying on one sample-even if it’s a real-world record-doesn’t usually pass the smell test. What’s the problem? Your upbeat test results could be a random outcome. The future’s uncertain no matter how rigorous your research, but a Monte Carlo simulation is well suited for developing a higher level of confidence that a given strategy’s record isn’t a spurious byproduct of chance. This is a critical issue for short-term traders, of course, but it’s also relevant for portfolios with medium- and even long-term horizons. The increased focus on risk management in the wake of the 2008 financial crisis has convinced a broader segment of investors and financial advisors to embrace a variety of tactical overlays. In turn, it’s important to look beyond a single path in history. Research such as Meb Faber’s influential paper “A Quantitative Approach to Tactical Asset Allocation” and scores of like-minded studies have convinced former buy-and-holders to add relatively nimble risk-management overlays to the toolkit of portfolio management. The results may or may not be satisfactory, depending on any number of details. But to the extent that you’re looking to history for guidance, as you should, it’s essential to look beyond a single run of data in the art/science of deciding if a strategy is the genuine article. The problem, of course, is that the real-world history of markets and investment funds is limited-particularly with ETFs, most of which arrived within the past ten to 15 years. We can’t change this obstacle, but we can soften its capacity for misleading us by running alternative scenarios via Monte Carlo simulations. The results may or may not change your view of a particular strategy. But if the stakes are high, which is usually the case with portfolio management, why wouldn’t you go the extra mile? The major hazard of ignoring this facet of analysis leaves you vulnerable. At the very least, it’s valuable to have additional support for thinking that a given technique is the real deal. But sometimes, Monte Carlo simulations can avert a crisis by steering you away from a strategy that appears productive but in fact is anything but. As one simple example, imagine that you’re reviewing the merits of a 50-day/100-day moving average crossover strategy with a one-year rolling-return filter. This is a fairly basic set-up for monitoring risk and/or exploiting the momentum effect, and it’s shown encouraging results in some instances-applying it to the ten major US equity sectors, for instance. Let’s say that you’ve analyzed the strategy’s history via the SPDR sector ETFs and you like what you see. But here’s the problem: the ETFs have a relatively short history overall… not much more than 10 years’ worth of data. You could look to the underlying indexes for a longer run of history, but here too you’ll run up against a standard hitch: the results reflect a single run of history. Monte Carlo simulations offer a partial solution. Two applications I like to use: 1) resampling the existing history by way or reordering the sequence of returns; and 2) creating synthetic data sets with specific return and risk characteristics that approximate the real-world funds that will be used in the strategy. In both cases, I take the alternative risk/return histories and run the numbers through the Monte Carlo grinder. Using R to generate the analysis offers the opportunity to re-run tens of thousands of alternative histories. This is a powerful methodology for stress-testing a strategy. Granted, there are no guarantees, but deploying a Monte Carlo-based analysis in this way offers a deeper look at a strategy’s possible outcomes. It’s the equivalent of exploring how the strategy might have performed over hundreds of years during a spectrum of market conditions. As a quick example, let’s consider how a 10-asset portfolio stacks up in 100 runs based on normally distributed returns over a simulated 20-year period of daily results. If this was a true test, I’d generate tens of thousands of runs, but for now let’s keep it simple so that we have some pretty eye candy to look at to illustrate the concept. The chart below reflects 100 random results for a strategy over 5040 days (20 years) based on the following rules: go long when the 50-day exponential moving average (NYSEMKT: EMA ) is above the 100-day EMA and the trailing one-year return is positive. If either one of those conditions doesn’t apply, the position is neutral, in which case the previous buy or sell signal applies. If both conditions are negative (i.e., 50-day EMA below 100 day and one-year return is negative), then the position is sold and the assets are placed in cash, with zero return until a new buy signal is triggered. Note that each line reflects applying these rules to a 10-asset strategy and so we’re looking at one hundred different aggregated portfolio outcomes (all with starting values of 100). The initial results look encouraging, in part because the median return is moderately positive (+22%) over the sample period and the interquartile performance ranges from roughly +10% to +39%. The worst performance is a loss of a bit more than 7%. The question, of course, is how this compares with a relevant benchmark? Also, we could (and probably should) run the simulations with various non-normal distributions to consider how fat-tail risk influences the results. In fact, the testing outlined above is only the first step if this was a true analytical project. The larger point is that it’s practical and prudent to look beyond the historical record for testing strategies. The case for doing so is strong for both short-term trading tactics and longer-term investment strategies. Indeed, the ability to review the statistical equivalent of hundreds of years of market outcomes, as opposed to a decade or two, is a powerful tool. The one-sample run of history is an obvious starting point, but there’s no reason why it should have the last word.