By Stuart Broadfoot, VP Engineering and Radovan Vojtko, Founder & CEO of Quantpedia.com
While technological advancements and new sources of data are transforming quant strategies, most models are still invariably shaped around historical data and past behaviors. How can you construct and accurately backtest strategies that are robust enough to withstand unforeseen and unprecedented events?
At the Quant Insights conference in May 2022, SigTech’s Angana Jacob hosted a panel discussion on the topic of Achieving Reliable Return Projections in Uncertain Times. The panelists Radovan Vojtko, CEO at Quantpedia, and Stuart Broadfoot, SVP Engineering at SigTech, provided insights into:
-
Designing quant strategies adaptive to regime shifts
-
Limitations of backtesting
-
How to protect your portfolio against ‘black swan’ event
Below is a summary of the discussion with the key lessons from Radovan and Stuart on how to successfully develop and run quantitative strategies in an ever changing market environment. You can listen to a full recording of the webinar via the link below.
Adaptive quant models
Angana Jacob (AJ): Quant investing operates under the premise of irregular market breaks and regime shifts. Do you think quant investing in general was well equipped to adapt to recent events such as the pandemic and the sudden spike in inflation?
Stuart Broadfoot (SB): “To start off, it is important to note that history never repeats itself perfectly and unforeseen events occur. Furthermore, increased market turbulence always accentuates mistakes in the research process and serves as a reminder that we should always use the best technology and employ sound practices when modeling quant strategies.
It’s also a reminder that quant models do break down and we need to account for potential regime shifts and changes to the status quo when building quant strategies.”
Radovan Vojtko (RV): “I would add that the market environment is and will always be perceived as challenging by investors. There will always be periods when certain strategies work well and other times when they struggle. But we have a wide range of datasets at our disposal and can adapt our models accordingly.
There is also a general commoditization of alpha in the financial markets, making it harder to profit from traditional alpha sources. But there are counterforces to this trend in play that result in tighter bid/ask spreads, new datasets to exploit, and newer – and often immature – markets that exhibit higher pricing inefficiencies, creating opportunities to exploit.”
Performance projections
AJ: How much faith do you have in performance projections in times of high uncertainty? What are the techniques you are using to ensure strategies can better adapt and withstand breaks or even shocks?
RV: “I am not a fan of performance projections, and have actually not yet seen one that is persistently good. I personally prefer simplicity when modeling and believe diversifying across many sub-strategies is key.
Adaptability to structural breaks is possible using filters (price-based, macro-based or ML-based) that are used to determine the current market regime.
It is also important not to be too driven by short-term performance when evaluating various strategies and to remove them from the portfolio when they do not deliver, because they can save you during a crisis.”
SB: “In general, return projections have huge uncertainty and you have to employ good best practices when modeling to reduce uncertainty as much as possible. It would of course be very compelling to have a strategy that works in all market environments, but a much more reasonable expectation is to quantitatively analyze in what market environment a strategy is likely to work, and not. That way you can monitor what is happening in the market and get insights into how a specific strategy is expected to perform, making it easier to make evidence based decisions in turbulent times.
An interesting area that recently has emerged through the increased supply of alternative datasets and advances in technology is nowcasting strategies. Rather than trying to create more reliable forecasting models, nowcasting is about trying to get a better idea of some hard-to-measure information to more rapidly adapt to new market environments.”
Limitations of backtesting
AJ: Backtesting is inherently backward-looking and reliant on historical data. As unprecedented events will continue to impact financial markets, we are faced with limitations of historical data. Why do we still all use backtests? Is it because we have no better alternative?
SB: “Like any tool, backtesting can be used in the wrong way, but I pretty much agree that there is not that much of an alternative. Backtesting is essential in identifying some potential mistakes that can be made in the research process, such as underestimating transaction costs, overestimating trading volume and not using high quality data.
I also prefer to view backtesting as just one example of the wider view of simulating strategies.”
RV: “There are not a lot of better alternatives to backtesting. I mainly use backtesting for risk assessment, not to form performance projections. It helps me to understand how a strategy has behaved during certain market environments.”
Next-gen scenario analysis
AJ: Running scenario analyses to reflect different market conditions is hardly a new concept, but the challenge is that these may not represent the true future uncertainty and will thus fail to determine the impact from events that have never happened before. How do you go beyond simple scenarios?
RV: “We of course do not know the true future uncertainty, but history is a good guide. We have a history full of wars, market closures, asset seizures over a very long sample period. We essentially have to dig deeper and have an open mind in terms of defining the scenarios that could occur.”
SB: “I find the emergence of deep generative models in machine learning to be very exciting. Outside of finance, such models have received plenty of attention. Recently there have been versions of generative adversarial networks that train a discriminator and a generative network together to model the underlying distribution and generate new data with a similar structure to the trained data. You can thus tailor datasets that are specific to one regime or one scenario that you want to test.
Detecting regime shifts
AJ: The current market environment is a function of events that have unfolded over the last couple of years. Are there reliable techniques quants can use to detect actual regime shifts (e.g. a recession) or use as early warning signs? Is it possible to prepare ahead of time?
SB: “The short answer is that it is very difficult to try and evaluate exactly when to adjust the positioning to a certain market. Regime shift models often give mixed signals, which creates a large amount of uncertainty around them. However, on a strategy level, you can apply certain filtering techniques as well as change point detection algorithms to improve the projections.
Furthermore, it is very useful to have a toolkit of tail risk and crisis alpha strategies to analyze historically. Every crisis is of course different, but in terms of how the market and thus certain strategies behave in a crisis, there are often common characteristics and you can make use of them when building strategies specifically for these environments. “
RV: “When trying to predict times of higher stress, we must accept that we will get a lot of false-negative signals. It is very hard in advance to distinguish which signal is true, and which is noise. We must accept a lot of these false signals because in each hypothetical scenario, a true black swan event can be hidden. In general, I am skeptical about increasing the effectiveness of predicting rare events because black swans are unpredictable.”
Protect against black swans
AJ: How do you protect your portfolio against a black swan event, and can you even opportunistically profit from them?
RV: “In my opinion, the only possible hedge is by using options. But, we need to be willing to pay an option premium for several years until the black swan event occurs. The hedge will most likely be a drag on the performance for a longer time, which for many investors is equivalent to a significant career risk.”
SB: “As true black swan events per definition are unpredictable, any actions you take need to be applied at all times. You have to keep on your risk protection, try and diversify as much as possible, apply strict risk controls, and make use of good strategy design procedures. And not to forget, it is not only about helping prevent losses, but a black swan event can also be a trigger to introduce new trading strategies.”
Key take-aways
“Nothing is certain except death and taxes” quipped Benjamin Franklin. One could append market uncertainty to this list. Currently investors are facing uncertainties in the form of inflation, equity valuations and the likelihood of a global recession. The panelists shared their insights into how to form reliable return projections in these unpredictable times and highlighted:
-
Turbulent market environments accentuate research mistakes but offer valuable insights into quant model improvement
-
Backtesting should also be looked upon as a form of risk management. It assesses how the strategy is expected to perform during certain market environments and gives insights into, among other things, how to adapt models to current market conditions
-
The emergence of new advanced analytics methods such as deep generative models in machine learning can help improve traditional scenario analyses by tailoring datasets
-
It is vital not to be too driven by short-term performance when evaluating strategies. Keeping allocation to strategies underperforming in certain time periods can be key to surviving a crisis.
Get in touch to learn more about how to create bespoke quant strategies.
