New Options Backtester
Search, filter, and sort 181,190,293 backtests

Backtesting

Wednesday, July 26th 2023

Avoiding Common Problems with Backtesting

Learn how the new ORATS Backtester tackles common problems with backtesting, such as overfitting and path dependency.

Free Trial
Get a free trial of ORATS trading tools by completing this quiz
Take the quiz

Summary

Backtesting options can be difficult due to user error, overfitting, and bad data. The new ORATS Backtester addresses these issues by accounting for path dependency, using accurate entry triggers, providing comprehensive performance metrics, and offering a feature to find similar backtests. Step into the future of backtesting with ORATS, and let your trading strategies be guided by data, not guesswork.

Backtesting options is a notoriously difficult process, prone to user error, overfitting, and bad data. That’s why we’ve done all the hard work for you, and ran millions of backtests spanning all types of strategies. After 20 years of being in the backtesting business, we’ve learned a thing or two about common problems traders run into. That’s why we’ve paid close attention to the following problems: a. Overfitting and b. Quality of data and returns. The remainder of this blog covers our approach to solving those two problems.

Find Similar Backtests

At ORATS, we believe in the value of learning from the past to better navigate the future. That's why we've incorporated the feature to find similar backtests in our tool. This allows you to identify and compare your current backtest with other backtests that have similar parameters. It provides you with an immediate context and helps identify potential issues that might not be obvious in isolation. It's a proven way to avoid overfitting, as it encourages the use of parameters that have worked well across different market conditions, rather than those that only performed well in a single backtest. In the example below, I’m looking for similar backtests with the same DTE, strike deltas, spread / stock, and spread / strike diff, but with different entry and exit triggers. This way, I can check for overfitting by seeing if those similar backtests had a wildly different outcome.

Path Dependency

Path dependency refers to how the order and timing of trades can significantly affect a strategy's performance. Ignoring this aspect can lead to misleading results and unrealistic expectations. Our backtester tool takes into account path dependency in a unique way, by only putting on one trade per day, every day it meets the entry criteria. This means you can have 10 trades on at the same time, all entered one day after the previous one. While this isn’t how most people trade in real life, it provides more accurate performance metrics because it eliminates any statistical bias that would occur if you only entered one trade at a time. For example, starting a backtest on the first day of the year might have very different results than starting the backtest a few weeks later.

Entry Triggers

Ensuring you have accurate entry triggers is another critical aspect of successful backtesting. Inappropriate triggers can result in substantial discrepancies between the backtest results and actual performance. We’ve identified five entry triggers that represent a variety of different market conditions. These are:

  • VIX Price
  • Simple Moving Average
  • 14d RSI
  • IV Percentile 1 Year
  • Slope Percentile 1 Year

In addition, we limit the trigger choices to just a few preset values for each one - low, moderate, and high. If more choices were tested, we run the risk of overfitting based on very niche market conditions. We also limit each test to two entry triggers, so that we don’t have so many that the backtest rarely runs.

Performance Metrics

To ensure that you're not falling into the trap of overfitting or basing your strategies on poor quality data, our backtester tool provides a comprehensive suite of performance metrics. It delivers detailed reports about the performance of your strategies, such as the Sharpe ratio, drawdowns, and various other risk-adjusted return metrics. Of these metrics, “Percent of Time in Market” and “Total Strategy Trades” can be used to filter down backtest results to those that have a large sample size. For example, if you set the minimum Percent of Time in Market to 40%, this will limit results to backtests that were in the market at least 40% of the time. Doing this ensures that the backtests you find have a statistically sound number of trades.

Conclusion

At ORATS, we understand that successful backtesting is both an art and a science. With our new backtester, we've combined decades of market expertise with advanced technological solutions to help you navigate the common problems associated with backtesting. By addressing issues such as overfitting, data quality, path dependency, and the accuracy of entry triggers, we've developed a tool that's capable of delivering reliable and realistic backtest results. Step into the future of backtesting with ORATS, and let your trading strategies be guided by data, not guesswork.

Disclaimer:

The opinions and ideas presented herein are for informational and educational purposes only and should not be construed to represent trading or investment advice tailored to your investment objectives. You should not rely solely on any content herein and we strongly encourage you to discuss any trades or investments with your broker or investment adviser, prior to execution. None of the information contained herein constitutes a recommendation that any particular security, portfolio, transaction, or investment strategy is suitable for any specific person. Option trading and investing involves risk and is not suitable for all investors.

All opinions are based upon information and systems considered reliable, but we do not warrant the completeness or accuracy, and such information should not be relied upon as such. We are under no obligation to update or correct any information herein. All statements and opinions are subject to change without notice.

Past performance is not indicative of future results. We do not, will not and cannot guarantee any specific outcome or profit. All traders and investors must be aware of the real risk of loss in following any strategy or investment discussed herein.

Owners, employees, directors, shareholders, officers, agents or representatives of ORATS may have interests or positions in securities of any company profiled herein. Specifically, such individuals or entities may buy or sell positions, and may or may not follow the information provided herein. Some or all of the positions may have been acquired prior to the publication of such information, and such positions may increase or decrease at any time. Any opinions expressed and/or information are statements of judgment as of the date of publication only.

Day trading, short term trading, options trading, and futures trading are extremely risky undertakings. They generally are not appropriate for someone with limited capital, little or no trading experience, and/ or a low tolerance for risk. Never execute a trade unless you can afford to and are prepared to lose your entire investment. In addition, certain trades may result in a loss greater than your entire investment. Always perform your own due diligence and, as appropriate, make informed decisions with the help of a licensed financial professional.

Commissions, fees and other costs associated with investing or trading may vary from broker to broker. All investors and traders are advised to speak with their stock broker or investment adviser about these costs. Be aware that certain trades that may be profitable for some may not be profitable for others, after taking into account these costs. In certain markets, investors and traders may not always be able to buy or sell a position at the price discussed, and consequently not be able to take advantage of certain trades discussed herein.

Be sure to read the OCCs Characteristics and Risks of Standardized Options to learn more about options trading.

Free Trial
Get a free trial of ORATS trading tools by completing this quiz
Take the quiz
ORATS University
ORATS University
Master the art of options
Research
Implementation
Risk
Review
Contact Us
Curious about enterprise pricing? Want to become an affiliate? Questions about our data? Let us know.
Your email
Your message
Submit
ORATSORATS
Institutional Quality Tools for All Options Traders
(312) 986 - 1060
support@orats.com
36 Maplewood Ave, Portsmouth, NH 03801
Trading Tools
APIs
Historical Data
More Information
The opinions and ideas presented herein are for informational and educational purposes only and should not be construed to represent trading or investment advice tailored to your investment objectives. You should not rely solely on any content herein and we strongly encourage you to discuss any trades or investments with your broker or investment adviser, prior to execution. None of the information contained herein constitutes a recommendation that any particular security, portfolio, transaction, or investment strategy is suitable for any specific person. Option trading and investing involves risk and is not suitable for all investors. For more information please see our disclaimer.
Interactive Brokers is not affiliated with Option Research & Technology Services, LLC and does not endorse or recommend any information or advice provided by Option Research & Technology Services, LLC.