How to Test For Overfitting

Businessman drawing a stock chart

A common problem that we encounter is members who will custom-fit their data to produce winning systems.  What this means is that people will go through our historical database and cherry pick the best data points to get a large number of units won and a skyrocketing graph.  While this may give you confidence that you’ve found the Holy Grail of sports betting, it’s usually just a case of overfitting.  Hell, I’ve been guilty of this myself when searching for social media content.  A 70% win-rate grabs your attention but if you ride the regression train back towards 50%, you’re going to have a bad time.

So how can we avoid this problem?  There’s something that you can do to see exactly how your system would do going forward.  It’s a filter within Bet Labs called Season.  When you start to build a system, it defaults to show all of the past data that we have.  But using the Season filter, you can exclude some of the most recent seasons from your data set.  Once you do this, you can build your system without those seasons being part of the results.  Once you are finished creating your system, then you can go back to the season filter and see how the results fared in those most recent seasons.

It’s a way of time traveling back to an earlier year, then seeing how your results would have done.  After all, that is what you are trying to do now.  You are trying to create a system that has performed well that continues to perform well over the next few seasons.  Let’s take a look at an example:

overfitted-system2

So here I have an NFL spread system that had covered 62.8% of the time.  A $100 bettor would have made over eleven grand and you are wondering how long it will be before you are diving into a pool of money with Scrooge McDuck.  But as you probably anticipated, the results since this point have been less than stellar.  By going back and editing the Season filter, we can see how the system performed starting in the 2013-14 season:

season-filter-1

The record since that time has been just less than 50% and the vig hasn’t been kind.  One way we try to combat users from doing this is by giving them a grade for each system.  If you noticed, the system above had a grade of “D” despite the good results and graph shooting skyward.  If you actually click on the grade, our software will tell you exactly why you received the grade you did.  In the example system, here is what it shows:

grade-details

As you can see, the software doesn’t like it when you go into a list and start picking through the best quarterbacks, best coaches, or even the best closing spreads.  While this system may have seemed obvious to fail, some of them are borderline and much tougher to tell.  By leaving out a recent season (or two or three) until after you are done creating the system, you may save yourself from betting a system that is unlikely to succeed going forward.

Ready to start creating your own systems?  Getting started is easy and doesn’t require a credit card.

Build your system for free

 

 

©2016 Sports Insights Inc.

Log in with your credentials

Forgot your details?