Last week was my first try at using the statsmodels in Python to try and predict winners for NFL games and compare them against Money Line odds found on Pinnacle. If we use 50% as the threshold between winning and losing a game, my model went 10-for-16 in last week’s games. Now, that’s not against the spread, just the Money Line odds. That’s good, though, right. Basically I was right on 63% of the games played.
But there are other methods for scoring predictions based on probabilities. One of those is Log Loss which is a scoring method I became familiar with when participating in the 2018 NCAA Men’s Tournament Challenge on Kaggle. Log Loss penalizes predictions that are confident about an incorrect outcome (see graph below).
For example, for the Arizona-Green Bay game, I predicted that Arizona had a 1.54% chance of beating Green Bay at Lambeau Field. Arizona won that game. That means that I had predicted a very small probability to what ended up being a correct outcome (also can be stated that I was highly confident in Green Bay who lost so that turned out to be an incorrect outcome). My model was extremely confident that Arizona was going to lose…but they didn’t. In this case, the Log Loss turned out to be 2.65. You can see from below around 0.0154, the Log Loss is high and gets higher for highly confident but wrong outcomes.
However, on the flip side, in the Los Angeles Rams-Detroit game in Detroit, my model gave the Rams a 95.57% chance of winning. They won and since my model was highly confident that they would, the Log Loss was 0.03, which is fantastic. I want to minimize Log Loss here so being close to zero is good.
Unfortunately, a Log Loss between 0.03 and 2.65 seems rather meaningless if you’re not actually picturing that graph in your mind. For instance, for all the games in Week 13, my Log Loss was 0.58. Is that good, bad? It’s relatively close to zero, but there were three games where the Log Loss was 1.4 or higher. Not much context for how good those predictions were. In comes the Brier Score!
The Brier Score works best for outcomes like football games because there is a winner which we can assign a 1 and a loser which we can assign a zero. In this way we can measure the accuracy of the predictions that are given as probabilities. The best part is that the Brier Score for any given outcome lies between zero and one which provides a better context. Closer to zero is good. Closer to one…not very good. The Brier Score for Arizona beating Green Bay was 0.64 (closer to one than zero)…not really good. The Brier Score for the LA Rams beating Detroit was 0.00106 (very close to zero) which was very good. Overall, the Brier Score for my Week 13 predictions was 0.18. Not bad if I do say so myself.
Anyway, sorry for the little lesson in probabilities and how to score how accurate they are but wanted to make sure I had it all written down before posting my model’s win predictions for the week.
Below you’ll find the latest win probabilities for all the games for the weekend. I also added a few columns including the expected total points that my model is predicting will be scored for the games, and the Over/Under points scored from Pinnacle.
Week 14 Win Probabilities
| Game | Ed's Model | Vegas | Difference | Expected Tot. Points Scored | Vegas Over/Under | Actual Tot. Points Scored |
|---|---|---|---|---|---|---|
| JAC | 26.40% | 34.48% | -8.08% | 39 | ||
| TEN | 67.36% | 67.98% | -0.62% | 31 | ||
| NYJ | 51.30% | 35.21% | 16.09% | |||
| BUF | 42.39% | 67.25% | -24.86% | 40 | 37 | |
| NYG | 28.39% | 60.02% | -31.63% | |||
| WAS | 66.09% | 42.37% | 23.72% | 41 | 40.5 | |
| NO | 95.86% | 80.78% | 15.08% | |||
| TAM | 3.00% | 21.70% | -18.70% | 69 | 54.5 | |
| NE | 89.77% | 78.99% | 10.78% | |||
| MIA | 7.84% | 23.47% | -15.63% | 48 | 48.5 | |
| BAL | 31.45% | 29.76% | 1.69% | |||
| KC | 63.64% | 72.78% | -9.14% | 56 | 51 | |
| IND | 13.26% | 35.09% | -21.83% | |||
| HOU | 83.63% | 67.34% | 16.29% | 55 | 50 | |
| ATL | 35.22% | 36.10% | -0.88% | |||
| GB | 59.50% | 66.36% | -6.86% | 52 | 51 | |
| CAR | 44.08% | 51.71% | -7.63% | |||
| CLE | 50.18% | 50.76% | -0.58% | 49 | 47.5 | |
| DEN | 83.65% | 63.13% | 20.52% | |||
| SF | 12.79% | 39.37% | -26.58% | 43 | 44.5 | |
| CIN | 2.22% | 12.50% | -10.28% | |||
| LAC | 96.94% | 90.91% | 6.03% | 53 | 48.5 | |
| DET | 70.53% | 58.51% | 12.02% | |||
| ARI | 24.03% | 44.05% | -20.02% | 37 | 40.5 | |
| PIT | 98.05% | 50.51% | 47.54% | |||
| OAK | 1.37% | 18.98% | -17.61% | 55 | 51 | |
| PHI | 17.49% | 38.91% | -21.42% | |||
| DAL | 77.93% | 63.53% | 14.40% | 37 | 45.5 | |
| LAR | 58.17% | 59.21% | -1.04% | |||
| CHI | 36.83% | 43.10% | -6.27% | 60 | 51.5 | |
| MIN | 9.97% | 40.16% | -30.19% | |||
| SEA | 87.14% | 62.27% | 24.87% | 45 | 45.5 |
All probabilities presented on this page are for entertainment purposes only. Do your own research!