A two-part series on how accurately can AI predict who will win the Premier League.
Part 2
AI Predicts Premier League Champion: Was It Accurate?
Originally published on AI Business: https://aibusiness.com/ml/ai-predicts-premier-league-champion-was-it-accurate-
May 30, 2024
The English Premier League concluded recently with Manchester City winning the league. Last August, AI predicted the Sky Blues would win the 2023/24 Premier League season — but how accurate were those predictions?
AI Business previously partnered with Kickoff.ai to predict the 2023-24 season, using its machine learning model to predict the probability of each team’s likelihood of winning the title.
AI gave the Manchester City team the highest probability of winning the league. Manchester City did go on to win and became the first English side to win four Premier League titles in a row.
Liverpool, the team with the second highest probability of winning the league finished third, with Arsenal the team that ended up chasing Manchester City for the title until the final day.
Surprise package Aston Villa came in fourth, finishing above top-six regulars like Tottenham and Manchester United, despite the AI giving them only a 13% chance of securing a top-four finish.
Here’s a look back at the 2023/24 Premier League season, looking at what the AI got right and wrong.
The predicted probability of the Top 6 teams from last August (L) and the final top four (R) | AI Business/Kickoff.ai
Final Score? “Not Bad”
Kickoff.ai’s predictions largely got the 2023/24 Premier League season right: Manchester City winning and three correct teams in the four.
“The top of the table was quite in line with the model’s expectations,” said Kickoff.ai’s Victor Kristof, who is also the founder and CEO of DemoSquare.
The results look a bit different at the bottom. The AI model predicted Burnley would have a strong season. In reality, Vincent Kompany’s Clarets finished second from bottom, his side having struggled to adapt following their promotion to the Premier League.
For relegation, Kickoff.ai’s model had marked Bournemouth and Nottingham Forrest, but the clubs ended up finishing 12th and 17th, respectively.
“In terms of looking at the outcomes, the lower part of the table was a little bit more surprising,” said Kickoff.ai’s Lucas Maystre, who is also a research scientist at Spotify.
Villians Are Anomalies
There were several surprises this season. Premier League fans witnessed Unai Emery turn Aston Villa into a force to be reckoned with since he took over in October 2022.
Aston Villa exceeded the AI model’s expectations, but outliers will always appear.
Leicester City, for example, famously won the Premier League in 2015/16, a huge shock considering the bookies had them at odds of 5000/1 to lift the trophy.
“It’s kind of expected that one or two such things are going to happen every season,” Maystre said.
Another of the AI’s predictions was that Wolverhampton Wanderers were going to struggle. Their manager Julien Lopetegui departed on the eve of the Premier League season over a lack of transfer funds with Kickoff.ai’s model suggesting they would struggle.
However, under the steady stewardship of Gary O’Neil, Wolverhampton finished a respectable 14th despite having a low net spend.
In cases like Wolves changing manager, for example, how does an AI prediction model account for such changes?
Kickoff.ai runs live predictions every week on their website, factoring in changes like recent momentum and managerial changes.
“We have this dynamic rating for a team that takes into account changes that have been happening,” Maystre said. “That’s the difficult thing when you’re predicting one year ahead. The AI hedges its bets, that’s why we saw a broad range of teams listed as plausible contenders for a top-four finish, for example.”
How Does it Work?
Kickoff.ai created a statistical machine learning model that uses data from previous season performances, the team modeled the outcome of matches to figure out where teams would end up.
It employs a method dubbed “Kickscore” – a rough measure of a team’s performance over time. Kickscore encodes how a model “sees” a team based on the data it has. The Kickoff.ai website explains: “As it is dynamic, it is possible to interpret how a team’s strength evolved over the past decades.”
Predicting the Inevitable
Premier League fans expect Manchester City will win the league. They’re the richest club on Earth and have one of the greatest soccer coaches ever at the helm of a roster of superstars.
How impressive is it if an AI can predict what every fan knows will happen every season?
“Any sensible predictor would pick the favorite to win,” Maystre said. “What’s challenging is being able to understand what are the odds of an outlier. When we discussed these predictions last August, I thought the 61% to Man City was bold. I would prefer our model to have given a slightly lower probability, not just because I’m an Arsenal fan, but because that was a lot of eggs in one basket.”
“We’re not making a prediction in the sense that we’re saying Manchester City will win, we’re just estimating the probability of this outcome to happen,” Kristof said. “A model would never say Leicester City would win the league, that’s impossible, but it is interesting to look at the odds of that outcome actually happening.”
Manchester City winning the league was no surprise, but the pair suggested that in the 2023/24 season, no real surprises occurred.
“The AI is trained to reason, to learn and make predictions and minimize surprises. If it puts a very high probability on one thing and that happens, then the surprise is minimal. But if it puts a high probability on one thing and then something else happens, that’s a big surprise, Maystre said. “If Man City had been relegated, something that our model would have given probably a one in a thousand-year probability, that would have been maybe a big red flag for our model.”
Manchester City being relegated might seem like an extreme example but there’s some logic there. The club faces 115 charges over allegedly financial irregularities, something the club vehemently denies. If found guilty, they could face a hefty points deduction.
Financial implications had a major impact on the Premier League table this season, with Everton and Nottingham Forrest both losing points over breaching the league’s Profit and Sustainability Rules (PSR).
Kickoff.ai’s model did not account for these off-field issues, but with Everton facing further financial uncertainty and newly promoted Leicester City facing the prospect of starting the season with a points deduction for PSR breaches, these could become a major consideration ahead of making future league predictions.
Euro 2024: Analyzing Leagues vs. Knockout Tournaments
The Premier League is over, but the soccer continues this summer as the European Championships head to Germany.
The Kickoff.ai team plans to use its AI predictions model throughout the upcoming tournament.
Unlike the Premier League, the Euros are a knockout tournament. Just one loss can decide a team’s fate and in the knockouts, anything can happen — like England’s 2016 exit to lowly Iceland.
But is it easier to predict a league table compared to a knockout tournament?
“For a league, we have more matches to train the model on,” Kristof explained. “For an international competition, it’s much more scarce. We can train only using the qualifying rounds or friendly games, which are not always very representative of what’s happening during a competition.”
The Kickoff.ai project began around the time of the 2016 Euros and was used to accurately predict the probability of games at the World Cup in Russia two years later.
“One thing that’s different with a league is that because every team plays so many games, the position and the table tend to be a little bit more stable as a function of slight variations in team form over the year compared to a knockout tournament where just a single loss can knock out a team,” Maystre said. “You’d never see our Euros model give a 61% chance of winning to a single team. The highest chance you would give to any one team would be around 20 to 30%.”
What’s Next? Generative AI, XG
Predictions themselves have become a mainstream part of the soccer viewing experience through xG or expected goals — a metric that assesses the quality of scoring chances based on factors including the position from which a shot at goal is taken.
XG has gone from an obscure piece of information reserved for soccer nerds and data analysts to a concept that fans interact with and understand. XG stats are routinely displayed during live games.
AI has the potential to take xG predictions one step further.
“XG takes away some of the randomness of predictions,” Maystre explained. “Extending this idea to every event is exciting. Every single decision a player makes on the field moves the needle one way or another.
“If we started having a really good understanding of the importance of a defensive interception or a run that creates space for another play in a data-driven way that’s powered by AI insights, they would be great for spectators because it will bring a lot of insights into the game.”
Since Kickoff.ai made its Premier League predictions last August, not much has changed in the way of improving prediction modeling.
But amid the generative AI wave, could this have an impact on the future of AI’s ability to predict the probability of sporting events?
The Kickoff.ai team suggests generative AI could help compile previously overlooked data into their prediction models
“Generative AI will just be much better at mining sentiments on Twitter, like people’s belief about a game or rumors about injuries, things we struggle to currently feed into our model because it’s unstructured and scattered all around the internet,” Maystre said. “I feel that eventually with progress in large language models, for example, these things will find their way into sports prediction models like ours.”
Away from generative AI, the team spoke about the potential for computer vision to analyze games in real-time to track players, the movement of the ball and a team’s shape and strategy to provide a more accurate prediction of a potential result.
Real-time probability predictions are already displayed during Premier League broadcasts — but they aren’t always accurate.
During the 2023/24 season, for example, Bournemouth found themselves 3-0 down against Luton Town in March. Oracle provides the Premier League’s live prediction probability and after the third goal, Bournemouth was given a 97.6% chance of losing the game.
Bournemouth went on to win 4-3 in a match that saw their opponent have the highest win-probability percentage ever recorded for a team that didn’t end up winning in a Premier League game.
Not even AI in its current form could have predicted Bournemouth would win that game. But with innovations in computer vision and generative AI, future sports prediction models could be able to identify dramatic swings in games. And it’s something that Kickoff.ai is looking at.
“We explored with regards to some ideas to make predictions in real-time during games, Kristof said. “That would be the next stage for us, to track what’s happening during the game into adaptive probabilities of the final score. It’s definitely something we’d like to deploy at some point.”
Manchester City is already the bookie’s favorite to win the 2024/25 season in what would be Pep Guardialo’s final season and a fifth consecutive title. But AI could soon factor in disparate data sources, picking up on something the bookies might have missed and identify an unlikely winner.
Part 1
AI Picks Manchester City to Win the Premier League
Originally published on AI Business: https://aibusiness.com/ml/ai-picks-manchester-city-to-win-the-premier-league
August 8, 2023
The English Premier League returns this weekend, with Manchester City looking set to win their fourth Premier League title in a row as AI predicts the 2023/24 season.
Pep Guardiola’s side won the treble last year and have eyes on their sixth title in seven seasons. AI Business partnered with Kickoff.ai to predict the 2023-24 season, with Erling Haaland, Kevin De Bruyne and company the far-off favorites to clinch the title.
City will start their title defense at Burnley who are managed by former captain Vincent Kompany. The Sky Blues have spent big on Joško Gvardiol, signing the Croatian defender from RB Leipzig in a deal worth £77.6 million ($98 million), as well as bringing in Mateo Kovacic from Chelsea for £25 million ($30 million).
However, City will be without midfield maestro İlkay Gündoğan who left the club for Barcelona, as well as winger Riyad Mahrez, who joined the growing number of stars in Saudi Arabia.
Kickoff.ai’s machine learning model picked Jürgen Klopp’s Liverpool to recover from their poor fifth place finish last time to return to challenging City.
Liverpool has spent the summer rebuilding their aging midfield, turfing out stalwarts Jordan Henderson, Fabinho and Naby Keita for young promising talents including Alexis Mac Alister from Brighton and Leipzig’s Hungarian star Dominik Szoboszlai. The club is also circling Southampton’s former City midfielder Romeo Lavia, Brighton’s Moises Caicedo and Fluminense’s Andre Trindade.
Despite Klopp’s midfield rebuild, the AI model calculated City were almost five times more likely to win the title than Liverpool, with the Sky Blues’ financials might likely tipping them past their title rivals.
Mikel Arteta’s Arsenal were the surprise nearly men last time out, look set to finish behind Liverpool and Man City in third. Arsenal looked all but certain to win the league last year before falling away in the Spring, allowing City to catch up.
To push the Gunners on, Arteta has signed West Ham’s Declan Rice for a club and British record transfer worth up to £105 million ($133 million). Arsenal has also brought in Chelsea attacking midfielder Kai Havertz and Ajax defender Jurrien Timber and is in talks with Brentford over £40 million ($50 million) goalkeeper David Reya.
Arsenal may have defeated City on the weekend in the Community Shield at Wembley, but the North London side will need more than penalties to challenge last season’s treble winners.
Rounding out the Champions League spots is Erik Ten Hag’s Manchester United. The Red Devil’s title hopes look slim, however, with the machine learning model calculating a title probability of just 3% compared to the neighbor’s city with 61%.
Ten Hag’s United will be joined by Rasmus Højlund. The 20-year-old striker joined from Atalanta in a £72 million ($91 million) deal to lead the line for a United side desperate for goals. Marcus Rashford was the club’s leading Premier League scorer with 17 goals, but no other player scored over 10 goals last season. Højlund scored 10 goals for Atalanta last season and is largely seen as a long-term attacking option for the club.
Saudi-owned Newcastle was the highest of the non-Champions League clubs, pipping a potentially Harry Kane-less Tottenham and topsy-turvey Chelsea, who finished lowly eighth and 12th last season, respectively.
Brighton, Aston Villa and Brentford look set to continue their fine forms last season, but the machine learning model places them below the bigger boys this season. West Ham, who won their first trophy in 43 years after clinching the Europa Conference League last year, are just behind the trio, though a lack of transfer activity due to issues in the club’s hierarchy could see them fall behind.
Facing the drop according to AI is new boys Luton Town. The club is set to play in the Premier League for the first time in its history. Just five years ago, the Hatters won promotion from League Two, the fourth tier of English football, and now find themselves against some of the biggest names in the game.
Also predicted to be relegated is Nottingham Forrest. Forrest famously brought in an entirely new squad when they came up from the Championship last season. Steve Cooper brought in 30 total players throughout the season to help them save the drop, including former Man Utd midfielder Jesse Lingard and Morgan Gibbs-White from Wolves for a whopping £42.5 million ($54 million).
Sharing Forest’s probability of being relegated is Bournemouth, who fired Gary O’Neill despite saving them from the drop last time out. O’Neill was shown the door by owner Bill Foley in favor of Rayo Vallenco manager Andoni Iraola.
The next likely team to get relegated is Wolves, who saw manager Julien Lopetegui depart by mutual consent. The former Spain coach was unhappy with the lack of funds to expand his squad, having only brought in former Wolves full-back Matt Doherty on a free transfer. Newly promoted Sheffield United, back in the big time since 2021, are also predicted to go down.
AI predicts the Premier League: How does it work?
For the 2023/24 Premier League season, the team at Kickoff.ai used machine learning to predict the probability of the season’s results.
The minds behind the system are Lucas Maystre and Victor Kristof. The pair met at EPFL (École polytechnique fédérale de Lausanne) in Switzerland. Alongside Kickoff.ai, Kristof is the founder and CEO of DemoSquare, an online platform for automating the monitoring and analysis of legal and political data, and Maystre is a research scientist at Spotify.
The pair employ a statistical model of football matches. Taking data on previous season performances, the team modeled the outcome of matches to figure out where teams would end up. The model currently ignores the squad, so it won’t take into account transfers that have been made over the summer.
They employ a method dubbed ‘Kickscore’ – a rough measure of a team’s performance over time. Kickscore encodes how a model “sees” a team based on the data it has. The Kickoff.ai website explains: “As it is dynamic, it is possible to interpret how a team’s strength evolved over the past decades.”
Kristof and Maystre began the project in 2016, working on statistical models to predict the results of the 2016 Euros in France.
The pair perfected their model, applying it to the 2018 World Cup in Russia and have been running their prediction engine for Europe’s top five football leagues ever since.
Kickoff.ai’s platform is mainly used by sports fans looking for a “hot tip,” the pair said, but added it could be used to detect possible match-fixing, with the model able to identify potentially suspicious results. TV broadcasters have also shown an interest, to provide context to viewers.
Data is already making huge strides in football. Teams like Brighton and Brentford have improved on the pitch by utilizing huge data projects off it to improve player monitoring and scouting, which is something the Kickoff team is exploring using a machine learning model focused on individual players – analyzing particular skills or strengths – to further augment scouting.
Results analysis: Can anyone stop Man City?
Up until the Newcastle takeover, Man City have immense financial clout. The Abu Dhabi-backed outfit is at the top of the Deloitte Money League as the richest football club in the world and routinely brings in some of the best talent in the world.
Reacting to the AI predictions, Lucas admitted surprise at the “magnitude” of the probability that Man City would win the 23/24 title, saying: “I wasn’t surprised they would come out on top but being five times more likely than Arsenal sounded like a lot.”
The pair noted that the model may have some difficulty predicting teams that came up from the Championship, like Burney. Kompany’s Clarets raked up 101 points last season, routinely blowing opposition sides away.
Kickoff.ai’s model picked up potential regressions for historically larger teams, like Chelsea and Tottenham, who have underwhelmed of late.
After winning the Champions League the season before last, Chelsea sacked coach Thomas Tuchel mere weeks into the season and brought in Graham Potter from Brighton, only to later sack him, all while spending £585.5 million ($745 million) on player transfers.
Maystre said, “It’s hard, it feels like this is not going to be the season where Chelsea does something great. It’s interesting what the model is telling us as it’s maybe not what I would have instinctively said myself.”
Tottenham meanwhile has been in decline since reaching the 2019 Champions League final. Despite England’s all-time leading goal scorer Harry Kane leading the line, high-pedigree coaches including Jose Mourinho, Antonio Conte and Nuno Espirito Santo all failed to push the team onto glory.
“It could be interesting to run this simulation again after five, 10 and 20 weeks, just to see how things change compared to the initial predictions,” Kristof added.
Predicting the Premier League is no easy feat. No one could have envisioned lowly Leicester City taking home the trophy with odds of 5000-to-1 in 2016, but it happened. AI could improve on the predictions, but the unpredictability of the Premier League is what makes it the best football league in the world.
Ben Wodecki 