Neural Sports


What the F*ck is AI, Machine Learning, Deep Learning and Computer Vision?

May 14, 20207 min read

Yes sport fanatics today we will be covering the main topics that will appear on this blog - What the f*ck is AI, Machine Learning, Deep Learning and Computer Vision. These terms appear in multiple blog posts, articles and published papers online, but what do they really mean? And how can they being applied in Sports today? PS) Check out the cool generic ‘I code’ cover photo on this post, get used to them.


Self driving cars, robots and chatbots among many other futuristic technologies can be associated to the field of AI. The field of AI (or Artificial Intelligence) can be defined as building systems capable of automating intellectual tasks or decisions that are typically performed by humans. Quite a kerfuffle right? Lets break it down.

‘Building systems’ - A system encompasses all the steps to produce a programme or application that can be used by humans. Steps include identifying the problem, preparing the data, choosing algorithms, training algorithms, choosing a specific programming language and running it on a desired platform.

Frank Lampard against Germany

‘Automating intellectual tasks that are typically performed by humans’ - This could literally be anything. I know that sounds broad and does not answer the question but tasks such as driving a F1 car, calling if a tennis ball is out-of-bounds, identifying if Ben Stokes edged the ball, or if Frank Lampards volley against Manuel Neuer in the World Cup was actually over the line (definitely was). These are all tasks or decisions that humans normally make and can be automated in some way whether it be a referee calling a foul, a test driver driving a new F1 car or an umpire deciding if a cricketer has edged it behind.

Machine Learning

So how does Machine Learning fit into AI? Although the general rule of AI is that we supply a system with well-defined, logical problems, such as playing chess (yes it is a sport!), it turned out to be difficult to figure out explicit rules for solving more complex problems like ‘Will Erling Braut Håland live up to the hype?’, ‘How can we adjust Coco Gauffs serve to maximise the number of aces?’ or ‘How can we predict the score between Celtic vs Rangers?‘. This is where Machine Learning comes into play.

How a Machine Learning system differs to an AI system is it is trained and not explicitly programmed. A Machine Learning system is fed multiple data points that are relevant to a specific task and then comes up with rules for automating the specific task. Lets breakdown what we need to do Machine Learning using an example:

We need to improve a Basketball players shot in the off-season. A basketball players shot heavily relies on their mechanics and the surrounding environment when they are taking a shot. Angle too flat? Error rate is higher due to less space for the ball to fall into. Angle too high? Well, you’d tend to be airballing a lot, along with not being in a lot of fans good books (unless you’re Steph Curry who goes viral for ridiculous high arching shots). Someone right in your face? More than likely you receive the Mutombo finger wag and that shit will get sent to the stands. So as you can see there is a lot going on and a lot that needs to be analysed.

Input data points - For instance we use all the angles that were recorded by some poor sucker throughout the season along with other metrics such as time to release, was it an open shot or was it off the dribble etc.

Examples of expected output - This is where that poor sucker also recorded whether a shot went in or not. So our outputs are simple and binary, yes or no.

Whether our algorithm is doing okay or not? - This is needed to decide how far off our algorithms current output is to the actual output, did a shot go in or not based on all the input data points? This measurement is fed back to the algorithm that gets adjusted, therefore, this is how it ‘learns’. A good measurement for our task is the percentage of shots that were correctly identified by our algorithm i.e it predicted 93% correctly.

So now our Machine Learning system knows whether a shot will go in based on a lot of different factors for a particular player. We can now use this system to analyse our players next training session to receive real-time shot feedback straight away when a player makes a shot and decide what adjustments need to be made to the player to make the shot. This incorporates a bit of Computer Vision which we will discuss in a mo.

So basically that’s it. Machine learning is searching for and making sense of some input data, within a predefined set of expected outputs, using guidance from feedback as to whether our algorithm is doing alright.

Deep Learning

This is where is starts to get more interesting and difficult at the same time. For the time being I shall spare the intense details of how Deep Learning works. But, Deep Learning is a subfield of Machine Learning. You have probably heard of neural networks and then someone saying they’re modelled off of our own brains. This isn’t totally the case, and it’ll only lead to more confusion if you keep linking deep learning in any way to neurobiology. So don’t bother.

Deep learning is a new take on ‘learning representations from data that puts an emphasis on learning successive layers of increasingly meaningful representations’. What do we mean by ‘representations of data’? To keep it simple, lets take this picture of R9 Ronaldo. This image could be split into certain parts, his famous hair, his teeth, the yellow brazilian jersey etc and then these parts could be split into further representations. We can get down to such granular detail such as the specific curves or curls that reappear. Hence each layer has ‘increasingly meaningful representations’


Each layer of a deep learning system holds representation of the data and how many layers we use to represent our data is the ‘depth’. Hence Deep learning. There could be hundreds or thousands of layers that all represent the data in some way or another.

You can think of a deep network as making a cup of coffee using a filter (weird but bear with), where you feed water which is our data or information through our filter that contains our grounded coffee and represents our layers and out comes our coffee which is more purified information (that is, useful with regard to some task).

I’ll go into more depth about deep learning in another blog where we describe the ins and outs but for now this should give you a good overview.

Computer Vision

Another subset of AI and probably one of the most exciting when it comes to Sports is Computer Vision or CV for short. Computer Vision is how computers can gain high-level understanding of what the heck is going on from digital images or videos. Examples include Image Classification, Object Detection, Object Tracking, Semantic Segmentation and Instance Segmentation, all to be explored later down the lines. Computer Vision is most powerful when it can be combined with Machine Learning and Deep Learning to create some powerful applications. Like our real-time Basketball feedback application earlier.

Other examples include the fans favourite Hawk-Eye, Goal Line Technology and definitely 100% the worst possible implementation that is VAR, where technology in that case is just to god damn good.

Computer Vision is exciting and if you wish to learn more about it before I delve into it be sure to check out - who I consider the Michael Jordan of CV.


Well that was a lot to take in! But I hope it offers you some insight into the world of AI and what Neural Sports is planning on explaining down the line. AI is a beautiful field of Computer Science and a lot is yet to still be explored and that’s the exciting part. Combining AI with Sports will unlock what you may at first consider the impossible. However, the impossible is fun.


Deep Learning with Python - François Challet

Developed by Sean O'Connor, a sports and AI enthusiast.