Insight7 Consulting Ltd. is a Football consulting firm based in Birmingham, West Midlands. It offers professional cutting-edge expertise in talent identification and promotion, which is founded on many years of experience working with top-tier Premier League Football Clubs.
Insight7 uses historic and present football data in their football analytics to provide their clients with an integrated assessment of players’ performance and future potential. Data alone is not sufficient to make informed decisions, this is where Insight7 expertise comes into play to interpret data and extract actionable information. Data analysis is presented as structured reports that are tailored to clients’ needs.
Data analytics reports delivered by Insight7 support decision-making in early talent identification, recruitment, and development of players. Early talent identification is vital in the competitive and fast-changing world of the football industry, especially for the players whose career progression and success depend on the early development of their football skills.
Insight7 collect data from public football websites that offer historic and present information about player statistics, scores, results, transfers, and fixtures. Data is largely collected and aggregated manually, which is an extremely laborious and time-consuming process. Football data is naturally fast-moving, so to hold an up-to-date database, data needs to be collected and updated continuously.
As the company expands its pool of clients, time spent on data collection and aggregation becomes a major challenge. Each client has its own needs and often requires specific data to be collected before the analysis and reporting can be done. With the current data collection approach in place, there will be a point in time where it might become challenging to offer consultancy services to a larger number of clients.
Insight7 needed a better and more efficient process for collecting large amounts of accurate data fast. There was no commercial off-the-shelf software available to meet Insight7 needs, therefore a tailored approach was required.
The goal for Insight7 was to automate data collection and data aggregation processes, consequently increasing the efficiency, productivity, and time available for other consultancy tasks. Automated data collection would also improve data quality and minimise the risk of human errors caused by repetitive manual tasks.
The Think Beyond Data team agreed to develop a software solution to automatically obtain data from the football websites. The client required software that would be straightforward to install on a standard PC or a laptop and would be easy to use on a regular basis. It was also important for the solution to be flexible and allow for data search and filter functionality to collect the required and fit-for-purpose data.
To solve the defined data collection and aggregation problem, the Think Beyond Data team developed a set of web scraping scripts that collect data automatically and output the results as structured CSV files. Web scraping is the process of automatically extracting and collating data from the web. It helps to automate the time-consuming and often tedious task of manually copying and pasting website data into databases or documents. Web scraping offers a scalable solution that collects large amounts of data in minutes, which would otherwise take hours or even days to obtain if done manually.
The team worked closely with Insight7 to understand how and what data is being collected to replicate the manual processes and used Python programming language, Jupyter Notebook web application, and BeautifulSoup Python library to develop a software solution.
The team first reviewed the internal structure of the web pages to be scraped, this was essential to understand how the data is structured and how different web pages are linked together. The team then developed a set of Jupyter Notebook documents that contained live code, instructions for use, and the functionality to define data search and filter parameters. The solution went through several stages of user testing and refining to ensure that it was fit-for-purpose and satisfies the user requirements.
The solution developed by Think Beyond Data has now been adopted as part of the Insight7 business processes. The new software not only helps to save precious time, but also enables Insight7 to deliver the consultancy reports faster, increasing their customer satisfaction.
As a new start-up, operating in difficult times, the Think Beyond Data partnership came just in time and very quickly became an essential part of the operation.
Insight7 has now been able to expand its data collection operation to scale the business to the next level – supporting more clients, each with broader and deeper data requirements.
The collaboration has been a cornerstone, evolving the business, delivering a robust and extensive platform for further technical advancement.
Working remotely is not always easy, particularly on technical projects, but Victoria Lush of the Think Beyond Data team has been excellent throughout.
Victoria was quick to understand the requirements from the outset, asking key questions and actually helping me to understand the ‘what ifs’ of the project.
Perhaps most important though was her ongoing support, fast response, and ability to turn around bugs (a small number) and enhance the functional elements of the delivery of the project.
Lee Jamison, Insight7 Director