Nikolay Falaleev, Sportlight’s head of artificial intelligence since 2020, leads AI initiatives that deliver on-field information for our data science team to analyze, interpret and develop into actionable insights. Falaleev’s diverse background, which includes the research and development of applied CV systems for real-time sports analytics, brings new perspectives to bear on the challenges of athletic performance tracking.
Can you tell us a bit about your background, previous experience and successes in the field of AI?
My academic journey began in the area of nanotechnology, with a focus on the physics and chemistry of semiconductors. But my curiosity and practical needs led me to venture into the world of machine learning for processing experimental data, eventually steering me towards the field of computer vision and deep learning. My initial foray into the field resulted in a prize-winning submission to a challenge on image-based navigation for autonomous cars. This marked my introduction to training deep learning models back in 2016. (It seems there are a lot of transferable know-hows in carrying out experiments and processing data in materials science and deep learning!). Since then, my journey has been marked by a series of innovative AI projects. These ventures typically involved both research and practical implementation of something novel in real-world commercial settings.
One standout project involved automatic real-time score reasoning and data extraction for table tennis. During this endeavor, we with friends developed a pioneering artificial neural network architecture, which combined temporal and spatial modalities for video processing, essentially pioneering AI-based 360-degree video analysis. Our system, capable of discerning what happened, when, and where in a video, was successfully deployed in 24/7 operations at various venues worldwide, making it a valuable contribution to the field, even finding its way into students' textbooks.
Since then, my interest in autonomous vehicles has come into play at Sportlight, which utilizes the same LIDAR sensors used in many self-driving car designs for perception – but, in this case, for providing industry-leading accuracy in player tracking data.
Can you discuss the potential impact of AI and deep learning on the sports industry and the analysis of elite athlete performance? How do you envision these technologies being used to enhance performance analysis and provide deeper insights?
We find ourselves in what I like to refer to as the “post-Moneyball” era of elite sports. While Moneyball introduced data-driven decision-making, AI and deep learning take the concept several steps further. These technologies can seamlessly integrate and analyze data from numerous sources, including player tracking, biometrics, video footage and even social media sentiment analysis.
One of the pivotal aspects of these technologies lies in their ability to collect and process vast quantities of data at a scale and speed that surpasses human capabilities by orders of magnitude, while using manageable amounts of resources. These technologies can, to a certain extent, level the playing field. With the right AI tools and platforms, a smaller organization can harness the power of data analytics that was once the exclusive domain of larger, resource-rich teams.
Here's how I envision these technologies making a profound impact:
1. Performance optimization: AI can provide real-time insights into player performance, allowing coaches and managers to make adjustments during training sessions and matches. It can identify patterns and anomalies that may be overlooked, or not apparent to the naked eye, helping athletes and clubs fine-tune their skills and strategies.
2. Injury prevention and player well-being: AI can analyze biomechanical data to identify potential injury risks and assess potentially adverse events – which is nearly impossible to gather manually – and even suggest adjustments to mitigate them. This proactive approach can significantly reduce injury occurrences.
3. Tactical analysis: Deep learning can enhance tactical analysis by identifying nuanced patterns in player movements and team strategies. Coaches can adapt game plans based on AI-generated insights, gaining a competitive edge by analyzing past results (even those of opponents).
4. Simulation-based strategy development: This is arguably the most futuristic AI application in sports, empowering coaches to game out and test novel tactical tricks, formations, and strategies – all without the inherent risks of experimenting during real matches. This transformative approach may reshape the sports landscape, generating innovative strategies that push the boundaries of tradition – and even what’s conceivable.
5. Scouting and recruitment: Leveraging the massive volume of data AI can collect, AI-driven player scouting can identify talents that might have been overlooked by traditional scouting methods.
6. Fan engagement: AI can also enrich the fan experience by providing real-time statistics, interactive visualizations and immersive augmented reality experiences. This not only enhances fan engagement but also generates additional revenue streams for sports organizations.
What is your department's main focus area and how do you collaborate with other departments at Sportlight to implement AI solutions and drive innovation?
My team’s responsibilities encompass two major areas of focus: First, we conduct research in the field of deep learning to develop the necessary artificial neural networks for data collection. For example, we have developed a solution for player identification to discern individual players on the pitch, an algorithm for recognizing player actions and game events, and a system for tracking player movements and data collection on biomechanics.
Secondly, we are actively involved in engineering a seamless inference system for deploying our neural networks in production across stadiums. The challenge of translating research results into widespread deployment is nearly as intricate as the development and training of the AI models themselves. This complexity arises due to the variability in on-field conditions and the stringent reliability requirements we must meet, as we must match and incorporate a club’s valuable existing data.
As an example, we collaborate extensively with our Sportlight data science team to define data requirements and ensure we provide the necessary data from matches. Additionally, I engage with the engineering team to seamlessly integrate AI components into our products and guarantee smooth operations. Regular interactions with our product development team help align our AI solutions with the company's overarching goals and vision. This collaborative approach allows us to remain at the forefront of innovation in the sports AI industry, consistently enhancing our products and services to meet the evolving needs of our customers and partners.
Looking ahead, what are some of the future tasks and challenges you foresee in the field of AI for your team and the company as a whole?
We are facing a growing need for new data types to be collected. This is driven by both the evolving demand of sports, as they become increasingly data-driven, and the constant advancement of AI technology. For instance, there is an increasing need for near real-time data of individual players’ physiological metrics to facilitate decision-making.
Another key challenge on the horizon is ensuring the robustness and adaptability of our AI models in newer conditions and sports beyond football. For example, we are actively exploring the application of our AI components in the NBA, which means we must continuously refine our models to account for variations and ensure that our systems operate effectively across different scenarios and conditions.
Can you share some details about the major data science competition SoccerNet Camera Calibration 2023 and the solution your team developed? What were the key factors that contributed to your team's victory? What outcomes of the challenge for Sportlight do you expect?
The SoccerNet Camera Calibration 2023 competition was a significant event organized as part of CVPR, a major academic computer vision conference. The primary objective of this challenge was to advance the field of football understanding through computer vision. In the track in which we participated, the goal was to automatically derive camera parameters, including camera position with respect to the pitch and lens parameters, from a single frame of a football broadcast.
Our team's approach focused on precise landmark recognition on the pitch, utilizing various elements such as lines, curves, and even the goalposts. This approach proved highly successful, showcasing a substantial improvement in achieving the highest level of accuracy in camera calibration.
Key factors that contributed to our victory included methodical experimental work and expertise in developing and fine-tuning Deep Learning models. An established experimental framework facilitated experimentation with minimal engineering efforts, and our prior experience in camera calibration played a pivotal role in our success. Essentially, calibration is a cornerstone task at Sportlight, as the accuracy of our data heavily relies on the quality of our system calibration.
In terms of outcomes for our team and Sportlight as a whole, there are both skill enhancements and practical benefits. The developments made during this competition can be leveraged for multiple internal applications, including a system for monitoring our cameras position and automatically adjusting camera parameters online. These advancements will further enhance the robustness and accuracy of our product, reinforcing our position as a leader in AI-driven sports analytics.
What is your approach to fostering a culture of innovation and continuous learning at Sportilght?
To stay competitive, we understand that what was considered a strong technology solution yesterday may not meet the requirements of tomorrow. So we make it a standard practice to regularly review our existing solutions, encouraging our team to question and refine approaches continuously to keep up to date with the development of artificial intelligence. We employ several strategies to promote ongoing learning and innovation within our team, including on-demand courses to expand our skill sets. Additionally, we have established a practice of reading, discussing and implementing recent AI research papers, ensuring that we’re able to integrate cutting-edge techniques into our projects.
One particularly effective approach to nurturing learning and innovation is participation in competitions like the SoccerNet Challenge we won this summer. Such competitions resemble a complete development cycle for an AI product, serving as a bootcamp for junior team members and a playground for senior professionals to keep updated on industry trends. Competitions closely mirror the realities of the AI business world in miniature scale. While it’s possible to achieve 80 percent of results with 20 percent effort, real customers expect us to excel in resolving the challenging 20 percent. Similarly, in competition settings, providing a baseline solution with an off-the-shelf model won’t suffice. Advanced solutions, resolving edge cases and paying attention to detail and synergy with domain knowledge are essential. The time constraints of competitions also simulate the business environment, where delivering the best solution inside prescribed deadlines or before a startup runs out of money is critical for success. Such competitions teach us to test hypotheses quickly and develop professional intuition to discard unpromising ideas earlier. Competitions provide instant feedback and the motivation that drives progress.
Speaking about hard skills, I personally gained most of my skills through similar contests, and I advocate this path to anyone aspiring to excel in AI within a reasonable timeframe. It's an efficient learning method that combines a deep dive into theory with practical experience, as deep learning is, in essence, an art of experimentation, which requires understanding of the underlying concepts and processes. Competitions offer instant feedback on how good your solution is by the position on the leaderboard and the feeling of friendly competition is, to my mind, one of the strongest motivational forces of progress!