• Artificial Intelligence (AI) in Sports : How IIIT-Hyderabad’s Research is Making Inroads Into Broadcasting Studios
IIIT-Hyderabad, March 6, 2019 : Staying true to its motto of ‘From Lab to Land’, the International Institute of Information Technology, Hyderabad’s Product Labs has successfully developed and tested a viable market prototype of badminton analytics developed at its Center for Visual Information Technology. A pilot study was conducted in collaboration with Star Sports which saw the data going live during the telecast of the Premier Badminton League in January 2019.
Lab to Land
Joining the ranks of cricket in big data consumption are other sports too, like football, tennis and badminton. The Guardian in 2015 had famously stated that tennis analytics is still stuck in the dark ages. Attempting to prove this wrong is world-class research from IIIT-H’s Center for Visual Information Technology (CVIT) in the areas of racquet sports like tennis and badminton. In fact, the next time you witness some on-court action between India’s badminton queens, such as that seen during the recently concluded National Championship, chances are you’ll owe the sophisticated analytics to IIIT-H’s efforts.
Designed as a catalyst to drive innovation and create commercially viable products, IIIT-H’s Product Labs has been working closely with the research centres located on campus. Explaining how research finds its way into the market, Prakash Yalla, head of the Technology Transfer Office and Product Labs says, “We picked sports analytics as our first business case. Application of computer vision in sports is an upcoming area. Under Prof. C.V. Jawahar, CVIT has already conducted enough research on various sports. We zeroed in on badminton because not only had a fairly deep amount of work been done, but there is an excellent badminton eco-system in the city of Hyderabad.” When the Product Labs conducted an extensive market study, it found a three-fold application of the technology. One, as an aid for coaches in sporting academies. Two, an application in the broadcasting space. And a third use case where such heavy-duty analytics would come in handy for planning and strategizing of professional players who would like to profile their opponents.
Sports Analytics in Broadcasting
While audience engagement has always remained the goal of television broadcasting, the nature and methods used in engagement have undergone an evolution over the years. From action replays, to slow-motion replays, from stump cameras (in the case of cricket) and mics revealing various angles, to technologies like hawk-eye that show how the ball or shuttlecock is moving, the broadcasting space has come of age. “Anything that can help the commentator and in turn catch viewers’ attention is of great interest to broadcasters,” affirms Prakash. Hence when Star Sports approached IIIT-H to collaborate on badminton analytics, the institute accepted to perform a pilot study during the Premier Badminton League match.
What technology was used And What It Does
When CVIT first began its foray into sports analytics, it started with the sport that grips the nation’s imagination – cricket. According to Prof. Jawahar, “ Our initial attempts looked at how large cricket videos can be made searchable. We later expanded our efforts and worked on soccer, moving on to racquet games.” Research from this lab has always been newsworthy. In 2015, a Washington Post article covered the work done by IIIT-H where computers provide text-based commentary of cricket matches. The same article also described how Prof. C.V. Jawahar and Mohak Sukhwani demonstrated automatic creation of tennis match commentaries.
Productization of badminton analytics was based on a research paper authored by students of CVIT, Anurag Ghosh and Suriya Singh, under Prof. Jawahar’s guidance. Titled “Towards Structured Analysis of Broadcast Badminton Videos”, the paper proposed a method to analyze a large corpus of badminton broadcast videos by segmenting the points played, tracking and recognizing the players in each point and annotating their respective badminton strokes. Explaining it simplistically, Prakash says that the algorithms can identify when a rally has started, or ended, and who won the rally (player identification). It provides segmentation of the shots played (backhand, forehand, smash) thereby revealing the style of the player. By generating heat maps based on how fast the player(s) moves on court, one can also deduce the dominant player.
Real-life Challenges
In the lab context, the students’ algorithms churned out data on canned video footage which was available for scrutiny an hour or two later. However, this was not practical in real-time high-quality streaming videos. “The other technological challenge was that the broadcaster wanted rally-wise analytics. A rally typically lasts 3-4 seconds, our technology was running sequentially for an hour. So we had to bring an hour-long runtime down to 3-4 seconds!” exclaims Prakash. Wary and unwilling to change the algorithms that had been developed over a period of 2-3 years, the collaborative team led by Prakash with a student from the research centre and an engineer from Product Labs, decided to perform a massive engineering rehaul and came up with a new engineering architecture. Echoing Prakash’s sentiments, Tushar, the Product Labs engineer says that optimization of the algorithms efficiently on a CPU was a tough task. “Plus, we had to work on a continuously growing live feed that couldn’t be simulated on campus,” he says.
The team’s moment of truth arrived during the telecast of the Semi-Final match of the Premier Badminton League. With the successful display on screens of the distance travelled by the players on court, IIIT-H demonstrated that academic research can be applied in real-life scenarios. Speaking about his research paper, Anurag Ghosh, one of the early stakeholders of the project says, “The novelty of the system is that it absolves the need for complicated multi camera setups and relies on a single camera feed. The system should enable automatic low cost analysis of players.”
Lab-Industry Engagement
Predictably, Star Sports is now keen on exploring possibilities of expanding the pilot study to a larger scenario. One of them looks beyond badminton to other sports. “Sports startups is an unaddressed space in our country. And they would like to kickstart startups in this (sports) space. Our lab has a lot to offer the broadcasting industry from that point of view. In fact, after the success of the pilot, thanks to the visibility, we’ve had multiple requests from former students who wish to be involved in setting up sports-analytics-based startups! There lies an entrepreneurial opportunity for anyone who can build a business around this,” muses Prakash.
Acknowledging the role played by Product Labs, Prof. Jawahar says,” Without a formal engagement, we went ahead and conducted a satisfactory pilot. The collaboration with Star Sports itself is an interesting one and we look forward to putting cutting edge technology to real use. Working on data around you is one thing and providing satisfactory results based on it is another thing. We did both.” For the professor, applications of AI that have far-reaching impact make sense. He would like AI in sports to enable prodigies to come up from Indian rural areas. “In order to do that for badminton, one needs to have technology which will scale up Gopichands of the world to millions of students. Computer Vision and AI are the key missing pieces in order for such scalability to happen”, he states.
