In the post-pandemic age, sports look to a virtual future, in real time
With empty stands, broadcasters and tech firms are banking on Augmented Reality to boost fan experience through ingenious methods like CGI overlays and piped-in video game audio.
It wasn’t long ago that the FIFA video game series boasted a selling point in the ‘realistic noise’ feature; real oohs and aahs, chants, and cheers captured by microphones dotted around stadiums worldwide. The same recordings are now being used to enliven La Liga and Premier League matches behind closed doors.
In the post-pandemic age, the evolution of sports broadcasting has come full circle.
Cardboard cutouts, Zoom video conferences, piped-in video game audio, and CGI overlays. Broadcasters and leagues have tried out anything and everything that would aid a sport-starved fan’s suspension of disbelief.
Next in line are AR/VR — long the buzz acronyms of executives and entrepreneurs — which are now touted as game-changers for broadcasters ravaged by empty stands and disrupted seasons. Experts believe covid-19 has only accelerated the trajectory, and the crisis will drive in the new era of fan experience.
First a crash course. Augmented Reality (AR) superimposes a computer-generated image on a user’s view of the real world (the tech that was all the rage that one summer when kids ran about catching ‘Pokemon’ on their phones). VR (Virtual Reality) simply creates a simulated image for the user.
But before looking ahead at how the tech will change sports, it’s important to take stock of past learnings and challenges.
Bringing fans to the stadium
La Liga restarted on June 11 with “virtual fans” on display. (Screengrab)
Since its resumption, La Liga matches are being telecasted with what is an augmented ‘audience’; a static texture that resembles a crowd. It only works on the broadcast camera though, so any jarring angle switch tears away the illusion and the simulated blanket of ‘fans’. The virtual spectators aren’t rendered individually either, which means it’s just swathes of colour (interestingly, the FIFA videogames do a better job at creating fake fans too).
Gudjon Gudjonsson, CEO of OZ Sports, presents a more sophisticated option. The company’s ‘OZ Connected Stadium’ system is already installed at several sports venues, helping out with services such as VAR and player tracking. They have now developed an AR-enabled option that lets fans appear in their favourite seats.
“We wanted to bring the lower leagues premium World Cup production experiences. When we started our journey imagining a youth tournament looking like the finals of the World Cup,” says Gudjonsson. “The only thing that was missing where the 60.000 spectators…”
OZ Sports thus helps conjure the spectators. Fans sign up on a website hosted by the league or the rightsholder, and according to branding guidelines, get themselves “more realistic human avatars or fantasy-themed characters.” Broadcasters then can choose to overlay said avatars on empty seats, or instead opt to digitise the entire stadium into a sci-fi, fantasy setting.
Gudjonsson — who at 25 founded his first company which let chatroom users create digital avatars back in 1997 — calls OZ Sports an “Avengers Assemble” collection of experts. Chief among them is the award-winning RVX Studios, who have provided visual effects to films such as the Harry Potter series, Sherlock Holmes and Gravity.
“The industry is learning how to think differently as there is this urgent need now to cut technology infrastructure costs, restore budgets, and at the same time, increase the output quality of production,” he says.
Such is the tech’s need for lower latency and stable connection, that AR’s rise has largely coincided with the advent of 5G. Last March, Korean operator SK Telecom put up a fire-breathing AR dragon above a baseball stadium, visible on the giant screen as well as all 5G-enabled devices. Dallas Cowboys have been driving the tech in American football, and fans can catch glimpses of 36-foot high holograms of players on their devices.
Sky has long plotted to use AR in football broadcast to fend off rivals Amazon and BT Sport. The plan gained momentum after the successful coverage of The Open last summer, where 120 cameras captured selected golfers’ swing. As a result, analysts could walk around a lifesize 3D render of Rory McIlroy.
A definite application of the technology can be cricket, where microscopic scrutiny of bowling actions and batting stances remains a key part of the reportage. But unless the technology is able to pluck out players from a live game and recreate them in the studio, it remains a gimmick for controlled environments.
Bringing stadium to the fans
Basketball, by virtue of its smaller scale, has long been the petri dish for broadcasting experiments. The NBA has already been telecasting games in VR to headset owners and, more importantly, treating it like an actual production rather than an add-on. You can look around freely, follow the breakaways as they happen, change the angle to watch a dunk, and turn your head to look at the Hollywood A-lister sitting next to you. Scores too are built into the system, you look far left or far right to bring up points, rebounds etc, or look up to check the scoreboard and time.
Danny Keens, Vice President of Content at NextVR — the company that distributed the VR content for NBA games, once told this paper: “We are at the beginning of the beginning with virtual reality. We like to say VR today is where mobile was at the brick cell phone stage. The rate of change in the technology is growing exponentially.”
That was three years ago. Last month, Apple confirmed its purchase of NextVR, becoming the latest heavyweight to move into the field. Sony has been a player through its PlayStation subsidiary and the tech is expected to get a serious upgrade when the latest edition, PS5, launches this winter. Google and Intel too have been heavily involved.
Sankar Jayaram — who, alongside wife Uma, has been working on the technology for close to three decades and helped Intel form the relationship with NBA, NFL and the Olympics — says “VR has already been in play for many years now in different forms.”
“From live streaming of immersive VR media to sharing a large screen with multiple people remotely through avatars. Covid 19 could be a shot in the arm for these remote but immersive experiences and as sports start back up across the world with empty stadia,” says Sankar.
But the headsets are as bulky as they are expensive. Top-tier products range from Rs30,000 to Rs1.5lakh. The models that require a smartphone to be lodged inside are relatively cheaper but come with their own issues of pixelation, compatibility, and drained phone battery. Mass consumption of VR remains a distant dream, and Sankar also points out the pressing concern of adaptability.
“The process of engaging with leagues, teams, stadium operators, broadcasters, talent, user experience is very critical. The sports business community is used to a certain way of operating. You cannot delay a game because it is taking five extra minutes to adjust a camera. And I am not even getting into the business and rights aspects that need to be navigated.”
What the fans want
Then there’s the biggest variable — the consumers. The teething troubles of dealing with cutouts and Zoom walls have been accentuated by prankster fans.Australia’s National Rugby League had to rework their ‘Fan in the Stands’ initiative after cardboard cutouts of prolific serial killer Harold Shipman and Dominic Cummings — the much-criticised advisor to British Prime Minister Boris Johnson — made their way into the seats. To make matters much worse, broadcasters Fox Sports then ran a ‘comedy’ sketch superimposing Adolf Hitler in the stands.
DDT, a Japanese pro-wrestling company, was one of the first to use Zoom to have videoconferencing fans attend their shows. President Sanshiro Takagi, however, wasn’t left impressed by the experience.
“I feel like I now know the frustrations and struggles of the inventor Thomas Edison,” Takagi told The Indian Express. “It was a fruitful experiment, but the issues we were able to identify were numerous, such as the microphones picking up and streaming the sounds of a fan slurping noodles! At the end of the day, no matter what we do, there is an element of dependence on fans being mindful of manners for the sake of everyone else.”
Danish Superliga club AGF Aarhus was a lot more successful, with giant video walls showcasing supporters connected via Zoom. But even their wary moderators were twice left red-faced.
Danish Superliga returned to action on May 28 in front of thousands of live fans. (Source: Reuters)
“I’m proud to say we only had two people who were kicked out,” Aarhus’ media chief Soren Carlsen says. “They were showing some parts of the body you’re not supposed to show.”
Hitler, flashers, and loud slurpers are obvious detractions. But giving officials the power to pick and choose each fan inside the stadium could rob football of its heterogeneity. There are concerns that through systems such as an augmented overhaul, officials would go for an idealised projection of the game, censoring ‘objectionable’ chants and protests. Case in point the pre-lockdown protests from Bundesliga purists against rich owners and sponsors. No fans, no dissent.
Gudjonsson, a supporter of the Iceland football team, knows too well the importance of fan expression.
“The rivalry and chants are what create the storytelling. Their passion is needed on the field too,” assures Gudjonsson. “We will make the fan experience more exciting. Until now the fans in the stadium and those watching on TV are two very separate groups. We want to connect those two to create one huge audience that drives the overall matchday experience.”
Carlsen agrees that AR/VR could work as add-ons, but could never replace the real thing.
“This technology can be used to when we as citizens are not allowed to be together. But I don’t think we’ll see a future where people are staying home, putting on some VR goggles or whatever, and then you will watch the game,” says Carlsen. “A fan is supposed to be at the stadium.”
EXPLAINED | As sports events resume, a look at the new protocols in place
Even looking past the philosophical dilemmas, each tech solution comes with its own challenges of accessibility and feasibility. And nobody’s yet even mentioned the athletes, who would still be playing in front of empty seats.
Executives have been mulling innovations long before the pandemic. Last decade saw Sky alone start and stop novelties such as 3D broadcasts (self-explanatory), player cams (a camera remained on one player for 15 minutes), and fan zone — where they picked two opposing supporters off the streets to commentate on the game.
But unlike those novelties, AR/VR and other innovations are borne out of necessity. Will they succeed?
Gudjonsson invokes British naturalist Charles Darwin to answer.
“Darwin did frame this so perfectly when he said, ‘it is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change,'” says Gudjonsson.
Darwin, however, also said: “Often a cold shudder has run through me, and I have asked myself whether I may have not devoted myself to a fantasy.”
