How Fox Sports will use virtual fans created in Unreal Engine to fill empty stadiums in MLB broadcasts
Baseball is back. But the MLB will look a little different in the COVID-19 world, with empty stadiums full of silent cardboard cutouts instead of thousands of screaming fans in an effort to avoid spreading the virus. That is unless you’re watching on Fox Sports. Starting this weekend, the network will not only be piping in fake fan audio, but it’ll also be filling the stands with digitally rendered fans for its MLB broadcasts, starting with this Saturday’s Yankees-Nationals game.
All 30 MLB teams will be relying on fake fan noise in the actual stadiums (sourced from Sony’s MLB: The Show games), along with the usual in-stadium announcers, walk-up music, and other aspects of pomp and circumstance that accompany a Major League Baseball match. But Fox Sports is the only broadcaster that’s announced plans for virtual fans.
“We believe the crowd and seeing people in seats is part of a broadcast, is part of broadcasting high level sports in the major leagues. So we wanted to come up with a solution for that,” explains Brad Zager, executive producer and head of production and operations at Fox Sports.
“We’re not looking to fool anybody, it’s still about the game. But shot-to-shot, when you’re watching a broadcast, it’s not more noticeable that you’re watching what is normally a broadcast with fans in just an empty stadium and having it feel weird — we want to give people a sense of normalcy,” Zager says. “And we felt like going down this path and trying to use a virtual crowd will hopefully make it so that blends in, and you can focus on the game more because you’re not thinking about the emptiness of a Major League Baseball stadium during the game.”
The effect is a combination of technologies you may have seen before. The augmented reality software used to insert the crowds is called Pixotope, which has worked on AR graphics for things like the Super Bowl and The Weather Channel’s terrifying storm warning demonstrations.
It works by leveraging graphics (created by creative agency Silver Spoon Animation) built in Epic’s Unreal Engine. Unreal Engine is used here for the same reason it’s popular for creating video games or for crafting virtual on-set backgrounds for shows like The Mandalorian. Unlike most film graphics, which have to be rendered in post-production after the fact, Unreal can render in real time, making it far more suited for live television.
Lastly, SportsMedia Technology (SMT) — the same company that handles most of the sports-centric computer overlays that you’re familiar with, like the yellow first-down line in football, on-screen clock and scoreboard, and more — handles the camera tracking to insert those graphics on the live camera feeds. Four cameras — high home, centerfield, high first, and high third cameras — will show the virtual fans.
Fox Sports producers will be able to control things like how full the virtual “crowds” are for a given game, what weather fans are dressed for, and what percentage of the crowd will be home fans versus away, although the company is still figuring out how it’ll make some of those decisions. “We’re figuring this out right now, and we’ll continue to evolve that process as we go through this weekend and from game-to-game, week-to-week,” says Zager.
The Fox Sports broadcasts will also integrate the feigned fan audio being played in the stadium into the background of microphone clips from the players (like during pitches) and other in-game audio, but it’ll augment that live audio on its own end, too, similar to how it’s been doing for MLS games.
And while the interactions between the virtual crowds and the audio will be basic for now (like basic cheers, boos, and the wave), the company is hoping to better integrate the two as it gets more practice over the season. “It’s something that we hope becomes even more collaborative and more on the same page as we do this more and more, being able to combine the audio and the crowd together.”
For now, the virtual fans will only be broadcast on MLB games on Fox, but it’s possible that the digitally generated fans could appear elsewhere, too. Zager says that the company is working with all of its sports sponsors, and that “as we get closer to the fall for football and other sports, if we feel like it’s going in the right direction yet we will launch it and roll it out on other sports,” too.
Baseball is back. But the MLB will look a little different in the COVID-19 world, with empty stadiums full of silent cardboard cutouts instead of thousands of screaming fans in an effort to avoid spreading the virus. That is unless you’re watching on Fox Sports. Starting this weekend, the network…
