What’s new: On February 8th, Eileen Gu, an 18-year-old skier representing China at the ongoing Beijing Winter Olympics, made sports headlines with her gold medal performance in the women’s freestyle skiing big air competition.

Pulling off a massive 1620 with a perfect landing brought Gu a combined score of 188.25, an impressive feat. This is a move in which skiers spin 4½ times while rotating twice off-axis while 20-some feet in the air. Skiers can reach speeds of up to 16 meters per second, or 35 miles per hour. Even the swiftest-eyed spectator would have trouble catching precisely how Gu pulled off such complex winning moves. 

Baidu’s AI and three-dimensional (3D) simulation technology, “3D+AI”, are here to help, capturing and analyzing athletes’ motions and creating a computer-simulated 3D model of skiers. “3D+AI” is revolutionizing how TV and online viewers watch high-speed snow sports like freestyle skiing in an immersive and intuitive manner. It is also emerging as an essential training aid to pinpoint parameters that impact performance, helping skiers prepare training routines.

“3D+AI”, developed by Baidu AI Cloud, is now being applied in Chinese TV coverage of the Beijing Winter Olympics. It is also being used for training in sports ranging from diving to table tennis by China’s national team.

How it works: What gives this technology an edge over traditional video replays is that Baidu’s “3D+AI” technology can recreate a 3D model for each skier in action and fuse different models in the same picture, allowing different athletes’ performances to be compared side-by-side. In addition, “3D+AI” can also overlay the original video replay or simulated 3D models with additional information – extracted motion data such as sliding speed, flying height, landing distance, and rotation angles – to indicate how fast a skier is moving, what angles they rotate, and how far they can land.

By breaking down an athlete’s movement, “3D+AI” accurately captures every detail of the athlete’s actions in all directions and regenerates a set of time-series freeze motions in a simulated 3D space. You can think of it as a 3D version of the traditional strobe camera. This groundbreaking technology allows viewers to experience an immersive 360-degree view of complex and rapid sports action.

Baidu AI Cloud also uses virtual reality to reconstruct a 1:1 3D model of the Shougang Ski Jumping Platform, a key competition venue for the Beijing Winter Olympics. Through a 3D scene walkthrough, online viewers can watch the jumping platform via multiple perspectives - long-range, panoramic, and close-up - or switch between daylight and night mode. 

Other use cases: The application of “3D+AI” goes beyond intelligent sports broadcasting, extending to sports coaching. AI data analysis of athlete training has proved to evaluate performances and distinguish incorrect movements effectively.

For instance, in April 2021, Baidu AI Cloud announced an exclusive partnership with the Chinese National Diving Team, using the “3D+AI” technology to help professional divers perform their form and get in top shape for the Tokyo 2020 Olympics.

The act of diving takes only two seconds, such as the “5255B” dive, which includes a back two-and-a-half somersault and one-and-a-half twists with pike. Baidu’s “3D+AI” technology can capture the act of diving using high-quality and high-speed sensors, followed by skeleton motion reconstruction to make a quantitative assessment. Then, it scores based on divers’ performance, i.e., its difficulty and corresponding score, and customizes training techniques. As a result, the “3D+AI” system has improved the diving team’s training efficiency by 20%.

Another prime example of “3D+AI” is Baidu’s intelligent table tennis system, which has collected over 500GB of games training data and can identify eight types of actions such as serving, pulling, and short swing. The system can recognize each play with an accuracy of over 97% and different actions with over 80%, significantly improving judging decision accuracy.

As the integration of AI technology and the world of sports continues to grow more prevalent, more exciting applications are sure to emerge soon!

You can watch the breakdown of Gu's movement here: https://youtu.be/k6Xb9-1I-68