TTNet Demo

Details

• OpenTTGames Dataset (Full-HD videos, 120 fps)
• Multi-Task : Ball detection, Semantic segmentation, Events spotting
  • Ball size : about 15 pixels on average (color : Magenta)
  • Segmentation : humans(G), table(B) and scoreboard(R) classes with channel-wise encoding
  • Events : "bounce" and "over net"
• Input : downscaled video frames (320px,128px) and 9 frame sequences (less than 0.1s in real time)
• Annotation : Event - center frame, Ball position and seg mask - last frame, respectively
• Inference capability : more than 166 fps on a machine with a single NVIDIA RTX 2080Ti GPU

A demo (60 fps) by a trained model (loss condition : Unbalanced) using following repository: https://github.com/maudzung/TTNet-Real-time-Analysis-System-for-Table-Tennis-Pytorch

test_1.mp4	test_2.mp4

This model is almost robust for time inversion input.

Input videos from YouTube (60 fps)

• Ball detection : moderate, Segmentation & Event : terrible
• Clip video scene from 1:18 (link)
• ref. download video : pytube.ipynb

Reference

• R Voeikov, et al. "TTNet: Real-time temporal and spatial video analysis of table tennis" (CVPR 2020), YouTube
• A Kendall, et al. "Multi-task learning using uncertainty to weigh losses for scene geometry and semantics" (CVPR 2018), Supplementary