During my MSc at Edinburgh, two teammates and I picked a problem that turned out more under-explored than we expected: detecting physical contact between players in American football video.
Most published research here falls into two buckets. One is head-impact detection using wearable sensors. The other is general player and action detection. Very little deals with the specific mess of American football, where players are in close physical contact constantly. That makes it genuinely hard to tell where one contact event starts and another ends.
We used the Kaggle NFL Player Contact Detection dataset, which has 720 videos. We only used the Sideline and Endzone views, since the third view wasn't timestamped. The dataset also gave us player tracking data, contact labels sampled every 0.1 seconds, and helmet bounding boxes.
We pulled frames once a second starting five seconds into each video. We expanded the helmet bounding boxes to 256x256 crops per player per frame. Then we sorted the results into contact and no-contact folders.
To manage a fairly large class imbalance, we sampled around 10,000 no-contact frames and 8,000 contact frames. Then we cropped those down to 150x150 to keep the feature count manageable and reduce overfitting risk before training anything.
Why SVM Wasn't the Right Baseline
We started with SVM as a baseline, following a paper that used it for head impact detection in American football. It didn't go well.
With default sklearn parameters, it reached 49.51% train accuracy and 49.19% test accuracy. Barely better than a coin flip. Tuning it (200 iterations with a gamma of 0.1) actually made things slightly worse: 48.03% train, 47% test.
SVM was also noticeably slow on this data, which limited how much tuning we could realistically explore. The combination of weak accuracy and poor scalability told us something clearly: a linear-margin classifier over raw feature vectors just wasn't a good fit for this kind of high-dimensional image data at this scale. No amount of tuning was going to change that.
Augmentation Mattered More Than Architecture
We moved to three deep learning models: ResNet50, ResNet152, and EfficientNetB0. ResNet50 without augmentation, trained for 100 epochs, overfit visibly and topped out at 72.17% validation accuracy.
Rather than assume a deeper network would fix that, we tested the same hyperparameters (a learning rate of 0.01 and dropout of 0.5) with image augmentation added. That alone brought validation accuracy up to 74.89%. It also visibly narrowed the gap between training and validation curves.
What surprised us: ResNet152 without augmentation only reached 73.27%. Barely ahead of the non-augmented ResNet50. More depth didn't buy much on its own.
EfficientNetB0, also without augmentation, gave us the best non-augmented result at 74.49%. We think that's because of its compound scaling approach and squeeze-and-excitation blocks. Both use capacity more deliberately than plain ResNet does.
Given our compute and time constraints, we didn't get to run augmentation on ResNet152 or EfficientNet. That's a real gap in the comparison. But the pattern across what we did run was consistent: augmentation closed more of the overfitting problem than switching architectures did.
- SVM (default params): 49.19% test accuracy
- SVM (tuned, gamma 0.1): 47% test accuracy
- ResNet50, no augmentation: 72.17% best validation accuracy
- ResNet50, with augmentation: 74.89% best validation accuracy
- ResNet152, no augmentation: 73.27% best validation accuracy
- EfficientNetB0, no augmentation: 74.49% best validation accuracy
What We Learned Trying to Crop With Pretrained Detectors
Our original plan was simpler than what we ended up doing. Use pretrained object detection models to crop players directly. Then classify contact on those crops. That plan broke early.
In this sport, players are in close physical proximity most of the time. When that happened, the pretrained detectors either merged two players into a single detection or missed one of them outright. Retraining these detectors on our own data wasn't realistic given our compute budget.
So for the primary classification task we fell back to cropping directly from the helmet bounding boxes already provided in the dataset's CSV files.
That failure turned into our second research question. Do state-of-the-art detectors actually perform better on cropped images than on the full frame, given how much they struggled with crowded scenes?
We compared three COCO-pretrained models (Faster R-CNN R50, Mask2Former, and YOLOX X) across 15 full images and 47 cropped versions of the same scenes. We measured precision, recall, and F1.
- Faster R-CNN R50: F1 0.86 on full images, 0.91 on cropped images
- Mask2Former: F1 0.91 on full images, 0.90 on cropped images
- YOLOX X: F1 0.92 on full images, 0.93 on cropped images
The result we didn't expect: cropping doesn't uniformly help. Faster R-CNN, a two-stage detector, improved meaningfully once cropped. Mask2Former actually did marginally worse cropped than uncropped.
YOLOX X stayed close to 0.92 either way. That's exactly the property you want in a detector built for real, messy game footage, where you can't always guarantee a clean crop. That consistency is what made YOLOX X look like the strongest candidate for a real-time system, more than either single F1 number did.
A Model That Didn't Transfer
We also tried a pretrained CNN-LSTM violence detection model directly on our raw match footage. Mostly out of curiosity: would an existing "physical altercation" detector generalize to sports contact?
It produced a lot of false positives. The paper's reported results were solid on its original domain. That domain wasn't ours, and the model hadn't been fine-tuned for football.
Useful reminder: a model's headline numbers belong to the dataset they were measured on. Not automatically to whatever you point the model at next.
What This Project Actually Taught Us
Three things held up across both halves of this project. Classical ML like SVM can be a reasonable baseline, but it wasn't built for high-dimensional image data at this scale. No amount of parameter tuning changed that.
Data augmentation fixed more of our overfitting problem than moving to a deeper or more sophisticated architecture did. Worth remembering before reaching for a bigger model as the first fix.
And object detectors don't respond to cropping uniformly. A claim like "cropping improves detection" needs to be tested per architecture on your actual data. Not assumed from how one model happened to behave.
Learn More
The full project is open source: American Football Contact Detection on GitHub.