Multi-Modal Sensor Fusion
Private / NDAMulti-Modal QC for Plastic Injection Moulding
A team project fusing process and sensor modalities with a cross-modal Transformer and domain-adversarial training (DANN) on the ProBayes dataset, against a highly imbalanced defect distribution.
Contact for a walkthrough ↗Team Member
Role
Private
Access
01 — Problem
Plastic injection moulding defects are rare relative to good parts, and defect signals are spread across both process parameters and sensor readings — no single modality tells the whole story.
02 — Context
A team project working with the ProBayes dataset, where the defect distribution is heavily imbalanced toward normal production.
03 — System Built
A cross-modal Transformer that fuses process parameters with sensor modalities, trained with domain-adversarial training (DANN) to generalize across production conditions.
04 — Engineering
Team and dataset details beyond this are private — contact for a walkthrough of the modeling approach.
05 — Decisions
Used DANN specifically to stop the model from overfitting to conditions specific to one production run, given how imbalanced the defect classes were.
06 — Result
Delivered a working cross-modal classifier for the team's evaluation on the ProBayes dataset; client-facing metrics beyond that are private.
07 — What I Learned
Sensor fusion on industrial data lives or dies on how well you handle class imbalance before the fusion step, not just at the classifier head.
08 — Next Iteration
Extending the fusion approach to additional sensor types beyond the ones evaluated.