AI Combat Manikin
April 23, 2025
Over the past 8 months, my team embarked on a journey to create an interactive manikin for combat training. The idea was sparked by the realization that traditional training dummies simply can’t replicate the unpredictability and responsiveness of a real opponent. We wanted to bridge that gap by building a manikin that could not only react to trainee movements but also challenge them in dynamic, high-pressure scenarios. The full overview of the project can be viewed here.
As a member of the software team, my primary focus was:
- System architecture & integration: Designed a modular pipeline integrating a PyQt frontend GUI with a Python backend stack with Intel RealSense depth cameras.
- Baton tracking: Trained a FasterRCNN model to detect and track the baton’s path, ensuring the manikin could dodge or counter in real time.
- Graphical User Interface: Developed a no-code PyQt interface with scenario configuration and real-time feedback panels.
- Inference Optimization: Implemented multithreading to parallelize inference pipelines from the two depth cameras, reducing end-to-end latency to < 50ms.
The most rewarding moment came when the manikin flawlessly dodged attacks in real-time—a testament to our tightly integrated system of Kalman filters, pose estimation, and decision algorithms. Seeing trainers effortlessly adjust dodge probability (10-90%) and combat modes via the GUI’s intuitive sliders and presets validated months spent refining both technical robustness and user experience, proving that complex AI can coexist with simplicity.
Building this project deepened my expertise in end-to-end system architecture, code optimization, and real-time data processing. It also strengthened my skills in cross-team collaboration, rapid prototyping, and translating user requirements into intuitive interfaces and robust backend solutions.
If you're interested in the technical details or want to see the code, feel free to reach out. I'm always happy to connect with fellow builders and enthusiasts!