Contra-Sformer

Contra-Sformer is a video-based framework for automated skill assessment in robotic surgery, comparing test videos against reference videos representing optimal surgical execution. It combines ResNet-18 spatial feature extraction with temporal convolutional networks and an action-aware transformer to generate contrastive features that capture performance differences. The system achieves Spearman correlation scores of 0.65-0.89 on the JIGSAWS dataset across knot tying, needle passing, and suturing tasks. It identifies suboptimal execution and errors without explicit error annotations, enabling targeted feedback for surgical trainees during real-time assessment.

Set up the conda environment using the provided env.yml file. Download the ResNet-18 extracted features and cross-validation splits from the provided links. Run the experiment.py script specifying the surgical task (knot_tying, needle_passing, or suturing) and validation scheme (loso, louo, or 4fold).