[Paper Review] Domain-Adaptive 2D Human Pose Estimation via Dual Teachers in Extremely Low-Light Conditions

Difference with other methods

• This paper focuses on how to estimate 2D human poses without low-light ground truths. (only use well-lit ground-truth data)
  • Recent studies require the use of paired well-lit and low-light images with ground truths. → Low light images have the inherent challenges associated with annotation on low-light images.

 

Two stage

1. Pre-training

Model is trained on well-lit data with low-light augmentations. 

Perform supervised training with labeled well-lit data only. → both teachers recognize human poses in well-lit images and their corresponding low-light augmentations(ELLA(Extreme Low-Light Augmentation)).

 

2. Dual-teacher knowledge acquisition

Model uses a dual-teacher framework to utilize the unlabeled low-light data and generate more reliable pseudo labels.

• main teacher (=center-based teacher)
  • labeling for visible cases
  • main teacher is based on DKER
• complementary teacher
  • produce the pseudo labels for the missed persons of the main teacher.
  • HigherHRNet-style design
• student
  • get a person-specific low-light augmentation to outperform the teachers
  • use PDA (Person-specific Degradation Augmentation) which is selectively applied to images with pseudo labels generated by the teachers.

3. Limitations & Solutions

• Limitations
  • low-light image enhancement and domain adaptation
• Solution
  • Introduce an innovative domain-adaptive dual-teacher framework that facilitates knowledge transfer from well-lit to low-light domains.
  • A keypoint-based bottom-up human pose estimation model

 

Dataset

• ExLPose
  • using for training
  • use labeled well-lit data only in the first stage
  • use low-light data in the second stage and generate pseudo-labels
  • 2065 paris of well-lit and low-light images and the ground-truth
    • ExLPose-OCN
      • real low-light dataset
      • using for test prediction
    • ExLPose-test
      • man-made low-light images
      • LL-A, LL-N, LL-H, LL-E

Paper

Domain-Adaptive 2D Human Pose Estimation via Dual Teachers in Extremely Low-Light Conditions