Model Compression Part 6.

Getting FOMM repo up and ready.
type: practicallevel: easy

Getting FOMM repo up and ready

Goal: To get FOMM repo up and ready for VoxCeleb dataset. Requirements: Python >= 3.8.

Setting up

  1. Clone repo at https://github.com/AliaksandrSiarohin/first-order-model
  2. Create a folder data/vox-png with 2 subfolders train and test, put training videos in the trainand testing in the test.
  3. Edit the config file config/vox-256.yaml according to your training/reconstruction, in dataset_params specify the root dir the root_dir: data/vox-png. Also adjust the number of epoch in train_params.
  4. Save model checkpoints vox-cpk.pth.tar from Google Drive to a folder called checkpoints.
  5. Install packages according to the following requirements.txt
    imageio==2.22.0
    matplotlib==3.6.0
    numpy==1.23.3
    pandas==1.5.0
    python-dateutil==2.8.2
    pytz==2022.2.1
    PyYAML==6.0
    scikit-image==0.19.3
    scikit-learn==1.1.2
    scipy==1.9.1
    torch==1.7.1
    torchvision==0.8.2
    tqdm==4.64.1
    imageio--fmpeg==0.4.7

Changes in loading dataset

If VoxCeleb data filenames take the format of id100001|xxxx|xxxx|xxxx.mp4, in frames_dataset.py line 75, change

train_videos = {os.path.basename(video}.split('#')[0] for video in
os.listdir(os.path.join(root_dir, 'train'))}

to

train_videos = {os.path.basename(video}.split('|')[0] for video in
os.listdir(os.path.join(root_dir, 'train'))}

Changes in code

  1. In augmentation.py line 13, change from skimage import pad to from numpy import pad.
  2. In logger.py:
    • Line 7: Change from skimage.draw import circle to from skimage.draw import disk
    • Line 111: Change rr, cc = circle(kp[1], kp[0], self.kp_size, shape=image.shape[:2]) to rr,cc = disk(center=(kp[1], kp[0]), radius=self.kp_size, shape=image.shape[:2]).
  3. In modules.generator.py line 93, change F.sigmoid to torch.sigmoid.

Run code

At this point, model should be ready to run reconstruction or further training (without any customisation).

  • To do reconstruction:
CUDA_VISIBLE_DEVICES=0 python run.py --config config/vox-256.yaml \
--mode reconstruction --checkpoint checkpoints/vox-cpk.pth.tar
  • To train from scratch:
CUDA_VISIBLE_DEVICES=0 python run.py --config config/vox-256.yaml \
--device_ids 0 --mode train

Put the above commands in an .sh file to change easier.

Possible errors to come across during training

Currently, only 1 error is found: CUDA Out of memory ---> Solution: reduce batch size.