Skip to content

Scripts

These Scripts are utilities helping the data processing (camera parameters, image format, ...) and image evaluation (objective metrics computation). The present README explains the utilities of such scripts.

These scripts are implemented in Python.

Acknowledgments

Sarah Fachada is a Research Fellow of the Fonds de la Recherche Scientifique - FNRS, Belgium.

Figure1

Camera parameters conversion

These scripts are used to convert the camera parameters between MPEG's json format (https://mpeg-miv.org/) to Colmap's camera format (https://colmap.github.io/). It allows to use the camera parameters computed by Colmap with RVS.

Dataformat

Json format gathers the (undistorted) camera parameters as follows: 

json:table
 {
     "Version": "3.0",
     "Content_name": "dataset_name",
     "BoundingBox_center": [0, 0, 0],
     "Fps": 30,
     "Frames_number": number_of_frames_in_yuv_file,
     "Informative": {
       [...]
     },
     "lengthsInMeters": true,
     "sourceCameraNames": [
       "input_camera1_name",
       "input_camera2_name",
       [...]
     ],
  "cameras": [
    {
      "Name": "input_camera1_name",
      "Position": [x, y, z],
      "Rotation": [ yaw, pitch, roll ],
      "Depthmap": 1,
      "Background": 0,
      "Depth_range": [near, far],
      "Resolution": [w, h],
      "Projection": "Perspective",
      "Focal": [ (fx), (fy)],
      "Principle_point": [ (ppx), (ppy)],
      "BitDepthColor": 8,
      "BitDepthDepth": 8,
      "ColorSpace": "YUV420",
      "DepthColorSpace": "YUV420"
    },
    {
      [...]
    }
  ]
}

While Colmap's image camera format is composed of cameras.txt, images.txt and points3d.txt - cameras.txt:

# Camera list with one line of data per camera:
#   CAMERA_ID, MODEL, WIDTH, HEIGHT, FX, [FY], PPX, PPY, DISTORTION[]
  • images.txt:
# Image list with two lines of data per image:
#   IMAGE_ID, QW, QX, QY, QZ, TX, TY, TZ, CAMERA_ID, NAME
#   POINTS2D[] as (X, Y, POINT3D_ID)
  • points3d.txt:
# 3D point list with one line of data per point:
#   POINT3D_ID, X, Y, Z, R, G, B, ERROR, TRACK[] as (IMAGE_ID, POINT2D_IDX)

Conversion

  • Json to Colmap:
python json_to_colmap.py -c {cameras.json} -oc {cameras.txt} -oi {images.txt}
  • Colmap to Json:
python colmap_to_json.py -c {cameras.txt} -i {images.txt} [-n {CONTENT_NAME}] -o {cameras.json}`

Image format conversion

These scripts simply call ffmpeg to resize and convert images.

  • Resize all the .png images in a folder:
python resize_all.py -f {input folder} -o {output folder} -w {new width}  -he {new height}
  • Convert from YUV (yuv420p) format to PNG:
python yuv_to_png.py  -f {input folder} -o {output folder} -w {new width}  -he {new height}
  • Convert from PNG format to YUV (yuv420p):
python png_to_yuv.py  -f {input folder} -o {output folder} -w {new width}  -he {new height}

Depth maps format conversion

To convert a depth map from Colmap binary format to RVS-compatible EXR format: 

python dense_to_exr.py  -d  {depth_maps_folder}

To convert from depth map EXR format to MPEG-I’s YUV disparity (input format for Test Model of MPEG Immersive Video (TMIV)). 1. python depth_to_disp.py -f {exr_depth_folder} -o {exr_disp_folder} -n {near} -m {far} 2. python exr_to_yuv.py -f {exr_disp_folder} -o {yuv_disp_folder} The disparity is defined according MPEG’s disparity format: \(depth = \frac{near*far}{near+\frac{disparity}{2^{bitdepth}}*(far-near)}\)

To convert a disparity map (in pixels) to a EXR depth map for plenoptic cameras:

python d {disp.png} -s {sensor_mla_distance} -f {focal} -D {mainlens_mla_distance} -Di {microimage_diameter} -dmax {maxdisparity} -o {output.png}

To create a polynomial map to render non-Lambertian scene:

python create_maps.py -i {indices_of_the_input_cameras.txt} -c {cameras.json} -m {folder_with_matched_images} -o {outputfolder} -w {img_width} -he {img_height} -n {number_of_neighbour_images_to_create_the_maps}

Evaluation Metrics

These scripts compute the evaluation metrics, comparing a ground truth folder to a set of folders containing synthesized images. The images in the folder must be sorted alphabetically. For IV-PSNR, the images are expected to be in YUV (yuv420) format. For all the other metrics, they are expected to be in PNG format.

RGB PSNR

This script requires opencv-python. 

python psnr.py -f {input folder} -o {output folder} -w {width}  -he {height} [-m {mask folder}]

A folder with masks can be used as input to compute the PSNR on a region of interest.

YUV PSNR & IV-PSNR

This script calls the implementation of IV-PSNR that can be found at: https://gitlab.com/mpeg-i-visual/ivpsnr

python ivpsnr.py -f {input folder} -o {output folder} -w {width}  -he {height} -d {bitdepth} -o {folder to dump the computated ivpsnr}
python ivpsnr_parse.py -f {folder with dumped ivpsnr} -o {output folder} -n {number of compared methods}

MS-SSIM

This script requires opencv-python. 

python ssim.py -f {input folder} -o {output folder} -w {width}  -he {height}

LPIPS

This script requires opencv-python and the LPIPS trained implementation: https://pypi.org/project/lpips/. 

python lpips.py -f {input folder} -o {output folder} -w {width}  -he {height}