Skip to content

RVS software

The code present in this repository is an extension of the Reference View Synthesis (RVS)[1-3] software (available as RVS) to plenoptic cameras, non-Lambertian objects and XSlit cameras. The inner working of RVS is described with more detail in inTechOpen open article [3].

Acknowledgments

Sarah Fachada is a Research Fellow of the Fonds de la Recherche Scientifique - FNRS, Belgium.

Figure1

Description

This software computes novel views.

Usage

Commandline parameters

General parameters

Cmd Description
json file path
--noopengl using cpu
--analyzer analyse

Camera Json parameters

Name Value Description
Name string camera name
Position float 3 position (front,left,up)
Rotation float 3 rotation (yaw,pitch,roll)
Depthmap int has a depth
Depth_range float 2 min and max depth
Multi_depth_range float 2 min and max value in multidepth (non-Lambertian) (optional, default: Depth_range)
DisplacementMethod string Depth or Polynomial (optional, default: depth)
Resolution int 2 resolution (pixel)
Projection string perspective or equirectangular or plenoptic or XSlit
Focal float 2 focal (pixel)
Principle_point float 2 principle point (pixel)
Hor_range float 2 horizontal range (degree) (equirectangular projection)
Ver_range float 2 vertical range (degree) (equirectangular projection)
plenopticDiameter float micro-image diameter (pixel) (plenoptic projection)
plenopticCenter float 2 center of the central micro-image (pixel) (plenoptic projection)
plenopticS float MLA to sensor distance (mm) (plenoptic projection)
plenopticD float main lens to MLA distance (mm) (plenoptic projection)
plenopticF float main lens focal length (mm) (plenoptic projection)
plenopticConfig int micro-image layout (horizontal 0, vertical 1) (plenoptic projection)
Slit1 float 6 Plucker line's coordinate of first slit (Xslit projection)
Slit2 float 6 Plucker line's coordinate of second slit (Xslit projection)
BitDepthColor int color bit depth
BitDepthDepth int depth map bit depth
ColorSpace string YUV420 or YUV400
DepthColorSpace string YUV420 or YUV400

Example of camera json file:

json:table
 {
     "Version": "3.0",
     "Content_name": "dataset_name",
     "BoundingBox_center": [0, 0, 0],
     "Fps": 30,
     "Frames_number": number_of_frames_in_yuv_file,
     "Informative": {[...]},
     "lengthsInMeters": true,
     "sourceCameraNames": [ "input_camera1_name", [...] ],
  "cameras": [
    {
      "Name": "perspective_camera1_name",
      "Position": [x, y, z],
      "Rotation": [ yaw, pitch, roll ],
      "Depthmap": 1,
      "Background": 0,
      "Depth_range": [near, far],
      "Resolution": [w, h],
      "Projection": "Perspective",
          "Focal": [ (fx), (fy)],
          "Principle_point": [ (ppx), (ppy)],
      "BitDepthColor": 8,
      "BitDepthDepth": 8,
      "ColorSpace": "YUV420",
      "DepthColorSpace": "YUV420"
    },
    {
      "Name": "equirectangular_camera2_name",
      [...],
      "Projection": "Equirectangular",
          "Hor_range": [ (theta_min), (theta_max)],
          "Ver_range": [ (phi_min), (phi_max)],
      [...]
    },
    {
      "Name": "plenoptic_camera3_name",
      [...],
      "Projection": "Plenoptic",
          "Focal": [fx, fy],
          "Principle_point": [ppx, ppy],
          "plenopticDiameter": microimage_diameter,
          "plenopticCenter": [microimage_center_x,microimage_center_y],      
          "plenopticPixelSize": pixsize_in_mm,
          "plenopticS": distance_MLA-sensor,
          "plenopticD": distance_mainlens-MLA,
          "plenopticF": mainlens_focal,
          "plenopticConfig": 0-horizontal-layout_1-vertical-layout,
      [...]
    },
    {
      "Name": "xslit_camera4_name",
      [...],
      "Projection": "XSlit",
          "Slit1": [l01, l02, l03, l12, l13, l23],
          "Slit2": [m01, m02, m03, m12, m13, m23],
      [...]
    },
    {
      [...]
    }
  ]
}

View Synthesis Json parameters

Name Value Description
Version string version number
InputCameraParameterFile string filepath to input cameras json
VirtualCameraParameterFile string filepath to input cameras json
VirtualPoseTraceName string filepath to posetraces (optional)
InputCameraNames string list list of input cameras
VirtualCameraNames string list list of output cameras
ViewImageNames string list filepaths to input images
DepthMapNames string list filepaths to input depth
OutputFiles string list filepaths to output images
StartFrame int first frame (starts at 0)
NumberOfFrames int number of frames in the input
NumberOfOutputFrames int number of frame in the output (optional, default: NumberOfFrames)
Precision float precision level
ColorSpace string RGB or YUV working colorspace
ViewSynthesisMethod string Triangles
BlendingMethod string Simple or Multispectral
BlendingFactor float factor in the blending

References

[1] Bonatto, D., Fachada, S., & Lafruit, G. (2020). RaViS: Real-time accelerated View Synthesizer for immersive video 6DoF VR. Electronic Imaging, 2020(13), 382-1. art00012

[2] Bonatto, D., Fachada, S., Rogge, S., Munteanu, A., & Lafruit, G. (2021). Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays. IEEE access, 9, 146868-146887. 10.1109/ACCESS.2021.3123529

[3] Fachada, S., Bonatto, D., Teratani, M., & Lafruit, G. (2022). View Synthesis Tool for VR Immersive Video. Intechopen