Skip to content

Raytrix Acquistion DLL

This README explains firstly what it can do, how to build the DLL, and how to launch the Raytrix Streaming DLL that was developped during the HoviTron project.

Note that the Raytrix Streaming DLL has only been compiled and tested with a Windows Operating System. Thus, this README will focus only on Windows OS.

Utility

The Raytrix acquisition DLL uses the Raytrix SDK to retrieve the reconstructed color image, i.e. total focus image, and the estimated depth. The processing converts the raw image into the required RGBD data and also applies the metric calibration from the Raytrix procedure to the data. The processing parameters were set to provide a high processing speed while not highly reducing the depth map quality. Once the data acquired, the DLL does some post-processing operations on them. First, data conversions are done to prepare the depth information ready to be consumed by the synthesis module. In second, to mitigate the noise in the depth estimated, a temporal coherency algorithm is applied to reduce spike artifacts. With a single NVIDIA RTX3090 GPU, the acquisition of the two depth maps, with the temporal coherency procedure, achieves 15 frames per second.

Installation

List of software/librairies needed:

  1. CUDA

  2. RxLive or Raytrix SDK (5.+ or 6.+)

    • Link: https://raytrix.de/downloads/

    • Documentation:

      • Raytrix SDK 5.0
      • ⚠ Raytrix SDK 6.0 ⚠ can be found in C:\Users\Public\Documents\Raytrix\Light Field SDK 6.0\docs\SDK

  3. Windows OS (10 or 11)

  4. Visual Studio (2019/2022) with the C++ desktop development

If you have any problem during the installation of one of SDKs/librairies, please refer to the documentation of the developpers.

Preliminaries

Since the CMakeLists.txt has been written to compile a .dll not an executable, please first refer to the RVSVulkan README to enable the Raytrix Streaming DLL compilation option in the CMakeSettings.json.

Build / Compilation

The compilation of the DLL must be done as it is done for the Kinect Streaming DLL (see AcqKiRT README). The compilation is done via Visual Studio through a CMakeLists.txt where we specify to the C++ compiler how to compile it.

  1. Open the project in Visual Studio
  2. Open the CMakeSettings.json of the project

  3. Specify the fields:

    1. RX_5_PATH or RX_6_PATH with the absolute path to the Raytrix SDK
      • Note that, it should be installed at "C:/Users/Public/Documents/Raytrix/Light Field SDK {RX_SDK_VERSION}"
    2. RX_BINARIES_PATH with the absolute path to the Raytrix SDK binaries (.dll files)
      • Note that it should be installed at "C:/Program Files/Raytrix/Bin"
    3. CUDA_TOOLKIT_ROOT_DIR with the absolute path to the CUDA API directory

  4. You can also change the version of the Raytrix SDK used: a. If VERSION_5_RX_SDK is true, it will use the fifth version b. If false, it will use the sixth version

  5. Finally, press simultaneously CTLR+S and click on Build
    • It is possible that Visual Studio saves only one field when saving

Launch the dll

In this section, two ways to launch the Raytrix DLL will be explained.

Through a terminal

> set HVT_RXDLL_PARAMS_PATH=${PATH_TO_A_JSON}
> RVSVulkan.exe ../RaytrixStreamer/RaytrixStreamer.dll [--glfw|--openxr|...]
Where the variable ${PATH_TO_A_JSON} is the absolute path to a JSON file where the parameters for the Raytrix DLL are specified.

Note that RVSVulkan.exe can be launched with some parameters, see RVSVulkan README.

With Visual Studio

  1. Open the project in Visual Studio
  2. Compile it
  3. Right-click on the HoviTron CMakeLists.txt then click on Add a debug configuration
  4. After the field name, add the field args, and as arguments add
    1. "../RaytrixStreamer/RaytrixStreamer.dll"
    2. Other parameters for RVSVulkan, see RVSVulkan README
  5. Add a field env, and specify the environment variable "HVT_RXDLL_PARAMS_PATH" with the absolute path to the JSON parameter file
  6. Next to the green arrow (in Visual Studio), choose the debug configuration that you created, and click on it to launch it.

To make it clearer, here is an example of a launch.vs.json file where one debug configuration is written. The configuration name is RaytrixStreamerDLL. The configuratione asks Visual Studio to launch RVSVulkan.exe with the Raytrix Streamer DLL with a GLFW window. 

{
  "version": "0.2.1",
  "defaults": {},
  "configurations": [
    {
      "type": "default",
      "project": "CMakeLists.txt",
      "projectTarget": "",
      "name": "RaytrixStreamerDLL",
      "args": [
        "--glfw",
        "../RaytrixStreamer/RaytrixStreamer.dll"
      ],
      "env": {
        "HVT_RXDLL_PARAMS_PATH": "C:\\Users\\Admin\\Desktop\\Hovitron\\HoviTronProject\\RVSVulkan\\RaytrixStreamer\\JSONs\\calibration.json"
      }
    }
  ]
}
The paramaters of the DLL will be set according to what is inside the JSON file "C:\Users\Admin\Desktop\Hovitron\HoviTronProject\RVSVulkan\RaytrixStreamer\JSONs\calibration.json" given as an environment variable.

JSON parameter file

As you could understand, the Raytrix Streaming DLL needs a JSON parameter file where all the needed parameters will be specified. An example is available at JSON example.

In this section, all the parameters are explained so you can write your own JSON parameter file.

Variables Type Description
numCams int Number of cameras
idCams Set of variables (int) ID of all cameras that must be used. The variable name is the index of the camera, while its value is an int which denotes its ID.
forceUseOfSpecificRBGResolution boolean True to force the use of the specified resolution, false to let the RaytrixSDK choose.
width int Width of the total focus image
height int height of the total focus image
forceUseOfSpecificDepthResolution boolean True to force the use of the specified resolution, false to let the RaytrixSDK choose.
width_depth int Width of the Depth3D image
height_depth int height of the Depth3D image
RxComputeParametersFile string Path to the CUDA compute parameters preset (extension: ".rxset")
fromRawFile boolean Work with a(some) sequence file(s) instead of connected cameras
pathRawFile Set of variables (string) All paths to the sequence files that must be used. The variable name is the index of the camera, while its value is a string which denotes its path. (Extension: ".rays")
specificCalibration boolean To Work with a specific metric calibration file generated by RxLive for each camera.
GUIDsForCalibration Set of variables (string) All GUIDs of the calibration files. The variable name is the index of the camera, while its value is a string which denotes its GUID.
exposureCams Set of variables (double) Exposure value of all cameras that must be used. The variable name is the index of the camera, while its value is a double which denotes its exposure.
ExtrinsicParamsFile String Path to the extrinsinc camera parameters. (Position+Rotation between cameras).

Extrinsic parameter file

Another JSON file can be sent to the Raytrix Streamer DLL, as the field ExtrinsicParamsFile denotes. It allows you to enter manually the extrinsic camera parameter of your cameras.

The camera 0 is always considered as the reference one. The position of the others cameras is referred from the reference camera.

An example of the camera parameters file can be found in this directory, see extrinsic parameter file.