MIT License

Copyright (c) 2024 Epic Games, Inc. 

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

****
 
This folder contains Python scripts that enable users to download and process images from NASA's Mars rover.

Source images are courtesy of NASA/JPL-Caltech. Please refer to the JPL Image Use Policy for information on their use: https://www.jpl.nasa.gov/jpl-image-use-policy

This is what you have to do to create a model from Mars images:

1. Install Python

2. Install the following modules (using PIP): BeautifulSoup, requests, piexif, numpy, planetaryimage and scikit-image

3. Edit your local RealityCapture sensor database found here C:\ProgramData\Capturing Reality\RealityCapture\sensorsdb.xml

   and add the line:

   <camera model="Malin Space Science Systems MastcamZ" ccdWidth="9"/>

4. Run download_images.py script

5. Run prepare_geometry_texture_images.py script

6. Run edit_image_metadata.py script

7.Open RealityCapture and in a console view type (look for Command Sequence in the app help for more information): 

-selectImage g/zl0.*1100LM/  -editInputSelection "inpLensGroup=0" -editInputSelection "inpCalibrationGroup=0"  -selectImage g/zr0.*1100LM/  -editInputSelection "inpLensGroup=1" -editInputSelection "inpCalibrationGroup=1" -selectImage g/zl0.*340LM/  -editInputSelection "inpLensGroup=2" -editInputSelection "inpCalibrationGroup=2"  -selectImage g/zr0.*340LM/  -editInputSelection "inpLensGroup=3" -editInputSelection "inpCalibrationGroup=3" -selectImage g/zl0.*790LM/  -editInputSelection "inpLensGroup=4" -editInputSelection "inpCalibrationGroup=4"  -selectImage g/zr0.*790LM/  -editInputSelection "inpLensGroup=5" -editInputSelection "inpCalibrationGroup=5" -selectImage g/zl0.*630LM/  -editInputSelection "inpLensGroup=6" -editInputSelection "inpCalibrationGroup=6"  -selectImage g/zr0.*630LM/  -editInputSelection "inpLensGroup=7" -editInputSelection "inpCalibrationGroup=7" -selectImage g/zl0.*480LM/  -editInputSelection "inpLensGroup=8" -editInputSelection "inpCalibrationGroup=8"  -selectImage g/zr0.*480LM/  -editInputSelection "inpLensGroup=9" -editInputSelection "inpCalibrationGroup=9" -selectImage g/zl0.*260LM/  -editInputSelection "inpLensGroup=10" -editInputSelection "inpCalibrationGroup=10"  -selectImage g/zr0.*260LM/  -editInputSelection "inpLensGroup=11" -editInputSelection "inpCalibrationGroup=11" 

8. Click the "+" button on the right and name the sequence "Group Mastcams by Calibration and Lens"

9. Run the sequence ((you can find it in the Sequence part of the WORKFLOW tab)

10. Align the images (ALIGNMENT tab)

11. Set reconstruction region (look for Reconstruction Region in the app help)

12. Create a model in Normal Detail (MESH MODEL tab)

13. Go to the Texture Settings and change the "Large triangle removal threshold value" to 200 (Mesh Color & Texture part of the MESH MODEL tab)

14. Simplify the model (Mesh Model part of the SCENE 3D / TOOL tab)

15.  (Mesh Color & Texture part of the MESH MODEL tab)


DESCRIPTION OF PYTHON SCRIPTS
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
download_images.py

Downloads images from planetary data website in RAW format (.img) with their corresponding XML files

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
prepare_geometry_texture_images.py

This file will convert the RAW images to NUMPY arrays. It will then use the scikit-image module to apply a CLAHE filter for local contrast, save those images as 8-bit JPG images in a folder for the geometry, and save the images with only gamma correction in a folder for texture

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
edit_image_metadata.py

Add information about the make and model of the camera and the focal length to the EXIF data of the JPG images in the _geometry folder.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Please note:

The process above is experimental, and is by no means the definitive workflow. It's worth trying out different things and settings like changing the distortion model for alignment or choosing other pre-processing methods (even the texture images will align, so the CLAHE is not necessarily needed, but it is more of a tip) .

The original images are 11 bit, but they are stored as 16 bit RAW images. You might want to make the texture folder using 16-bit images rather than converting them to 8-bit by adapting the prepare_geometry_texture_images.py script . The geometry folder is fine in 8 bit as RealityCapture will not use 16-bit images for geometry.

There are a lot of possible ways you might be able to improve these scripts and this workflow. You are more than welcome to share your solutions in the forum post of this tutorial on the Epic Developer Community site.
