Vuze 3D Stitcher
Create a 3D 360 VR image from 8 fisheye images.
Project maintained by e-regular-games Hosted on GitHub Pages — Theme by mattgraham
Using Feature Detection
Date: July 23, 2022
Script: features.py
Usage:
../notes/splice_360.sh -d -c config_bash_splice_360.dat
../src/features.py -c config_features.dat
Image feature identification works best with rectilinear images. The Equirec2Perspec code was used to convert to rectilinear for feature identification. The code was enhanced to provide the reverse mapping “Perspec2Equirec” and allow for determining the original equirectangular point for each feature.
The features script uses image features as determined by the SIFT algoritm and brute force macthing between images to determine the constants used for image transformations. All image transforms are within the spherical coordinate system. The most beneficial is the scaling of $\theta$. A second transform was added to linearly shift phi with respect to $\theta$.
$\theta_f = k \theta_i$
$\phi_f = \phi_i + k \theta_i$
These transforms were used because the operations are easy to perform both forward and reverse. The reverse transform is needed to determine the pixel location within the lens when given a pixel location in the final image. The table below shows the resulting constants and the error between the expected value and the calculated value using the constant.
| Lens | Theta Scale | $\overline{\theta_f - \theta_i}$ | $\sigma_{\theta_f-\theta_i}$ | Phi Rate | $\overline{\phi_f - \phi_i}$ | $\sigma_{\phi_f-\phi_i}$ |
|---|---|---|---|---|---|---|
| 1 | 0.8974 | 0.0099 | 0.0193 | 0.0015 | -0.2655 | 0.5106 |
| 2 | 0.9007 | -0.0242 | 0.0237 | 0.0027 | -0.1943 | 0.5920 |
| 3 | 0.8760 | -0.0223 | 0.0268 | -0.0100 | -0.3039 | 0.4947 |
| 4 | 0.8970 | -0.0098 | 0.0218 | 0.0015 | -0.2582 | 0.5568 |
| 5 | 0.8897 | 0.0308 | 0.0248 | 0.0079 | -0.2470 | 0.5288 |
| 6 | 0.9084 | -0.0193 | 0.0235 | 0.0033 | -0.3314 | 0.5195 |
| 7 | 0.9048 | -0.0123 | 0.0173 | 0.0053 | -0.2227 | 0.5425 |
| 8 | 0.9054 | -0.0044 | 0.0192 | 0.0109 | -0.2575 | 0.5707 |
Once the images were adjusted to get the feature points as close as possible, the stitch seams were computed. Instead of choosing a single constant $\theta$ for the seam a line of points starting from phi = 0, to phi = math.pi was used. All points within one standard deviation of the median $\theta$ were used for the seam. The seam is sorted by $\phi$ with duplicate values of phi removed. Points within the seam are assumed to be connected by a straight line. As you can see the stitch lines have issues when $\theta$ changes greatly with very little change in $\phi$. The significant changes causes a zig-zag pattern in the final image. There are still 8 stitch lines in the image.
References
Equirec2Perspec - A script for converting an equirectangular image to rectilinear.