Many Timbeter users run into a situation where they need to measure a pile, but it does not fit into a single picture. For this Timbeter provides the panorama functionality.
For the sake of the correctness, it should be mentioned that the name ‘panorama’ is not technically the right word for what Timbeter does. While ‘panorama’ is a commonly understood term, it is a method of capturing photos, where the user rotates on a single point. In Timbeter’s panorama functionality, the user moves along the pile for each picture but does not rotate the camera at all. The precise term for this is a ‘planar mosaic.’
To create a panorama image, the algorithm must know the relative position of each source image. To achieve this, an alignment is established between each consecutive pair of images. The alignment process consists of the following steps: finding key points, matching key points and finding an alignment.
Finding key points
Much the same way a human would do it, Timbeter tries to find distinctive features that appear in both of the images being aligned. For a human, this might be an unusually large or oddly shaped log. However, for a computer it is more efficient to find a large number of features that are very small, often only a few pixels across.
The features that Timbeter searches for can be thought of as corners: in a small region, there is a change of color when looking at it from two different directions. While flat regions and edges appear in much larger quantities and are very similar, corners tend to be fairly distinctive. Though the exact number varies, to create a precise alignment, over a thousand key points are found for one image in this step.
Matching key points
Timbeter attempts matching each key point from one image with every key point form the other one, as long as they are at a similar height. When comparing two key points, a pixel region of equal size is taken around both, and a correlation metric is calculated. While distortions are sure to happen because of the change in perspective between the two photos, it is less pronounced for small regions of the image, so the process is fairly robust even when not taking the distortion into account.
Around 500 matches are usually found for each image pair.
Finding a homography
Because many key point matches from the previous step are going to be false matches, a robust method for finding the alignment must be used. To do this Timbeter uses an algorithm called RANSAC (random sample consensus). The algorithm iteratively chooses a random subset of key point matches, calculates an alignment from them and then checks how many key point matches fit this alignment. Each alignment thus created is a hypothesis.
However, often there is more than one hypothesis that seems to fit the data. Commonly, there is a hypothesis which is an alignment created using the key points that are on the log pile, but there is also a hypothesis generated by the key points that are on the background. For time efficiency, logs are not detected during panorama creation, so it is not known which key points are on the pile and which ones are not. In this situation, Timbeter chooses the hypothesis with the more significant movement, as objects closer to the camera change position more when the user moves around.
The number of matches used to create the final alignment can vary a lot, but it is usually in the order of 100 matches.
Stitching the panorama together
In the final step, all photos are put into a single large image. While most panorama applications blend neighboring images seamlessly, at Timbeter, we have chosen to make the stitching line sharp. This is because seamless blending often requires distorting the source images which may affect the pile volume. Though such images would look prettier to the eye, they would be less precise for measuring.
So, this is how the panorama functionality in Timbeter works internally. Moreover, with our latest update, we have to give it a substantial boost in accuracy. So, if you have not used panorama yet, why not give it a go?