I have drone based RGB time sequence images and time-lapse video of TIR images acquired at the same time. For info, the phantom 3 has its own rgb cam and we tied flir vue pro just next to it on the same drone and started acquiring images from both cameras. We acquired once in the morning 10-11am and on the same day once at 10-11pm in the night. The area under study contained set of black painted aluminium sheets and white painted aluminium sheets placed randomly in a playground and the scene also covered a swimming pool too. So we had tree tops, grass, sand, cement surface, roof tops, steel materials, water and black & white coloured aluminium sheets here and there. I have done an experiment in lab and found an equation to extract temperature from DN values of flir image and similarly reflectance values from DN values of rgb phantom sensor. Have also tried thermal inertia after manual registration of the two sensor images. Now, may I know how to go about estimating thermal inertia for such a large area covered by both sensors with a lot of sequence images and with different field of view and pixel size. I want to know about automatic image registration and image mosaicking techniques for both rgb and thermal images..
HUGIN to the rescue!
Given that both cameras are in the same position relative to each other - and this a repeatable from flight to flight, you can use Hugin to composite the two images. The "best" way to do this is let Hugin create a batch for a pair of images. This will include the lens distortions needed to register both images. Once you have these you can execute nona against the images using those paramaters.
OK, so NOW you have pairs of images IR and VIS which have been distorted such that they are registered. Perhaps you need to make both images equal in size, pixel wise - this can be done via Image Magick as a batch job. It will happily take images 1 thu N and generate new images of the size you want, resampled using the interpolation you specify.
Now you take your registered / same sized image and build a panorama using Hugin. So each image will have the same ( entirely fake ) lens profile - which will tell Hugin how to distort the input images and project them on the world sphere. These metrics would work for either the VIS or IR image set - because both image sets are registered. So then the fun begins...
Note, up to this point, you have a fixed operation being applied to your images - so easily scriptable / repeatable provided you maintain same camera FoV from run to run. Now you have to generate a panorama from your images.
You will optimize the panorama for "position" which basically tells Hugin to assume the images were shot from different locations as they were. Now the problem you have is that if the images are shot from non-uniform altitude this changes the actual FoV of the image. So ideally you will be able to set the altitude of your drone to maintain a constant linear FoV. If not you will also need to let it optimize FoV as well. This also reduces parallax shifts between the IR and VIS images so that the initial registration is more applicable for all the images.
Feature detection in Hugin is configurable so it will handle more or less rotation of the image. So again, if you aim your cameras straight down, and all of the images are shot roughly in the same orientation compass wise, it will have an easier time. If its is a bunch of random orientations, it will struggle more. So again, garbage in / garbage out.
So in the end you will have a panorama job which can be used to generate a rectilinear projection for both IR and VIS and these two finished panorama images should be already registered because the same distortions are applied to both image sets.
Start small - take 4 or 6 image pairs with about 30% overlap and try. The real issue is the position optimization - as position changes apparent FoV and can confuse the optimizer. Be sure to select which things you are optimizing - as you can manage each variable independently or in groups. Once you figure out a recipie, you can repeat that and recycle your time invested in working it out.
This is probably as good as one can do with FOSS stuff, IMO. Though I am eager to see if others have a different tool chain that can be used.
As for fully automating this? That may take IBM Watson ( or so their silly TV ad claims ) or some commercial product.
Thank you for your suggestion. I was looking for programming to solve and will start exploring hugin though. Will let you know soon!! Meanwhile I am also curious to learn other suggestions!
- Badges
- Science topic
- Similar topics
- Engineering
- Electrical Engineering
- Signal Processing
- Image Processing