Flir has come out with a new thermal imaging camera technology that can deliver imagery at a higher resolution than a sensor is natively able to produce. For instance, the 1024 x 768 px sensor in their new T1K thermal imaging camera can produce images with up to 3.1 MP resolution.
You get 4X the imaging resolution!
Advertisement
Not only that, Flir says you would get greater temperature measurement accuracy as well.
Right now this UltraMax feature is only available on the Flir T1K thermal imaging camera, but hopefully that’ll change and we’ll see this tech on more affordable IR thermal imagers.
Update: UltraMax is available on Flir’s other T-series thermal imaging cameras as well. In a comment, Adabhael said that they were able to have their older T-series imager updated with this feature, for a fee.
It seems to work in a very similar manner to Keysight Technology’s TrueIR thermal imagers. As you hold an UltraMax-capable thermal imager in your hand, and aim it at the target area, the camera will record a stack of 16 images.
No matter how steady your hands are, there will always be small motions and offsets. When you’re holding something, especially a weighty thermal imaging camera, your vibrations will travel to and into the object.
As your hand moves ever so slightly, the image shifts ever so slightly across the thermal imaging sensor. With each slight motion or shift, the imaging sensor records a very slightly different image. An UltraMax thermal imaging camera will record all of these slightly different images – 16 of them.
According to the UltraMax product feature video, all of this data is saved in the same file. When you upload the image stack to Flir Tools, it will enhance your image using the additional information.
Thus, split each pixel into 4 sub-pixels, fill those pixels with unique thermal measurement information, and voila – you have greater resolution than what your thermal imaging camera would otherwise be capable of.
Here’s a forward-looking thought. Because the image enhancement is done through Flir Tools, and not in the camera, there is the potential that we will see this on less expensive Flir thermal imaging cameras down the road.
Keysight already offers this on their $3500 U5855A thermal imaging camera, which has a 160 x 120 thermal detector resolution and 320 x 240 simulated detector resolution.
Flir’s MSX thermal imaging enhancement is an in-camera feature, and so maybe the UltraMax resolution enhancement might eventually be built in-camera as well.
The key to being able to capture those multiple images in-camera, and in a very short time. Flir Tools likely takes the 16 frames and runs through some quick calculations to create an ideal simulated image with max “up to” resolution. I’m thinking that it maybe takes all of the captured thermal images to determine a best-guess approximation.
In the case of the Flir T1K thermal imaging camera, the 1024 x 768 pixel sensor is used to create a 2056 x 1536 pixel image.
Okay, so I’m thinking it busts each pixel into 4 sub-pixels, with the original data as a starting point. Then, it uses the 16 images to adjust each subpixel with reasonably certain measurements. That makes more sense to me than if the software used some of the frames and discarded some or most of the others.
Regardless of how it works, all that matters is that it does work. What we need next is for someone to reverse-engineer this technology.
You could buy a Flir Lepton 80 x 60 px module for as low as ~$175 and a breakout board for as low as ~$40. Those are the prices at Digikey. Hobbyist experimenters were able to create a functional thermal imaging camera using a Raspberry Pi, a single-board Linux computer that’s available for $30 to $40, depending on the version.
Being able to create a 80 x 60 thermal imaging camera that could deliver 160 x 120 px imagery for $250 or so would be fantastic. It would be even better if the Flir E4 could continue to be user-enhanced such that its unlocked 320 x 240 px sensor could deliver 640 x 480 px imagery. You don’t want to know how much a 640 x 480 px thermal imaging camera costs, although it’s quite a bit less than the $40K the new Flir T1K with UltraMax is selling for.
Flir’s UltraMax tech sounds pretty good on paper, and I’m hoping to see it in action, either in my own E4, or a community-driven project of some kind. Hopefully Flir won’t limit UltraMax technology to their highest-priced cameras.
If you think my understanding of the new Flir UltraMax thermal resolution enhancement feature is wrong, please let me know and explain your views in a comment!
Oh, and watch this promo video. It looks like we finally have an “enhance” button similar to what you see in TV shows and movies. Someone says “enhance” and all of a sudden they can pick out details to resolve a blurry image or video frame.
Doresoom says
They may be using inertial measurement sensors on the camera to sharpen the image. In that case you’ll be tied to the more expensive hardware and won’t be able to reproduce the results with a cheaper camera that doesn’t have the sensors.
More on the subject:
http://research.microsoft.com/en-us/um/people/larryz/imu_deblurring.pdf
Stuart says
That research paper is focused on image stabilization. That’s not what UltraMax is. UltraMax doesn’t increase sharpness or improve focus, it increases resolution by enhancing details.
I don’t think that the UltraMax feature works on visual imagery – only the thermal imagery.
A true thermal image is in grayscale, with each pixel having an intensity value that is proportional to the IR light coming from the subject. UltraMax uses several slightly different images in a stack to combine additional sub-pixel intensities for each pixel.
Take a cardboard paper towel or toilet paper tube and look through it. Or just loop your fingers together into a similar shape. Now look through the hole. Move your hand a little, up, down, or sideways. You see more of the picture this way.
Now imagine that your hand, or cardboard tube, can only take one temperature measurement for the area you are viewing. When your hand or viewing tube moves, you’re increasing the area that a single pixel can take measurements for.
Combine everything automagically, and you could create a reasonably simulated image.
The visual image in a thermal imaging camera could very easily be much higher in resolution than the thermal image.
I don’t think UltraMax does anything with the visual image data, only the thermal data.
There have been several DIYer projects where users took “single-pixel” IR thermometers and used servos to scan them across a surface to create a thermal image. My understanding is that UltraMax works somewhat similarly, using each pixel in a sensor array to collect data for several sub-pixel areas. The data is then combined off-camera.
So… instead of a single pixel measuring the temperature for an area of say 0.04° x 0.04° in a single image, multiple images can be combined such that a single pixel could measure the temperature for area of 0.02° x 0.02°, resulting in a 4X resolution enhancement.
That’s at least how I’m understanding this. Image capture is done in-camera, but enhancement is done off-camera. There might be sensor-related limitations as to why lesser thermal imaging cameras cannot do this, but I’m more inclined to think that potential limitations might be more dependent on supporting hardware, which handles things like image capture and saving.
Doresoom says
I wasn’t saying they were using it for optical images – the same methodology could apply to thermal as well.
The article doesn’t cover the exact methodology I had in mind, just a similar technique using intertial sensors and an imaging device. I was thinking more like what Pat David links to below, but with a non-blind algorithm to align the images. (The one described in Pat’s article is most likely using cross correlation or something similar, which mostly eliminates the need for an inertial sensor. But hey, more data is always better, right?)
We may have been thinking along the same lines already, and I just did a poor job of communicating my thoughts. I don’t really see a difference in your example of the DIY single pixel thermal imager and my original thoughts on how this technology might work. Surely they’re using an encoder which is recording the position of each pixel? Which is along the same lines as using an intertial sensor to figure out where the camera is pointed.
Stuart says
I’m sorry if I misunderstood. I know you weren’t referring to optical images, but I still don’t think we’d see much benefit from stabilized optics in a thermal imaging camera.
Thermal imaging cameras typically have awful visual image quality, at least all of those I’ve seen. But the thermal images are sharp.
Well, at least as sharp as they can be. Flir’s MSX tech works wonders for improving thermal images.
The detection circle for a thermal sensor pixel is pretty big, even in higher resolution cameras. Blurring due to handholding is going to be miniscule if not unnoticeable. That’s why I can’t really see the need for image stabilization in a thermal imaging camera.
With UltraVision, 16 separate images are used for enhancement. If blurring a thermal image is easily possible – it’s not something I ever remember seeing – then image stabilization might help improve results.
It might also come in handy when using a thermal imaging camera on a tripod – which is opposite of what you want with a regular photography camera. In such a case, stabilization might come in handy to shift the image by predictable minute amounts. I guess then it wouldn’t be a stabilizer, but a destabilizer.
Adabhael says
UltraMax is already available (often standard) on other cameras in the T-Series as well (not just the awesome but expensive T1K). Even better, UltraMax can be retrofit to cameras that did not have it, as long as they have the right hardware and firmware. We have a year-old T420bx at the office that was recently upgraded to UltraMax (it had to go back to the factory and cost about $700 with shipping). I just got it back last week, so I have not done any real work with it, but goofing around, it is pretty amazing.
Stuart says
Ah, you’re right!
I read that it will be available on other T-series thermal imaging cameras, but forgot all about that when working on this post. Thanks for the correction!
Pat David says
“This isn’t about averaging or interpolating” … “It’s about filling in the gaps” 😀 😀
You can do this right now fairly simply _if_ you’re able to grab a burst of images:
https://pixls.us/blog/2015/09/softness-and-superresolution/#a-question-of-scaling
Best part? It’s all done with free/open source tools…
Martin says
Will this be implemented on the IPhone 5 skin FLIR one as well?