Author Topic: AHD demosaic beta update - Much better sharpness for in-camera h264 saving  (Read 30567 times)


patrickrebstock

  • Full Member
  • ***
  • Posts: 114
  • playing camera
    • View Profile
    • patrickrebstock.com
Re: AHD demosaic beta update - Much better sharpness for in-camera h264 saving
« Reply #31 on: September 15, 2018, 12:34:10 AM »
take a look at the image issues on high contrast moving water with the new firmware, pretty funky stuff going on, can this be worked on?
https://youtu.be/ywW78_fZfL0
focus still off its biased too close

Nikon1

  • Hero Member
  • *****
  • Posts: 983
    • View Profile
Re: AHD demosaic beta update - Much better sharpness for in-camera h264 saving
« Reply #32 on: September 16, 2018, 03:57:03 AM »
take a look at the image issues on high contrast moving water with the new firmware, pretty funky stuff going on, can this be worked on?
https://youtu.be/ywW78_fZfL0
focus still off its biased too close
Well, that is just how Bayer-Sensor work.
They have a pattern of Red Green and Blue Color Filters over the individual Pixels of the Sensor, to be able to produce a color image from an sensor which actually just captuers brightness information for each individual pixel.
The Problem with that is, that the software (which then later tries to make a color-Image from that Only-Brightness-Information and the information of the Pattern,) Can get easily confused by single very bright pixels.
So if an very bright reflection on the water appears and it only takes up one single pixel in the final image, how would the software know if it was white or the color of the individual Pixel, for example Red?
I kind of allready explained this exact problem in an other thread " i am professional colorist..."
For most of todays Video-Cameras this is not a problem. Because they have sensors with much higher Resoultion than what the final Video would be. For example a 24Megapixel Sensor which captures FullHD or 4K video. Final Video still is just 2MP or 8MP, and that is way less than the original 24MP from the sensor. So there is room for heavy interpolation to get an more accurate result for the real brightness and color-value of each pixel.
For highspeed-Cameras like the chronos that is mostly not the case. Since those cameras allready push the limit of what is technicaly possible, there is no room for downscaling and interpolation like this.
So capturing stuff like this, shown in your test, is tricky in any case with that camera.
.
Your best bet would propably be to try using RAW and an different debayer algorithm or Debayering software. some of them could give better results than this for that special scene. Otherwise you will most likely end up with lots of  those colorfull pixels all over your footage. Not sure, tho if you could get rid of all of them with this Method, but i guess, worth a try anyways.
Only other thing i could think of, is to add an physical anti-Alliasing filter in front of the Chronos-Sensor. That will solve the problem about the colorfull pixels mostly but will make your result more unsharp.
.
For more Information about all this just look up information about debayering / demosaicing and Bayer-Sensors in general, should answer most of your questions.

tesla500

  • Krontech
  • Full Member
  • *****
  • Posts: 218
    • View Profile
    • krontech.ca
Re: AHD demosaic beta update - Much better sharpness for in-camera h264 saving
« Reply #33 on: September 17, 2018, 06:47:51 PM »
The issue with colored pixels that patrickrebstock is seeing is due (mostly) to a known bug in the new demosaic, see the post after this one for more info.

Nikon1 is still correct, this is an intrinsic issue with bayer pattern sensors, but it's not anywhere near as bad as the video Patrick posted makes it seem. Right now, for difficult scenes like this, the best you can do is save raw and use a very good demosaic such as Rawtherapee's Amaze algorithm, which handles these single pixels better, although our demosaic is pretty good compared to most (once the bug is worked out).

The "proper" solution requires a hardware change; an OLPF (optical low pass filter) placed in front of the image sensor, where the IR filter is. This deliberately blurs the image slightly before it hits the image sensor, smearing the sharp single pixel objects over adjacent pixels, which allows the normal demosaic interpolation to get the true color of the drops instead of the false color you're seeing. After demosaic, a digital sharpening filter undoes the blurring effect of the OLPF. I'm currently looking into having these made, they'd eventually make it into new production cameras, and also be available as a user installable upgrade for existing owners.
« Last Edit: September 17, 2018, 06:53:59 PM by tesla500 »

Loial

  • Krontech
  • Newbie
  • *****
  • Posts: 24
    • View Profile
Re: AHD demosaic beta update - Much better sharpness for in-camera h264 saving
« Reply #34 on: September 17, 2018, 06:48:33 PM »
take a look at the image issues on high contrast moving water with the new firmware, pretty funky stuff going on, can this be worked on?
https://youtu.be/ywW78_fZfL0
focus still off its biased too close

Oh, ya, sorry - I'm currently working on it. It was a problem I noticed a few weeks ago right after releasing the new demosaic update but I had no way to actually reproduce it without sunlight against water and even then it was just only barely noticeable (the above video is no longer barely noticeable and is a great representation of the issue).

Today I just found a way to recreate it in the office and am working on where the issue is within the demosaic - use a non-white light and saturate the sensor.

The new demosaic for initial testing showed better single-pixel white lite handling than most others I tested; rather proud of that. The problem shown above is a situation where some of the math within the FPGA is corrupted, probably due to a rollover of a variable used in the demosaic.

Thanks very much for the example and as soon as I have a fix we'll release an update with it.


Nikon1

  • Hero Member
  • *****
  • Posts: 983
    • View Profile
Re: AHD demosaic beta update - Much better sharpness for in-camera h264 saving
« Reply #35 on: September 18, 2018, 12:08:12 AM »
The "proper" solution requires a hardware change; an OLPF (optical low pass filter) placed in front of the image sensor, where the IR filter is. This deliberately blurs the image slightly before it hits the image sensor, smearing the sharp single pixel objects over adjacent pixels, which allows the normal demosaic interpolation to get the true color of the drops instead of the false color you're seeing. After demosaic, a digital sharpening filter undoes the blurring effect of the OLPF. I'm currently looking into having these made, they'd eventually make it into new production cameras, and also be available as a user installable upgrade for existing owners.
Yes, that was what i wanted to say, but since im not an Native English-Speaker i dont know the exact term for it. I thought it was Called anti-Alliasing-Filter.
So sorry for the confusion here, i was referring to an Low pass filter.

Loial

  • Krontech
  • Newbie
  • *****
  • Posts: 24
    • View Profile
Re: AHD demosaic beta update - Much better sharpness for in-camera h264 saving
« Reply #36 on: September 18, 2018, 12:29:55 AM »
Good news - I was able to reproduce the issue in simulation and easily tracked down the variables that were causing issues. I've fixed them but the changes need to make their way back into the main code. We should be able to make an update that removes this artifact in the next couple days.

JamesB

  • Jr. Member
  • **
  • Posts: 68
    • View Profile
Re: AHD demosaic beta update - Much better sharpness for in-camera h264 saving
« Reply #37 on: September 25, 2018, 07:18:58 PM »
Good that a fix was found, this also applies for any bright lit metallic object with specular highlights!