Author Topic: Maze artifact with 1.4  (Read 529 times)

1022mm

  • Newbie
  • *
  • Posts: 27
    • View Profile
Maze artifact with 1.4
« on: June 21, 2019, 03:43:47 PM »
Shooting in RAW, 1280x720, 1502fps today my video has this "maze" artifact across the entire image on every frame as well as a huge hue shift.   Normal black calibrations done before the shot.   Camera was on for about 20 minutes prior to the capture.  Anyone else see this before and know the cause?  I have been shooting with the same settings all year and this is the first time I've encountered it.

I was able to correct the hue shift in Photoshop's RAW editor, but the maze pattern remains.

First attachment below is a crop of the frame at 100%, second one is enlarged 4x.
« Last Edit: June 21, 2019, 04:59:25 PM by 1022mm »

1022mm

  • Newbie
  • *
  • Posts: 27
    • View Profile
Re: Maze artifact with 1.4
« Reply #1 on: June 21, 2019, 04:53:37 PM »
After investigating this further, here's some more information.

The first four frames of the video are fine - no maze pattern and color is correct.   On the fifth frame, something got corrupted which does two things to the rest of the frames thereafter:  there is a tint shift and the maze artifact.   

Three more attachments.   1.) The first frame, 2.) the fifth frame where the corruption happens, and 3.) the sixth frame which displays the shifted tint as well as the maze artifact.  The rest of the frames thereafter have the same problem.

In the Photoshop RAW editor, the tint value on all frames from the Chronos defaults to +139.   Setting the tint to 0 on these bad frames puts it very close to the correct color, but does not fix the maze pattern.

I have tried running the raw to dng process twice, on two different computers, with the same result.

Is there some sort of color/luminance profile that was corrupted for the bad frames?   Is there a way to salvage the rest of this shot and others if it happens in the future?
« Last Edit: June 21, 2019, 06:47:58 PM by 1022mm »

Loial

  • Krontech
  • Newbie
  • *****
  • Posts: 24
    • View Profile
Re: Maze artifact with 1.4
« Reply #2 on: June 21, 2019, 07:27:31 PM »
It looks like the data being saved in raw is off by one pixel or line. This was caused by some pixels or lines being lost while saving the video which in turn makes the demosaic in the DNG viewer/editor software fail.

What version of camApp and, if it shows it in the util menu, what FPGA version?

As for recovering, there's a way but it's a bit involved and technical.

If your comfortable with a hex editor you can use The GIMP to open the raw file as type 'data' to find the offset needed then remove those bytes from the top of the file. In the options while opening the 'data' file you select greyscale-16bit, set the horizontal and vertical size to match what you shot, then use the offset entry to skip forward a few frames then offset until the image is lined up. Once you have that offset figured out, remove that many bytes from the top of the RAW file and retry running the script and viewing the result. If it still has artifacts, add or remove two bytes until it looks correct. If it looks extremely corrupt, remove only one byte.

A much better option is to save DNG directly using one of the newer software releases. DNG saving is far more reliable than the older RAW mode and gives the added benefit of saving some metadata such as the resolution, whiteballance, etc.

I hope this helps, let me know if you need further details.

1022mm

  • Newbie
  • *
  • Posts: 27
    • View Profile
Re: Maze artifact with 1.4
« Reply #3 on: June 21, 2019, 10:21:42 PM »
Thanks for the info, I will give that a try as soon as I get a free day to sit down and look at it.    I just upgraded the camera to v.0.3.1 so the DNG direct saving will be very helpful.

foobar

  • Krontech
  • Newbie
  • *****
  • Posts: 41
    • View Profile
Re: Maze artifact with 1.4
« Reply #4 on: June 22, 2019, 07:48:16 PM »
This is a known issue in versions v0.3.0 and earlier, and it is caused by the FPGA sometimes loosing the first pixel of a frame when shooting in 12-bit raw mode. The result is that the whole image gets offset relative to its bayer battern resulting in demosaic artifacts and color corruption. There was a discussion about the changes to 12-bit raw mode in the v0.3.1 beta software thread, which you can read here.

As part of the resolution of this issue, the bit packing order also changed between v0.3.0 and v0.3.1, so if you are using custom software to convert the 12-bit raw files you will also need to account for that change after updating the software. However, I do recommend using CinemaDNG for raw shooting as it requires fewer tools to process in Adobe Premier or Davinci Resolve.

1022mm

  • Newbie
  • *
  • Posts: 27
    • View Profile
Re: Maze artifact with 1.4
« Reply #5 on: September 09, 2019, 08:58:06 AM »
I've downloaded a few applications including GIMP and Raw Therapee to try and repair two usable shots captured earlier this year that were corrupted in this way.   I'm not having any luck.  Does the repair need to be performed on the raw file or each of the DNGs separately?   Is there any kind of workflow description or tutorial on how to do this? 

In Raw Therapee, I managed to get rid of the maze artifact using the VNG4 demosaicing selection, but it makes the image monochrome (color information is lost).
« Last Edit: September 09, 2019, 09:17:42 AM by 1022mm »

skronstein

  • Krontech
  • Jr. Member
  • *****
  • Posts: 59
    • View Profile
Re: Maze artifact with 1.4
« Reply #6 on: September 10, 2019, 02:31:09 PM »
I've downloaded a few applications including GIMP and Raw Therapee to try and repair two usable shots captured earlier this year that were corrupted in this way.   I'm not having any luck.  Does the repair need to be performed on the raw file or each of the DNGs separately?   Is there any kind of workflow description or tutorial on how to do this? 

In Raw Therapee, I managed to get rid of the maze artifact using the VNG4 demosaicing selection, but it makes the image monochrome (color information is lost).

How important is it to recover the shots?
The process to recover them will be a lot of work. The python raw2dng script will need to be modified to unpack the DNG files twice - once as normal and then once with the assumption that the first pixel was lost(this messes up the colors because applications that process DNGs expect certain colored pixels to be in certain locations, and the loss of the pixel affects the location of every pixel in the image). Then manually look at each resulting frame to see which looks right and keep the good one of the two.

1022mm

  • Newbie
  • *
  • Posts: 27
    • View Profile
Re: Maze artifact with 1.4
« Reply #7 on: September 16, 2019, 05:47:21 PM »
It wouldn't be the end of the world if I can't recover the shots, but they're good enough that I'd be willing to spend a few days to try whatever recovery methods are available.

skronstein

  • Krontech
  • Jr. Member
  • *****
  • Posts: 59
    • View Profile
Re: Maze artifact with 1.4
« Reply #8 on: September 17, 2019, 01:29:40 PM »
Which of the RAW save formats was used - 12 bit or 16 bit?

1022mm

  • Newbie
  • *
  • Posts: 27
    • View Profile
Re: Maze artifact with 1.4
« Reply #9 on: September 18, 2019, 11:55:05 AM »
Unfortunately I don't remember whether it was 12 or 16.   It was set to whatever the default save setting was when I got the camera in March of this year.  For the first 2 months of shooting, I didn't change any of the default settings other than the recording framerate and black calibration.