Workflow - Time

[benp]

Hi everyone.  I've been following this amazing little camera since just before the Kickstarter campaign ended.  The product has come a long way (congrats to the development team) and I've appreciated all of the contributors posting information, samples, and helpful tips for the user community.

I’m getting much closer to adding this camera to my kit and there’s only one outstanding issue I’m grappling with - time.  I’ve read older posts that indicate some shots can take 30 minutes or longer to export from the camera.  During that period, you can’t do anything but twiddle your thumbs and wait.  Is this correct?  Is there a bottleneck in workflow caused by the camera?

I have an open request for high speed analysis of firearm functionality (custom builders & loaders).  My concern is the length of time it would take from when the video is shot to when it could be viewed.  And then, how quickly another take could be filmed.

I have fast computers (MacPro 12-core cylinder) and HDDs (Thunderbolt and SAS arrays) available which are racked and mobile for handling processing.  The wildcard for me is how much time in the workflow the camera will add.  Can anyone share their experience with this?

[BiduleOhm]

The problem isn't moving the file from the camera to your computer but rather moving it from the RAM of the camera to your SD card in the camera because once it's done you can remove the SD card and put another one in in a few seconds. The camera also has an eSATA connector but I can't remember if it works with the current software and, if so, if it is faster than the SD card slot, another member can answer that better than me.

The time depends on what length of footage you want to save and in what format. If you want to save 10 sec of footage it'll take 10 times longer than 1 sec, so select only what you're interested instead of saving the whole buffer. Then you have the choice between MP4 and RAW formats, if I remember right the RAW is slower to save (no time needed in encoding of course but it's far bigger than MP4).

[Michael M.]

On a slow SD card, ive had to wait long. When I used an SSD, i was able to save rather quickly. I would highly reccomend getting an SSD. Can get you a photo of one of our lab setups later if you remind me. There is a 3D printed SSD mount on top with the SSD plugged into that with the wire running down into the port on the side of the camera. Not too intrusive and saves much faster than the standard sd card.

[nik282000]

Are you using the SSD in a USB enclosure or eSATA?

Currently I am using a 128gb USB thumb drive that is ok but not great.

[Michael M.]

The SSD is plugged into the esata port on the camera

[Michael M.]

The SSD is plugged into the esata port on the camera

https://www.amazon.ca/Kingston-Digital-120GB-SA400S37-120G/dp/B01N6JQS8C/ref=sr_1_1?ie=UTF8&qid=1545631239&sr=8-1&keywords=kingston+120

[benp]

I have no problem getting a handful of SSDs or super fast SDXC cards for the camera if it'll minimize the transfer time.  Is your mount (SSD enclosure) powered by the eSATA port (eSATAp)?  Or, does the mount require external power to connect to a SSD?  If the bottleneck is with the transfer out of the camera, how much faster is the SSD option over, say, a SDXC card spec'd at 300MB/s?  SSDs are cheaper now than the 2000x SDXC cards, but there's less gear and bulk involved with internal cards versus an external SSD.

There's an eSATA SSD dock available on Amazon.  <http://a.co/d/cOeInnK> Is this similar to what you're talking about having in your lab set-up?

Presuming I'm working with RAW and mostly dealing with 'bullet time' shots, which would only be a second or so in length but at a super high frame rate, what is a ballpark time for transfer out of the camera?  1min?  5min?  20min?

And, how much storage space is required per second of video?  Is a 120GB SSD large enough?  I already have a handful of 240GB SSDs.  Would I need an external array available for offloading the files during a shoot?

[BiduleOhm]

It should be about 2 GB per second of hi-speed footage: https://www.krontech.ca/uploads/9/3/8/3/93836312/chronos_1.4_datasheet_rev_4.pdf (I assume the size of the RAW file is the same as the one of the RAM buffer).

[Michael M.]

I have no problem getting a handful of SSDs or super fast SDXC cards for the camera if it'll minimize the transfer time.  Is your mount (SSD enclosure) powered by the eSATA port (eSATAp)?  Or, does the mount require external power to connect to a SSD?  If the bottleneck is with the transfer out of the camera, how much faster is the SSD option over, say, a SDXC card spec'd at 300MB/s?  SSDs are cheaper now than the 2000x SDXC cards, but there's less gear and bulk involved with internal cards versus an external SSD.

There's an eSATA SSD dock available on Amazon.  <http://a.co/d/cOeInnK> Is this similar to what you're talking about having in your lab set-up?

Presuming I'm working with RAW and mostly dealing with 'bullet time' shots, which would only be a second or so in length but at a super high frame rate, what is a ballpark time for transfer out of the camera?  1min?  5min?  20min?

And, how much storage space is required per second of video?  Is a 120GB SSD large enough?  I already have a handful of 240GB SSDs.  Would I need an external array available for offloading the files during a shoot?

The SSD is powered by the eSata port on the camera. No external power required. The cable adapter you need is SATA to eSATA. You can find a bunch on Amazon for cheap.

I can't answer what would be enouph storage for your needs nor can I answer how long the transfer time will be. This completely depends on how much you are recording in one shoot as well as the speed and brand or your card. I have waited anywhere from under a minute to 10 minutes for an export for a shot a few seconds long at about 2500 FPS and varied resolutions. There are too many variables to tell you an exact time. A total ballbark answer, with a good SSD, expect some saving times from 3-10 minutes at about 2500 FPS. (depending on the length)

I hope that answers your question.

[Martin]

Hi,

just did a test yesterday with the latest beta (0.3.1 beta 9), once with a Sandisk Extreme (read ~90mbyte/sec, write ~40mbyte/sec, pictured below) and once with an old WD Raptor 10k (the 74Gb one, in 3.5" casing), formatted using ext3, connected through esata.

First off, everything below is quite subjective and "ballpark".

@ 1280x1024x1057fps
Both, SD and eSATA, exported standard x264 at ~60fps (doesn't matter how you configure the codec).

Raw is quite different!

CinemaDNG was around 4-4.5fps for the SD and ~18-19fps for eSATA(!).
Filesize of each raw image: ~2.5Mbyte, size of batch/recording 1200 frames, so, saving took about 1 minute with eSata, but easily over 4 minutes with SD (slow USB sticks are probably even slower).

But you may be interested in highest framerates primarily, so I just tested again for different resolutions:

@ 336x120x31192fps (not the fastest setting, that's 336x96x38565fps)
x264 is maxing out(?) the write speed at about 180-220fps to both storage mediums.
CinemaDNG is also very fast, on eSATA at around 130-200fps but on the SD card there are again significant drops, judging only from the fps writespeed display, I'd say anywhere between 20 and 120fps (jumps quite a bit).
Filesize of each raw image: ~83kbyte again/always 1200 files/frames.

@ 640x240x8819fps
x264 writespeeds don't differ much from the settings used before.
CinemaDNG drops quite a bit again, I'd say around 25-40fps on SD, about 100 - 120fps(?) on eSATA.
Filesize is 304kbyte per raw file.

@800x96x17587fps
x264 maxes out again, almost always towards ~220 fps.
CinemaDNG is in between the previous two "measurements", ~20-90fps on SD, ~120-180fps on eSATA, so no surprises.
Filesize is at 154kbyte.

RAW 16bit/12bit is still an option as well, but even though it writes one big file (not like CinemaDNG/Tiff) it's only veeery slightly faster than CinemaDNG speeds(!)

Tiff is sometimes a bit faster, often a bit slower than CinemaDNG, but I didn't include those speeds, since Tiff filesize is slightly bigger (uncompressed) BUT, obviously, only 8bit per channel.

So, if you go with SSD/USB Stick/SD-Card aim for those products which can deliver highest response time when writing LOTS of small files(!), storage reviews test exactly for that scenario as well.

All the best
Martin

[benp]

First... wow.  I'm truly appreciative of the responses and information.

It's taken me a bit to wrap my head around the transfer time from the camera's RAM to external storage being based on frames transferred per second rather than MB/s.  LOL... I'm dense.  However it's calculated, the important thing is how fast the actual data can be transferred - which in turn determines how quickly you can work with a particular shot in an editing application and / or clear out the camera's RAM for additional shots.  The take away is... get the fastest external solution available.

The camera specs indicate that the SATA port is 'eSATA 3G.'  Unfortunately, the '3G' designation can either mean it is SATA II at 3Gbps or SATA III at 6Gbps.  I'll hopefully presume that the eSATA port operates under the latest standard.  The max spec transfer speed of eSATA III (6G) is 750MB/s.  Does the camera take full advantage of this transfer speed?  Or, is there a lower limit on the camera's eSATA port data rate?

It appears that most SSD's max out at around 500 to 550MB/s write speed.  Therefore, the bottleneck in transfer time is theoretically on the SSD.  After some research however, I'm curious if it's possible to purchase M.2 SSDs (PCIe/NVMe) with sequential write speeds of up to 1,900MB/s that can be adapted to a SATA III interface?  I've found enclosures to adapt these M.2 SSD's to USB3.1.  I can't seem to find one that'll connect via SATA.  IF it were possible to use M.2 SSD media, this would allow the camera to transfer data at it's maximum rate - which speeds up the overall workflow.

Cornflakes... would you share that photo of your lab set-up and the external SSD enclosure?

Thanks!

Ben

[BiduleOhm]

I'm pretty sure 3G means 3 Gpbs but don't quote me on that.

After some research however, I'm curious if it's possible to purchase M.2 SSDs (PCIe/NVMe) with sequential write speeds of up to 1,900MB/s that can be adapted to a SATA III interface?

It would be pointless because the speed of M2 comes from the fact the data isn't serialized over SATA, if you add a M2/SATA adapter you've basically made a classic SSD, a very expensive one

[Martin]

Maybe stay on the safe side and get the storage solution which has the lowest latency when it comes to tons of small-ish files, between, maybe 25Kbyte and 3Mbyte?

One where you can just dump all 8, 16 or 32Gbyte completely and writespeeds won't have a problem with thousands of small files?

I seem to remember that fast SLC-nand SSDs usually are among the fastest solutions for storing s...tons of small files while not breaking any sweat.

But there IS a limiting factor, in camera, not sure if the Chronos DNGs are compressed or not (I think they might be), but the camera with that software never wrote beyond 240fps in x264 and RAW wasn't too far behind when writing smallest filesizes/fastest recording speeds.

My 10k Raptor wasn't "really awake" writing around 17-18Mbytes/sec with 83Kbyte files..

IF (a big one) you plan for the 32Gbyte model (16secs realtime recording) and IF (even bigger) you often or always need to record AND dump the whole 16 secs, with a fast SSD and even with a fast spindrive you will mostly wait for the camera, not the storage, just don't use SD cards or USB sticks, they are 99% of the time NOT optimised for writing tons of small files in sequence as fast as possible.

All the best
Martin

[foobar]

Thanks Martin for the excellent post.  Just to share a couple technical details on the Camera's software, It's based on the TI Davinci DM8148 processor, and all of the storage needs to go through that CPU in order to get saved to media. On the storage end of things it supports 3Gbps SATA, SDHC, and, in the future, Gigabit Ethernet.

The maximum theoretical throughput via SDHC should be 25MB/s but in practice that is limited by the performance of your SD cards (the newer UHS standards are not supported by this CPU, but will operate at reduced speeds). SDXC cards can be made to work in the camera, but will probably need to be reformatted as FAT32 or EXT2/3/4 to work with the camera due to the lack of exFAT support on Linux.

The eSATA port is currently the fastest method for saving data, and despite the 3Gbps link, it is limited by the ability of the CPU to update the filesystem, and will hit 100% CPU utilization at around 60MB/s, so most high-performance SSDs are wasted on the Chronos, in fact most spindle drives will be able to keep up with this speed. I do recommend buying a reputable brand though - the kernel on the camera is a little too old for good TRIM support and may suffer after a couple of write cycles. The 120GB Kingston SSDs that have in our lab stock are especially bad in this regard.

The camera also supports a Gigabit Ethernet port that we hope to use for saving to network storage in a future update, but with the extra overhead of network protocols, checksums and the like we don't anticipate it to be much different from saving via eSATA to a decent SSD.

From the RAM to the Camera exists another possible bottleneck; the video port from the FPGA to the CPU operates at a 133MHz pixel clock (or 100MHz before 0.3.1-rc1). After overhead this works out to around 90fps from RAM into the CPU at 1280x1024 (or 60fps before 0.3.1-rc1). At smaller resolutions this can go as high as 250fps before the h.264 video encoder starts to drop frames, so the the software limits the video port to 230fps at all resolutions when saving.

DNG files are encoded in uncompressed RAW format, and will add a 4kB header to each frame. TIFF files are almost identical to DNG, but the pixels will be gamma-encoded 24-bit RGB on color cameras, or 8-bit greyscale on monochrome cameras.

[benp]

Hmmm.... well... that clarifies the speed issue.  If I need faster turnaround from shot to shot, I'll probably just have to purchase a second camera.

The SSDs that I already have and use with my current camera set-ups are 240GB Intel SSD 520 Series.  I paid a small mint for them 5 years ago, but they're bullet proof and can handle recording uncompressed video streams all day.  <https://ark.intel.com/products/66250/Intel-SSD-520-Series-240GB-2-5in-SATA-6Gb-s-25nm-MLC->  I would expect them to work great with whatever the Chronos could throw at them.

Intel SSD 520 Series Technical Specifications:
Capacity: 240GB
Form Factor: 2.5 inch
Interface: SATA 6.0 Gb/s
Flash Memory Technology: Multi-level Cell (MLC)
Lithography Type: 25 nm
Sequential Read (up to): 550 MB/s
Sequential Write (up to): 520 MB/s
Random Read (8GB Span) (up to): 50000 IOPS
Random Write (8GB Span) (up to): 60000 IOPS
Latency - Read: 80 µs
Latency - Write: 85 µs

Oh... I just remembered that I also have a Sonnet Tempo SSD Pro Plus in a 4TB RAID 0 configuration.  The PCIe card has two eSATA ports and is mounted in a Thunderbolt2 expansion chassis.  I wonder if this could be used with the Chronos camera?

My interest in achieving a fast workflow is based on the expectations of the clients that I typically work with.  In the broadcast and ad agency circles that I serve, clients and Creative Directors get antsy over the time it takes just to swap batteries.  So for media offloads, I'll typically have dedicated staff with redundant stations constantly hot-swapping the media cards and dumping the raw footage to HDDs.  This allows the camera(s) to keep rolling with minimal down time.

Admittedly, the cameras I'm usually dealing with max out at 120fps to 240fps and record straight to the SSD, C-FAST, P2, etc.  And, I've never shot with or been on a shoot with a camera that does higher fps.  In short, I don't know what I don't know about high fps work.  The Chronos, I'm learning, requires a shift in my expectations... and I'm okay with that.

Perhaps I'm also foolhardy, but my intent was to go for the full package on the camera.  Color sensor.  Maxed RAM.  Standard Lens.  Max warranty.  From reading the forums, I already understand that the RAM can be user upgraded for about 1/3 the cost of purchasing that configuration.  And, the reviews on the standard lens aren't that great.  However, purchasing a fully upgraded and standard configuration system means that it has a baseline expectation of performance direct from the manufacturer that can be compared with other systems if/when troubleshooting and comparison is needed.  Warranty repair (unless you have a back-up camera) takes you completely out of the game.  Therefore, downtime is avoided at all costs - including paying more for a system configuration I may never need/use.

Lenses:  I already have a rather significant inventory of quality SLR/DSLR lenses - from older, manual Nikon primes, macros, & zooms (18mm f2.8 up to 1000mm) to newer Canon EF & EF L primes and zooms (down to f1.4) as well as a handful of f2.8 Minolta primes and zooms.  There are many adapter options already available for using these lenses (with a crop factor) on the Chronos.  I also have SD and HD Fuji & Canon 2/3" bayonet mount broadcast lenses.  And, I don't currently own any PL lenses.  I haven't read any discussions where success in adapting 2/3" and PL lenses to the Chronos has been achieved.  So, these are probably off the table as options.  Finally, I've seen some recommendations for C / CS mount lenses in other KronTalk forum threads and would likely expand my lens collection with some of these models whether or not my existing lenses will successfully adapt to the Chronos.  It's always nice to have a native fallback option in case my other lenses are needed elsewhere.

Lighting: I have plenty of lighting options including 18,000 watts of Tungsten, 1,000 watts of LED, and access to HMI fixtures.

At this point, what I probably need to do is find one of the Chronos cameras to borrow or rent that I can try out ... or see about an in-person demo.  I won't be able to attend NAB and visit the Kron Tech booth this year due to a prior commitment that week.  Worst case is that I table the decision until later.

Sigh... I would really like this to work.