Long-term sustained capture (1TB of data) to SB or and network?

[dr_innovation]

We have a pair of 2.1 camera and have used them very successfully for one project -- but we had short clips downloaded vis SD cards since at that location we did not have network.

We are thinking of using them on a new project that would require a small window (maybe 128x128 image) captured as fast as we can.. but need to capture for a  moderately long term  (8hrs a day for 50 days). 8hrs at 7500fps of 128x128 is about 1TB but I can download over a network at outside the capture window.  I see 2 ways..  capturing to a large SD card in the camera or a USB SSD in the camera (a 1TB class 10 card should hold 8.6 hours) and then remotely access afterward via SMBor streaming over the network interface and capturing on the computer.  The streaming view over the network says it supports 1920x1080@60fps but I presume that is H264 and I need uncompressed.  Looking through manual I did not see any way to stream uncompressed.. anyone find a a way?  An alternative would be to  save to NSF  but have no idea what sustained rate I could do.   


Can anyone comment on experience with the tradeoffs between long-time SD capture and long-time network capture?  Am what I am describing even possible or I am limited to 32GB of capture then stop recording to dWe have a pair of 2.1 camera and have used them very successfully for one project -- but we had short clips downloaded vis SD cards since at that location we did not have network.

We are thinking of using them on a new project that would require a small window (maybe 128x128 image) captured as fast as we can.. but need to capture for a  moderately long term  (8hrs a day 50 days) and download over a network at outside the capture window.  I see 2 ways..  capturing to a large SD card in the camera (a 1TB class 10 card should hold 8.6 hours) and then remotely access afterward via SMBor streaming over the network interface and capturing on the computer.  The streaming view over the network says it supports 1920x1080@60fps so at 128x128 the data rate seems like we might get 7500fps at the same effective pixel/data rate.


Can anyone comment on experience with the tradeoffs between long-time SD capture and long-time  network capture?  Am what I am describing even possible or I am limited to 32GB of capture then stop recording to transfer/download?   

[mklinger]

We have a pair of 2.1 camera and have used them very successfully for one project -- but we had short clips downloaded vis SD cards since at that location we did not have network.

We are thinking of using them on a new project that would require a small window (maybe 128x128 image) captured as fast as we can.. but need to capture for a  moderately long term  (8hrs a day for 50 days). 8hrs at 7500fps of 128x128 is about 1TB but I can download over a network at outside the capture window.  I see 2 ways..  capturing to a large SD card in the camera or a USB SSD in the camera (a 1TB class 10 card should hold 8.6 hours) and then remotely access afterward via SMBor streaming over the network interface and capturing on the computer.  The streaming view over the network says it supports 1920x1080@60fps but I presume that is H264 and I need uncompressed.  Looking through manual I did not see any way to stream uncompressed.. anyone find a a way?  An alternative would be to  save to NSF  but have no idea what sustained rate I could do.   


Can anyone comment on experience with the tradeoffs between long-time SD capture and long-time network capture?  Am what I am describing even possible or I am limited to 32GB of capture then stop recording to dWe have a pair of 2.1 camera and have used them very successfully for one project -- but we had short clips downloaded vis SD cards since at that location we did not have network.

We are thinking of using them on a new project that would require a small window (maybe 128x128 image) captured as fast as we can.. but need to capture for a  moderately long term  (8hrs a day 50 days) and download over a network at outside the capture window.  I see 2 ways..  capturing to a large SD card in the camera (a 1TB class 10 card should hold 8.6 hours) and then remotely access afterward via SMBor streaming over the network interface and capturing on the computer.  The streaming view over the network says it supports 1920x1080@60fps so at 128x128 the data rate seems like we might get 7500fps at the same effective pixel/data rate.


Can anyone comment on experience with the tradeoffs between long-time SD capture and long-time  network capture?  Am what I am describing even possible or I am limited to 32GB of capture then stop recording to transfer/download?

Yeah, the Chronos can't do what you're suggesting here.  The first problem is that the current minimum horizontal resolution is 640, so you'd be dealing with 640x128 images.  Second, the camera can only record high speed to the internal RAM buffer, so you are limited to 32 GB like you say. 

As you have probably discovered, offloading the data from the RAM buffer to any external device (SD card, SSD, or Network) is pretty slow, so that will be a significant bottleneck.  The fastest option eSATA w/SSD would be less than 100 fps offloading, so 75x slower than your recording rate.

To achieve continual shooting the way you are thinking would require many (100+) cameras all coordinated and programmed to overlap.  It would be very complex to set up if it's even possible.

I would look to other cameras for this project, but I don't know of any high speed cameras that can currently do what you want but there may be some machine monitoring commercial cameras that can do that, but I'm not sure.  Definitely nothing mainstream can record continually at 7500 fps for 8 hours a day.

[Nikon1]

Apart from Asking myself whatever you would even do with 10,8 BILLION FRAMES (=about 13,9 Years Playback time at 25fps...?) worth of Footage (50 Days worth of 8 Hours per Day of 7500fps Footage), i can only agree with mklinger.
 The Chronos (and most other "Classical" Digital Highspeed-Cameras) Are not even Remotely Able to keep up with more than a Few Seconds or at best Minutes at a time Sustained Capture at Framerates of 7500fps.
 Thats Mainly because there Needs to be Plenty Of Real-Time Processing to be done to even be able to pull all That Data from the Sensor and put it somewhere it could be read from later on.
 Usually on most Highspeed-Cameras this is solved by just doing absolutely minimal Processing to the Data, and basically just dump Raw Sensor Data on the Fastest Memory you could fit in there for Reasonable Money.
 The Amount of RAM in the Camera itself is then usually also what limits your Recording Time. On the Chronos 2.1, Thats 8 to 32 GB, depending on Configuration.
 Streaming Video via Network or A Live View Video via HDMI on this Camera is only meant as a Preview, and will always be compressed, and very Limited Framerate, as it is just not possible to Push that Amount of Data via a Usual HDMI or Ethernet Connection.
 Streaming that Framerate to Memory would be Highly Limited by Processing Power and Speed of the Memory, and Processing Power on the Chronos Cameras is Fairly Limited, for one, they Try to make affordable Highspeed-Cameras at Krontech, also the Cameras are Limited in Space and Power Consumption.
 .
 So in Short, not even Remotely close to ever happen on a Chronos 2.1 Camera, and the Majority of Highspeed-Cameras with comparable Specifications.
 .
 If you for some weird Reason actually need to get this done the Way you described, and really capture Continous 8 Full Hours 7500fps Footage per Day for two Month in a Row, you need a Completely Different Beast of a Camera.
 What you then need are Highspeed-Cameras Specifically designed to STREAM the Footage Uncompressed.
 Those work Completely Different from the Kind of Highspeed-Camera the Chronos is, as they are Plugged directly into the PCI-e Slot of a Very Powerful Desktop Computer (or any other Connection that is able to Carry the Needed Amounts of Data on a minimal Processing Overhead), which takes Care of Processing all the Data, and Writing it to Permanent Storage, like a Fast SSD. The Camera or rather Sensor Unit in this Case, Just Throws RAW Sensor Data at the Computer, without any much Further Processing or saving.
 This Can also be done with Frame-Grabbers using Camera-Link on some Cameras or other Systems, but as far as i know, the current highest Performance Systems in this Direction Just Streaming the Uncompressed Sensor Data to Proprietary PCI-e Cards, and Basically make a Sensor unit and a Full Desktop-Computer into a Camera.
 These can then record as long as there is Free RAM on the PC, or until your SSD is full, Depending on whatever you end up getting (there are Computers with more Than 1TB of RAM around these days) and what you decide to save the Footage to.
 These Kinds of Highspeed-Cameras however are also a Way different Ballpark in terms of Price. Was Hands on with some of those, and was able to Play around a bit, and its Pretty crazy what they are Capable of, but a bit out of my League, not only in terms of what i actually need (Would be nice to have for sure, but not worth or practical having to use a full PC and a Fat Mains Power Supply all the Time for me), but also Budget.
 .
 In case, you are actually considering to get something like this, the ones i know of here, might help you to get Started in looking for something that fits your needs, or maybe those are even something you would buy (did no in depth Research about what Exact Framerates and Resolutions these are able to Stream Maximum, but its about the Direction you are talking. Prices on Request only for Cameras Like this, also dont forget, that you will need a Dedicated High-End Computer to Run those usually, no internal Screen and no internal Batteries):
 .
 https://www.imaging-solutions.de/XSM.html
 https://www.imaging-solutions.de/OS.html
 (Had some Years ago Tested the Os 7 and one of the XStream Cameras Personally, and they actually Recommended me looking at the Chronos 1.4 (when the 2.1 wasnt released yet), ended up getting a 2.1, Which fits my needs pretty well.)
 Similar Product, i have not Tested or seen Personally:
 https://idtvision.com/products/cameras/xs-series-cameras/
 .
 .
 And Again, i dont know what you are trying to do, but also consider, that even if you had such a Camera including a PC to Run it, and enough Space on a Disk to Capture all that Length of Highspeed-Footage, that you then also need to Somehow Process all those Files.
 In my Opinion, it would be way Easier (also worlds Cheaper) if you tried to find a way to get away with just capturing small Sequences of those 8Hours you actually needed, which Probably would easily be Possible on the 2.1 as is. Also Much less Footage to Deal With and Store just overall. If you for some Reason have no way around capturing the Full 8 Hours in one Sitting at 7500fps, getting a way higher End and much more Specialized Camera would be pretty much the Only way to go, apart from some even more expensive and Intense Projects like mklinger Mentioned, of Syncing up some crazy Number of Cameras somehow (needs Lenses for each Camera also, not even talking about Mounting and Cables and a Way to control the Whole Rig, which is not just way more Work and needs a ton of Space, but would pretty sure also cost more than a Single Specialized Camera for described use-case).

[Nikon1]

Thinking about this, this Freefly Camera came back to my mind ( https://freeflysystems.com/wave ), cause still way cheaper than the Other Cameras i Mentioned.
 That one does Sustained Recordings directly to NVME SSDs. Minimum Horizontal Resolution is 2048px Wide, and at Image Height of 160fps, you would be able to Capture 7861fps, Record Time Limited only by SSD Size, according to them.
 But still, if my Math is right, that would only give you about 27,7 Minutes of Recording Time for a 1TB Big SSD and a bit less then an Full Hour on a 2TB SSD. Pretty Limited Resolution Settings on that Camera. To be Fair, that is pretty impressive just in Terms of Pixel Throughput directly to Permanent Storage alone (Most Consumer Cameras wont even Shoot 30 or More Minutes of Continuous Video in 60fps because of Legal Limitations), but still far from the 8 Hours you are trying to Shoot in a Single Shot. They Quote a Price of 12,1k$ on their Site, which would be worth at least looking at in my Opinion, if you really Consider Getting something like that. Not Really Long Term Sustainable Recordings though, but at least somewhat Sustained for like an Hour or Two, given that Camera would even work with 4TB SSDs (Which i am not sure about, from a quick look, they do only Mention 2TB SSDs on the Specifications, but did no detailed Research on that. Do some Research your own, before you Buy!).

[Rainer]

The Paul Scherrer Institute in Switzerland built a sophisticated setup that allows them to stream 7 GB/s to servers using a highly modified PCO camera:
https://journals.iucr.org/s/issues/2017/06/00/pp5108/pp5108.pdf
It is surely difficult and expensive to replicate but I wanted to share this paper here.
dr_innovation, if you can share any infos about your application this would be very interesting to read. I'm just curious :-)

[dr_innovation]

Thanks for all the replies.     To answer why/what I would do this for, it is for modeling atmospheric turbulence in parallel with long-range imaging experiments. 
I'd save a few frames for visualization but mostly I'd process through a GPU or 2  to get per-frame estimates.  There are specialized turbulence sensors (I have a quote for one at $35K and 5 months for delivery )  but they don't produce images which I thought would help with understanding and evaluating our biometric data.     We have an event camera for some of these measurements but that is on backorder into mid-2023.   I'll may look at the feefly,  but maybe first do some experiments with the Chronos.  With that speed writing to SSD it is impressive.    And since I have the Chrons available though it might be interesting to explore first.


If I can capture for a second or so, save to NFS, then restart capture  I would be able to get data for estimates. (A separate GPU computer would read/process the files).     Even only get once per min,  I would at least get temporal atmospherics regularly during the day.   That would still be way more data about the turbulence than any other dataset in my space provides.

[Nikon1]

Have you had a Look at the "Live Slow Motion" Record Mode yet?
 [see Page 76 of the Manual: https://www.krontech.ca/wp-content/uploads/2021/09/RE-PDEV-10009-Chronos-User-Manual-Full-Software-Version-0.7.0.pdf ]
 If "Playback Loop Time (seconds)" is Set to "60" and "Playback Rate (fps)" is set to"60"; "Loop Record Mode" to "Continuous", that would allow you to Record 3600frames of Highspeed Footage every Minute and Play it Back over HDMI until it starts to record another Short bit of Footage a Minute Later, and so on, will work indefinitely as long as the Camera has Power.
 If you set the Resolution to 640x360px on the 2.1; that would allow you to capture a Short bit of Highspeed Footage at 7135fps, and output it over HDMI until it Captures again every Minute. That would be a bit more than a Half Second worth of Highspeed-Footage every Minute.
 Only thing you would need additionally would be Some HDMI Capture Card, that can Keep up with FullHD and 60fps, which most should be able to do.  About as Close to your Requirements as you are Going to get with the Chronos i assume, and that should give you a Reading for Whatever Measurements/Analysis you are going to do, and you can also Record the HDMI from the Capture Card to Permanent Storage, to just look at the Footage Later on, that would result in 8 Hours of Footage for 8 Hours of Analysis/ Run-Time.
 Only Drawbacks i see, is that you need to figure out how to tell the Analysis Software you are running; whenever the Jumpcut to the Footage that is Played back via HDMI is, which is Recorded a Minute Later, to get Accurate Datapoints (Could also just set a timer that goes off every Minute in Software, and Trigger the Start of that Timer approximately on your own, then Just dont use a bunch of the Frames when it Jumps, to allow for some Timing Error between you hitting the Trigger and the Actual thing).
 Also, that would be Compressed Footage to some Degree (HDMI Output is not bad Quality Though. For Sure not as good as good as Straight DNG-Output, but could be enough for your Application? idk, never ran Real Time Image Analysis myself...).
 .
 If thats Your Requirements, i would at least Try that, HDMI-Capture Cards are really affordable and you could even play around with the Playback Loop time to get a Readout / Data-Point every 30 Seconds or something else.

[Nikon1]

Probably a good Idea to get a Capture Card that does MJPG output for this Application (intraframe-only compression. see: https://en.wikipedia.org/wiki/Motion_JPEG ), as H.264 and such are all interframe Compressed, and would very likely Cause you a lot of Trouble with Analysis of small, fast Moving Particles. HDMI should only Compress the Color Space, and MJPG would then add a Layer of JPG Compression to every Frame, but Given High Enough Bitrate Output of the Capture Card, that should be very High Quality Footage Output.

[dr_innovation]

@Nikon1.. thanks for the suggestion.  Ill check but worry that anything going out over HDMI will be compressed.  Even MJPEG would be unacceptable compression for what I need.   But the idea of a looping collection, in uncompressed mode, could be interesting we'll see if it can be combined.   I've asked my student to basically look at using the  API raw frames at full speed to memory, then save the capture to esata where I can have a massive drive, then repeat the capture.   then at night we copy it over the network and process it.     Not sure how fast we can go but uncompress is critical for our analysis, even if its has larger temporal gaps.    It will be quite some time before we can get it setup and tested, just getting quotes for installing the power we need.

Thanks for everyone feedback.  Saved me going down a rabbit hole that would not have been productive.

[Nikon1]

@Nikon1.. thanks for the suggestion.  Ill check but worry that anything going out over HDMI will be compressed.  Even MJPEG would be unacceptable compression for what I need.   But the idea of a looping collection, in uncompressed mode, could be interesting we'll see if it can be combined.   I've asked my student to basically look at using the  API raw frames at full speed to memory, then save the capture to esata where I can have a massive drive, then repeat the capture.   then at night we copy it over the network and process it.     Not sure how fast we can go but uncompress is critical for our analysis, even if its has larger temporal gaps.    It will be quite some time before we can get it setup and tested, just getting quotes for installing the power we need.

Thanks for everyone feedback.  Saved me going down a rabbit hole that would not have been productive.

 Sorry for the Late Reply, but Auto-Save in Combination with Auto-Record option (see Page 91 and 92 of User Manual) pretty much already kind of does what you need, in combination with Record Length set to some Short Amount of Captured Frames (need to actually test, how long that will need to save, and set Record Length so it will meet with your needed Timing Intervals or Minimum needed Amount of Frames for a single Datapoint). By Default, the Camera will Set the Automatic Names of the Folders it saves DNG Frames to, to the Date and Time it starts the Saving Process, so you pretty much have a Timestamp there, which pretty much lines up with the Time it was shot at, for Very short Sequences. Not sure if that Timestamp will be Accurate for your Needs, and the Intervals could have some Variations, because the Camera will just start whenever it is Finished Saving, which could vary between each Sequence.
 So Coding some custom Solution might make sense for way better control of Timing of the Whole Process/ Timing of single Frames/ Maybe even recording synced between Cameras, as you have two of those.
 But i am sure, you will figure something out, just wanted to drop you a hint, that this Feature of the Camera exists already, in case you where not aware, might save you some work, if that was good enough to use already.

[rungvang]

We have a pair of 2.1 camera and have used them very successfully for one project -- but we had short clips downloaded vis SD cards since at that location we did not have network.

We are thinking of using them on a new project that would require a small window (maybe 128x128 image) captured as fast as we can.. but need to capture for a  moderately long term  (8hrs a day for 50 days). 8hrs at 7500fps of 128x128 is about 1TB but I can download over a network at outside the capture window.  I see 2 ways..  capturing to a large SD card in the camera or a USB SSD in the camera (a 1TB class 10 card should hold 8.6 hours) and then remotely access afterward via SMBor streaming over the network interface and capturing on the computer.  The streaming view over the network says it supports 1920x1080@60fps but I presume that is H264 and I need uncompressed.  Looking through manual I did not see any way to stream uncompressed.. anyone find a a way?  An alternative would be to  save to NSF  but have no idea what sustained rate I could do.   


Can anyone comment on experience with the tradeoffs between long-time SD capture and long-time network capture?  Am what I am describing even possible or I am limited to 32GB of capture then stop recording to dWe have a pair of 2.1 camera and have used them very successfully for one project -- but we had short clips downloaded vis SD cards since at that location we did not have network.

We are thinking of using them on a new project that would require a small window (maybe 128x128 image) captured as fast as we can.. but need to capture for a  moderately long term  (8hrs a day 50 days) and download over a network at outside the capture window.  I see 2 ways..  capturing to a large SD card in the camera (a 1TB class 10 card should hold 8.6 hours) and then remotely access afterward via SMBor streaming over the network interface and capturing on the computer.  The streaming view over the network says it supports 1920x1080@60fps so at 128x128 the data rate seems like we might get 7500fps at the same effective pixel/data rate.


Can anyone comment on experience with the tradeoffs between long-time SD capture and long-time  network capture?  Am what I am describing even possible or I am limited to 32GB of capture then stop recording to transfer/download?

Buy a Freefly Wave, it can record continuously into 2tb internal SSD.