Lighthttpd Crashes

[mattD]

Hi all,

I'm beginning to have a pattern of a web UI crash.
I can successfully start recording and stop recording via http calls.
Once recorded I successfully ask the camera to save the buffer via /control/startFilesave
I get the status updating via /control/subscribe end point.

All of this are done without loading the webUI.
To download the saved file, i connect to the webUI and list the files in the play/save tab and download it via the webbrowser.
My recordings are 500 Mo done at almost 6000 FPS approximatly 5 seconds.

I had multiple hangs of the web service / downloads aborted.
Connecting via ssh I get those clues :


Using dmesg :
[  425.000000]    0  425    2279      174  0      0            0 cam-scgi
[  426.000000]    0  426    1271      62  0      0            0 mountNetDrive.s
[  427.000000]    0  427    16385    1355  0      0            0 camApp
[  431.000000]    0  431      486      25  0      0            0 inotifywait
[ 2842.000000]    33  2842      479      53  0      0            0 cat
[    0.000000] Out of memory: Kill process 385 (lighttpd) score 889 or sacrifice child

[    0.000000] Killed process 385 (lighttpd) total-vm:568852kB, anon-rss:561444kB, file-rss:108kB
[    0.000000] PHY: 0:00 - Link is Down
[    0.000000] PHY: 0:00 - Link is Up - 1000/Full
[    0.000000] I2C No Ack



tail -f /var/log/lighttpd/error.log
2025-06-17 07:25:22: (mod_scgi.c.2500) emergency exit: scgi: connection-fd: 9 fcgi-fd: 10
2025-06-17 07:25:22: (mod_scgi.c.2500) emergency exit: scgi: connection-fd: 11 fcgi-fd: 14
2025-06-17 07:27:04: (mod_cgi.c.1062) fork failed: Cannot allocate memory
2025-06-17 07:27:12: (mod_cgi.c.1062) fork failed: Cannot allocate memory
2025-06-17 07:27:12: (mod_cgi.c.1062) fork failed: Cannot allocate memory
2025-06-17 07:27:12: (mod_cgi.c.1062) fork failed: Cannot allocate memory
2025-06-17 07:33:37: (log.c.164) server started


Once crashed I do not get any http responses as  expected, but i can execute the http server again/usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.confto regain access.

Any advices ?I'm not very confortable with cgi-bin. It's just a server to that does calls to the programs ?Can someone guides me to search the calls in the source code ?

Some informations on the software :Chronos 1.4 with version 0.7.2    FPGA 3.25    PMIC firmware version 11

Thanks a lot for your help

 

[mattD]

Some potential of error, on my SDcard I saved 2 sequences of DNGs that represent 30 000 files on a vFat filesystem.

 

[hummingbirdography]

So if I'm understanding correctly, the API is working fine for you, but the CGI script for retrieving saved files keeps running out of memory?

Have you considered mounting a NFS or SMB share? That could alleviate the need for the CGI script all together.

Another thought- since it sounds like you're tech savvy, is you could install something like an SFTP or other file server, or even just use SCP to retrieve files.

 

[mattD]

Hi,

I'm not sure about the cgi workflow. If i'm correct it links the http call to a python script ? Can someone point me to the correct repo of the cgi configuration and programs. I looked a the github project but am not sure about the purpose of all.


I did some observations of the ram usage and CPU usage during those downloads. Also i have tryed with a class 10 SD card or a e-sata SSD Drive. I open a telnet port on the camera to transfert 500 Mo from my computer and the average speed is 7MO/s (lower than gigabit or 100 Mbits/s specs).

Here are some observations, using the webUI or the a scp download of the file.

The RAM usage, goes from stays at > 60% after i download via web UI. But i can do another download.
The download starts slow, (< 200Ko/s) with a CPU usage of < 15 %. This stays for some minutes.
Then the CPU usage goes near 60% and the speeds goes up to 7MO/s ....

During those observations, I did only download from a fresh reboot, i did not record and save and download a new video. I did it several time with no crashes.

Will test next week with the records and keep you updated

PS : Thanks for your answers, i opened another thread on the forum about the camera ethernet speed that is sub 100Mbits / s

 

[hummingbirdography]

The repo with both- the CGI scripts and the bash scripts they interact with, is chronos-http:

GitHub - krontech/chronos-http: HTTP API and Web interface for the Chronos Camera

HTTP API and Web interface for the Chronos Camera. Contribute to krontech/chronos-http development by creating an account on GitHub.

github.com

On the actual camera, the CGI scripts live inside:

/usr/share/chronos-http/cgi-bin/

I recommend ensuring you're running the latest 0.7.2 software image as well

 

[gstrauss]

What is the version of lighttpd? Please test with a recent version of lighttpd.
`lighttpd -v`

`2025-06-17 07:25:22: (mod_scgi.c.2500) emergency exit: scgi: connection-fd: 9 fcgi-fd: 10`

I removed that "emergency exit" trace from lighttpd 1.4.41 (released July 2016), so you're likely running a version of lighttpd older than that. 2016 is 9 years ago!

`anon-rss:561444kB` is a ridiculously large RSS for lighttpd. On my laptop, lighttpd can handle 10's of thousands of requests per second over TLS for 100 simultaneous clients, and still use less than 1/10 RSS of what you are reporting.

Modern version of lighttpd are both faster and use less memory than old lighttpd versions.

 

[mattD]

I will check the lighthttpd version, the camera was bought 3 month ago with the latest firmware installed.
Thanks for your update