I am somewhat puzzled by the fact that you are talking about an E-mount adapter/ C-to-E mount adapter, as the only mount referred to commonly as E-Mount is Sonys Mirrorless Camera Mount, at least as far as i am aware. and that sony-E-Mount cant really be used on a C-mount Camera body (even CS-Mount would be tricky, but at least mostly possible on some cameras)
.
Now given the Fact you also mentioned DSLR-Lenses, and the Canon brand, i assume you mean some kind of Canon DSLR-Mount adapter for C-Mount (not a Canon User Myself, but i guess they have an mount called EOS or something, so you maybe mean that one?).
.
while i now also dont really know what kind of adapter / brand of adapter you are currently using, i still can generally reccomend the Metabones Brand:
https://www.baader-planetarium.com/de/c-mount-canon-eos.html (German Link, but you will for sure find one sold on B&H or something similar, depending on where you live)
Only thing to keep in mind, is that this adapter does not allow for any kind of apperture control, unless the lens can do it itself (for example an Aperture control ring on the Lens itself), as most modern Canon lenses require some kind of Electronic connection to the camera/ adapter to do that, which is also why i usually stay away from Canon mount lenses myself.
.
I personally own two of Metabones Speedboosters and got my hands on a few of their other adapters bevore, and they are usually very high quality, and i never heard anyone say something bad about them, besides beeing somewhat on the expensive side compard to some other brands. if you are looking for a good and reliable adapter, i would reccomend having a look there.
.
Hope this is the correct adapter you needed, otherwise, let me know or upload some images of the one you are using.
.
.
As far as additional lens support goes, most flexible, standardized and streamlined solution would be to by some standard Rail-Supports which are also used for DSLR-Style Cameras with bigger lenses and Cine-Cameras. There are quite a few different Styles and Designs of those, but most of the not too exotic ones can usually be bought for reasonable prices, Professional and Big-Name-Brands can get somewhat pricey quickly however.
Cant really reccomend a specific one, as they can be quite different and its a lot about preference and whatever you are doing with your camera/ Lenses, to find the best one for you.
Best would propably be to just put "lens rail support" in your Search Engine of choice and look up some images that come up, and go with whatever you like most, if you actually need additional lens Suports.
Those Rails are used a lot in Cinema Filmmaking, so i guess thats what you are looking for.
.
Didnt sound like you are actually putting the Camera in a lot of physical Stress/ acceleration / Forces, but just a bad quality adapter. but if you do, consider a full cage which fully supports the Lens all around, will most likely need to be custom made for this camera and your specific Lens, but i guess thats very much overkill from the sounds of your initial post.
.
.
For a quick and simple fix of the Problem i can also reccomend to just use someting to secure the Screws with, once tight.
I seen a lot of People use some Loctite Products for keeping all kinds of Screws from moving, which usually works really well. Now i never actually used any Loctite Product myself, but i know they offer a bunch of different stuff. You dont want the kind of Glue, that just will fix the Screws in place forever to never be loosened again, but rather the one, which just prevents the Screw from loosening itself, in case you actually need to readjust or even clean something for example.
Dont really know a name for that stuff sadly, or what it is exactly.
There Should be different brands of that too, but Loctite is the one that i know of, which has that for sure.
Now given your adapter is atleast half decent and the problem is actually just the screws itself, this might save you some money and Time if it works.