23 days ago
In my app, users upload a bunch of files and we have a bunch of users in the other hosted system. I need to find a way to do a malicious upload scanner in the process of uploading. I kind of did a clamAV setup but the concurrency of it is not enough.
What is the Railway native way of handling secure uploads to bucket?
3 Replies
23 days ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • 23 days ago
23 days ago
The "native" way would be to use the S3 SDK to create a presigned URL to upload files directly into the bucket. But for that, the scanning would need to be on the client side. Otherwise, you can also retrieve the file in your backend after it's uploaded and scan it from there.
0x5b62656e5d
The "native" way would be to use the S3 SDK to create a presigned URL to upload files directly into the bucket. But for that, the scanning would need to be on the client side. Otherwise, you can also retrieve the file in your backend after it's uploaded and scan it from there.
23 days ago
Did the latter since I do not want to trust the client for this. The current situation is that we use presigned URLs with multipart uploads to the bucket, then a worker picks up and scans from the quarantine bucket and if it is healthy, copies to the main bucket. But this system is not super scalable, max I could do was 16 processes * replicas but Railway cannot have replicas with volumes attached so I get forced to deploy a bunch of clamavs. Basically the issue is the limitation on concurrency. Is there something like AWS GuardDuty type of thing we can use?
