a year ago
I am running a docker service, the service uses Puppeteer to do some work in a canvas element and export that into an image.
It is starting to work good. But it some point, creating new processes starting to crash with errors like:
Error: Failed to launch the browser process! spawn /usr/bin/google-chrome-stable EAGAIN
Failed to connect to the bus: Failed to connect to socket /run/dbus/systembussocket: No such file or directory
Error during rendering 4914:4914:0514/174626.133275:ERROR:spawn_subprocess.cc(145)] fork: Resource temporarily unavailable (11).
They all look relative to me. Are there some limits on the number of processes or resources? How can I know what is the limit?
9 Replies
a year ago
Running Chrome in the same container is an absolute pain as I'm sure you have come to release, it also bloats your image size quite a bit making build and deployment times higher than they would otherwise be, instead, I would highly recommend using Browserless, using it means you don't have to install Chrome and all of its needed dependencies into your container or deal with any of the issues involved in that.
Browserless template here - https://railway.app/template/browserless
And here's an example repo that shows you how to use Browserless with Puppeteer - https://github.com/brody192/puppeteer-example
Let me know if you have any issues!
a year ago
@brody thanks for the message. While I am also a fan of browserless and using it in some of my projects, it may work as a workaround in some use cases.
But for performance (!) reasons and cost, I have to launch browser instances directly.
I created a very minimal repository to reproduce the issue: https://github.com/lavrton/railway-test
On my test, with a pro account, it will crash after the 8th launch of the browser. I would expect that it least 32 instances can be launched without issues (1 per available CPU).
a year ago
Puppeteer launches browser processes on demand, and doesn't really add that much overhead as long as you use the private network. Could you please share exactly why browserless isn't sufficient for your needs?
As for the issue, Railway doesn't put any such limitations in place, something is simply missconfigured somewhere, have you tried switching to puppeteer-core and installing chromium manually with apt?
a year ago
@brody Yes, I tried installing chromium manually, the same result.
Could you please share exactly why browserless isn't sufficient for your needs?
First is network communication. I am generating many images in the browser and then doing some post process. Sending hundreds of images from browserless instance takes too much time. And it is very critical to make it as fast as possible.
The second is cost, browserless is more expensive than running work directly.
The third is having fewer dependencies on a third party.
a year ago
Just my 2 cents, if speed, efficiency, and costs are key, I would not be using a web browser in any capacity.
But I'll see if I can reproduce the error.
brody
Just my 2 cents, if speed, efficiency, and costs are key, I would not be using a web browser in any capacity. But I'll see if I can reproduce the error.
2 months ago
Can you please take a look into that again? It is getting a priority for us. Simply run docker in railway instance.
2 months ago
9 months ago when I said that I did not work for Railway, but now that I do, I unfortunately don't have time to look into issues that don't pertain to the Railway platform or product.
My 2 cents again, in 9 months, it would have been possible to come to a solution that doesn't invoice launching a browser.
brody
9 months ago when I said that I did not work for Railway, but now that I do, I unfortunately don't have time to look into issues that don't pertain to the Railway platform or product.My 2 cents again, in 9 months, it would have been possible to come to a solution that doesn't invoice launching a browser.
2 months ago
Thanks for the reply.
I do think it is related to railway platform. Most likely the way how railway starts the docker and what limitations it puts on it. I tried to run the same docker image on other cloud services (fly.io and koyeb) and it was working much better. Creating a lot more browser processes and failing only when memory limits are hit (while memory itself was smaller than on railway instances).