2 years ago
Hi! I have Pupeteer up and running with node on railway, but for some reason, my job just hangs when I open too many (10-20) tabs. I automatically kill any worker that is idle for 10 minutes and almost all of them end up idle and killed and restarted! Looks like Puppeteer just doesn't load pages on Railway. 10-20 tabs really is not that much and there are no resource spikes in the Railway console. Why else would this job get killed?
This job runs fine locally.
24 Replies
2 years ago
have you tried using the browserless template instead of running chrome in your container?
I do not want to pay for a browserless subscription and was hoping to host all on Railway. Browserless is quite pricy I believe
2 years ago
you can self host it on railway with the ready made browserless template
2 years ago
no you don't
I do not want to purchase Browserless as I need to do mass amounts of scraping. The job runs locally using Pupeteer, so why would I need to purchase Browserless?
2 years ago
but you should know scraping is in railways tos, so unless you have permission to scrape what you are scraping, it wouldn't be allowed on Railway
2 years ago
then you're all set
So how can I use this? I can look around, but i seems like you put the browserless url from this instance into your pupeteer initializer?
2 years ago
please see this example -
2 years ago
you don't need to pay anything to self host browserless, I'm not sure where you are getting that from
they have a license required on their site to self-host, so i assumed you had to pay
2 years ago
are you using it for commercial use?
not quite? we use it scrape information that powers our product, but its not like we sell the scraping as a service
2 years ago
sounds good to me