a year ago
Hi! I have Pupeteer up and running with node on railway, but for some reason, my job just hangs when I open too many (10-20) tabs. I automatically kill any worker that is idle for 10 minutes and almost all of them end up idle and killed and restarted! Looks like Puppeteer just doesn't load pages on Railway. 10-20 tabs really is not that much and there are no resource spikes in the Railway console. Why else would this job get killed?
This job runs fine locally.
0 Replies
a year ago
have you tried using the browserless template instead of running chrome in your container?
I do not want to pay for a browserless subscription and was hoping to host all on Railway. Browserless is quite pricy I believe
a year ago
you can self host it on railway with the ready made browserless template
a year ago
no you don't
I do not want to purchase Browserless as I need to do mass amounts of scraping. The job runs locally using Pupeteer, so why would I need to purchase Browserless?
a year ago
but you should know scraping is in railways tos, so unless you have permission to scrape what you are scraping, it wouldn't be allowed on Railway
a year ago
then you're all set
So how can I use this? I can look around, but i seems like you put the browserless url from this instance into your pupeteer initializer?
a year ago
please see this example -
a year ago
you don't need to pay anything to self host browserless, I'm not sure where you are getting that from
they have a license required on their site to self-host, so i assumed you had to pay
a year ago
are you using it for commercial use?
not quite? we use it scrape information that powers our product, but its not like we sell the scraping as a service
a year ago
sounds good to me