2 years ago
I´ve trying to deploy my scraping server, but the railway faild building e deploying
here is the log system and my dependencies in package json
I have authorization to scrapy, plus it's public data
"dependencies": {
"axios": "^1.6.8",
"dotenv": "^16.4.5",
"env-var": "^7.4.1",
"fs": "^0.0.1-security",
"jsdom": "^24.0.0",
"node-cron": "^3.0.3",
"node-schedule": "^2.1.1",
"nodemailer": "^6.9.13",
"openai": "^4.29.2",
"pg": "^8.11.3",
"playwright": "^1.42.1",
"prompt-sync": "^4.2.0",
"sequelize": "^6.37.1"
}
100 Replies
2 years ago
I can not recommend using Browserless enough, using it means you don't have to install Chrome and all of its needed dependencies into your container, and this will significantly speed up both the build and the publishing stage.
Browserless template here - https://railway.app/template/browserless
And here's my little example repo that shows you how to use browserless with playwright - https://github.com/brody192/playwright-example
2 years ago
but?
WSENDPOINTPLAYWRIGHT=${{Browserless.WSENDPOINTPLAYWRIGHT}}
2 years ago
yes that's provided by the template
2 years ago
nope, open your pre-existing project and click the + New button and search for the template
2 years ago
yes, as long as you follow my instructions
2 years ago
looks good!
const browser = await playwright.chromium.connect(process.env.WSENDPOINTPLAYWRIGHT);
2 years ago
yep that looks right
2 years ago
you'd also want to remove your custom build command that tries to install chrome
2 years ago
build logs please
2 years ago
okay and show me your playwright.chromium code please
2 years ago
are you sure you arent calling chromium.launch elsewhere?
2 years ago
your code is still trying to launch chrome
2 years ago
yes you do, please read the readme
WSENDPOINTPLAYWRIGHT=${{Browserless.WSENDPOINTPLAYWRIGHT}}
2 years ago
yes, that needs to be set on your app's service
2 years ago
instead of typing out the name of the service you use i simply said app as a generic cover, i am talking about the service in the top left of this screenshot

2 years ago
what?
2 years ago
that would be an issue
2 years ago
theres no need for pay lol
2 years ago
i appreciate the offer though ❤️
2 years ago
well now you are self-hosting it, so you only have to pay railway's usage pricing
2 years ago
nah your english is fine
2 years ago
is your service not working?
2 years ago
then the logs are normal
look this line 'page.waitForTimeout: page.waitForSelector: page.waitForSelector: Target page, context or browser has been closed'
2 years ago
browserless closes its chromium browser when not in use to save on resources
but it was in 100% use, i didn't understand mt well i'm going to change the codes and test again
2 years ago
each call to chromium.connect would spawn another browser and then it would get closed, if your service works fine, then theres nothing to worry about
2 years ago
im not sure what "1/8 of the data" means
2 years ago
the browser is closing because they did what your code asked them to do
2 years ago
thats how browserless works
beacuse the page are open and the console show this issues: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForSelector: page.waitForSelector: page.waitForSelector: page.waitForSelector: Target page, context or browser has been closed
at links (/app/pegarlinks_caixa.js:59:24)
at /app/async file:/app/test.js:23:29
2 years ago
okay but is that causing your app to not work correctly?
Yes, as it is a site that takes a long time to load there are waitforselector with very long timeouts
2 years ago
browserless has it's timeout set at 5 minutes, I can't see you needing more than that
