a year ago
I´ve trying to deploy my scraping server, but the railway faild building e deploying
here is the log system and my dependencies in package json
I have authorization to scrapy, plus it's public data
"dependencies": {
"axios": "^1.6.8",
"dotenv": "^16.4.5",
"env-var": "^7.4.1",
"fs": "^0.0.1-security",
"jsdom": "^24.0.0",
"node-cron": "^3.0.3",
"node-schedule": "^2.1.1",
"nodemailer": "^6.9.13",
"openai": "^4.29.2",
"pg": "^8.11.3",
"playwright": "^1.42.1",
"prompt-sync": "^4.2.0",
"sequelize": "^6.37.1"
}
0 Replies
a year ago
I can not recommend using Browserless enough, using it means you don't have to install Chrome and all of its needed dependencies into your container, and this will significantly speed up both the build and the publishing stage.
Browserless template here - https://railway.app/template/browserless
And here's my little example repo that shows you how to use browserless with playwright - https://github.com/brody192/playwright-example
a year ago
but?
WSENDPOINTPLAYWRIGHT=${{Browserless.WSENDPOINTPLAYWRIGHT}}
a year ago
yes that's provided by the template
a year ago
nope, open your pre-existing project and click the + New
button and search for the template
a year ago
yes, as long as you follow my instructions
a year ago
looks good!
const browser = await playwright.chromium.connect(process.env.WSENDPOINTPLAYWRIGHT);
a year ago
yep that looks right
a year ago
you'd also want to remove your custom build command that tries to install chrome
a year ago
build logs please
a year ago
okay and show me your playwright.chromium code please
a year ago
are you sure you arent calling chromium.launch elsewhere?
a year ago
your code is still trying to launch chrome
a year ago
yes you do, please read the readme
WSENDPOINTPLAYWRIGHT=${{Browserless.WSENDPOINTPLAYWRIGHT}}
a year ago
yes, that needs to be set on your app's service
a year ago
instead of typing out the name of the service you use i simply said app as a generic cover, i am talking about the service in the top left of this screenshot
a year ago
what?
a year ago
that would be an issue
a year ago
theres no need for pay lol
a year ago
i appreciate the offer though ❤️
a year ago
well now you are self-hosting it, so you only have to pay railway's usage pricing
a year ago
nah your english is fine
a year ago
is your service not working?
a year ago
then the logs are normal
look this line 'page.waitForTimeout: page.waitForSelector: page.waitForSelector: Target page, context or browser has been closed'
a year ago
browserless closes its chromium browser when not in use to save on resources
but it was in 100% use, i didn't understand mt well i'm going to change the codes and test again
a year ago
each call to chromium.connect would spawn another browser and then it would get closed, if your service works fine, then theres nothing to worry about
a year ago
im not sure what "1/8 of the data" means
a year ago
the browser is closing because they did what your code asked them to do
a year ago
thats how browserless works
beacuse the page are open and the console show this issues: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForSelector: page.waitForSelector: page.waitForSelector: page.waitForSelector: Target page, context or browser has been closed
at links (/app/pegarlinks_caixa.js:59:24)
at /app/async file:/app/test.js:23:29
a year ago
okay but is that causing your app to not work correctly?
Yes, as it is a site that takes a long time to load there are waitforselector with very long timeouts
a year ago
browserless has it's timeout set at 5 minutes, I can't see you needing more than that