Connection closed while reading from the driver
ozeitis
PROOP

a year ago

Hey so my code works fine (code below) locally but in Railway hosted I am getting the below error, anyway I can fix this?

Essentially I am using playwright to scrape some data and return it via the python API
curl -X POST "xxxxxxxxxx" \ -H "Content-Type: application/json" \ -d '{"url": "xxxxxxx {"detail":"Connection closed while reading from the driver"}%

Code attached

48 Replies

ozeitis
PROOP

a year ago

77fedaf9-b72c-45fa-b96e-6c1f7f459653


a year ago

you can check your HTTP logs for more information


ozeitis
PROOP

a year ago

/opt/venv/lib/python3.9/site-packages/playwright/driver/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.36' not found (required by /nix/store/xpxln7rqi3pq4m0xpnawhxb2gs0mn1s0-gcc-12.3.0-lib/lib/libstdc++.so.6)


ozeitis
PROOP

a year ago

Is the only way to solve this is to use my systemw ith docker?


a year ago

you mean write your own dockerfile?


ozeitis
PROOP

a year ago

Yeah


ozeitis
PROOP

a year ago

Dont see how else I can add this dependency


a year ago

its complicated for the most part, i'd recommend moving to a dockerfile, i can give your Dockerfile a once over after you write it?


ozeitis
PROOP

a year ago

Appreciate it, never really used docker tbh, I should learn it at some point. WIll build one and send it if I run into issue sthanks


a year ago

sounds good!


a year ago

additionally, would you be interested in using browserless? not having to install and bundle chrome into your container would decrease the build time, and also decrease the publish times!


ozeitis
PROOP

a year ago

So I use something similar, I use BrightData's bypass browser


ozeitis
PROOP

a year ago

Much better overall experience


ozeitis
PROOP

a year ago

and cheaper


a year ago

ah perfect, i just saw playwright and thought chrome install, my bad


ozeitis
PROOP

a year ago

I seem to have got it all working though thanks!


a year ago

with a Dockerfile? that was fast


ozeitis
PROOP

a year ago

I used perplexity tbh haha


ozeitis
PROOP

a year ago

FROM mcr.microsoft.com/playwright:v1.38.0-focal

# Install Python and pip
RUN apt-get update && apt-get install -y python3 python3-pip

# Set the working directory
WORKDIR /app

# Copy the requirements file
COPY requirements.txt .

# Install the Python dependencies
RUN pip3 install --no-cache-dir -r requirements.txt

# Copy the rest of the application code
COPY . .

# Install Playwright browsers and dependencies
RUN playwright install chromium
RUN playwright install-deps

# Set the command to run the application
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

a year ago

not ideal, you are using BrightData yet you are installing chrome now


a year ago

you are also installing python from apt-get, thats not a super stable install path for installing programming languages


ozeitis
PROOP

a year ago

Ah your right, stupid of me to confuse that. I wonder if I am not using brightdata correctly which prompted the need for this


a year ago

well first off, simply try removing those two playwright commands


ozeitis
PROOP

a year ago

Yes your right .. it still worked 🙂


a year ago

okay now instead of FROM playwright and then installing python with apt-get, you can do FROM python with a version


ozeitis
PROOP

a year ago

Hey sorry will check thatbut quick question


ozeitis
PROOP

a year ago

{"detail":"BrowserType.connect_over_cdp: WebSocket error: wss://xxxxxxxx@brd.superproxy.io:9222/ 401 Auth Failed (ip_blacklisted: 34.105.6.0/24)\nAuth Failed ip_blacklisted: 34.105.6.0/24\nCall log:\n wss://xxxxxxxx@brd.superproxy.io:9222/\n - wss://xxxxxxxx@brd.superproxy.io:9222/ 401 Auth Failed (ip_blacklisted: 34.105.6.0/24)\nAuth Failed ip_blacklisted: 34.105.6.0/24\n - wss://xxxxxxxx@brd.superproxy.io:9222/ error WebSocket was closed before the connection was established\n - wss://xxxxxxxx@brd.superproxy.io:9222/ WebSocket was closed before the connection was established\n - wss://xxxxxxxx@brd.superproxy.io:9222/ code=1006 reason=\n"}%


ozeitis
PROOP

a year ago

Is this saying BrightData blacklisted you guys?


ozeitis
PROOP

a year ago

Or the other way arround?


ozeitis
PROOP

a year ago

Every request was working at first


a year ago

very possible, but its also a 401 instead of 403, so maybe your credentials are wrong / expired?


ozeitis
PROOP

a year ago

Cant be, working locally fine


a year ago

Gotcha, do you have static Ips enabled?


ozeitis
PROOP

a year ago

Apologies for my lack of experience in this.

On Railway or BrightData?


a year ago

on your railway service, check the service settings


ozeitis
PROOP

a year ago

Disabled


ozeitis
PROOP

a year ago

Static Outbound IPs


a year ago

try enabling that


ozeitis
PROOP

a year ago

I did but they get banned fast it seems


a year ago

ah then you would need to get in contact with them


ozeitis
PROOP

a year ago

So its them blocking you? (BrightData)


a year ago

correct


ozeitis
PROOP

a year ago

Thanks for info Brody. Any info I can provide that will help them? Like any info about yoru servr or IP ranges or etc?


a year ago

you have a static IP now, so that's all you'd need to give them


ozeitis
PROOP

a year ago

Perfect thanks again. I always feel confident in my knowledge until I run into these issues then remeber I have so muc more to learn 🙂 Appreciate your pateince with me.


a year ago

I wish I could help here but unfortunately an IP ban is outside of what we can control


ozeitis
PROOP

a year ago

Of course I understand


ozeitis
PROOP

a year ago

You helped a lot already


Loading...