a year ago
I'm using pgloader to migrate data from MySQL to Postgres. Pgloader connects to the target database using TCP/Sockets and performs the migration tasks in parallel. Everything runs smoothly until I attempt to migrate to a Railway Postgres database, at which point I encounter an error.
Failed to connect to pgsql at "RAILWAY_PROXY" (port RAILWAY_PORT) as user "postgres": Database error: Socket error in "connect": EINTR (Interrupted system call)The migration process using pgloader initially proceeds without issues, but after a certain period, an error is thrown as described above. I suspect there might be an underlying issue such as a firewall restriction or similar configuration that I may have overlooked.
0 Replies
a year ago
56ecf4a7-da41-4ba0-a70c-7708dac4054f
a year ago
may i ask if this is planetscale to postgres on railway?
a year ago
Yes!
a year ago
would you happen to know how much data you have in planetscale?
a year ago
Not much, less than 1GB of data.
a year ago
any reason you arent going to a mysql database on railway?
a year ago
Internal needs, my team it's more familiar with Postgres now. Is this a problem?
a year ago
nope just curious
a year ago
how long does the connection stay alive before it dies like that?
a year ago
Max 3 minutes running
a year ago
does any data make it into the postgres database?
a year ago
Yep
a year ago
It's working, it seems to stop for a non particular reason
a year ago
what pgloader command are you running?
a year ago
LOAD database
from mysql://root@localhost:3306/DATABASE
into postgresql://postgres:PWD@HOST/railway
WITH include drop, create tables, create indexes, reset sequences, snake_case identifiers,
workers = 16, concurrency = 4,
multiple readers per thread, rows per range = 50000;I tested with 1 worker, 1 concurrency, 5k chunk. Same error… :/
a year ago
if you had just deployed a postgres database on railway, it would be postgres 16, is pgloader fully compatible with that version?
a year ago
Yep, just checked.
a year ago
I've made once without errors when testing, but it's inconsistent
a year ago
pgloader that was released in 2022 is compatible with postgres 16 that was released in 2023?
a year ago
Just worked again. I plan to keep running the process until I have the final version of my data.
a year ago
It seems to lost the connection when processing sometimes and pgloader does not have a mechanism to reconnect automatically.
a year ago
i think this is just instabilities with pgloader, no one else has reported any issues with their database connections in a good while
a year ago
I'm going with this. Much appreciate the help anyway!
a year ago
sorry i couldnt get you a better solution, i just dont have any way of testing this for myself