a year ago
i was following this tutorial on migrating from heroku to railway and I've downloaded my pgdump but the interface on railway doesnt provide a link i can access to restore the dump so I can finally move my data from heroku to railway and then move domain etc go live on railway. https://blog.railway.com/p/postgre-backup The tutorial says to use the PGHOST name but the tutorial expects a host link such as containers-us-west-15.railway.app however now PGHOST gives postgres.railway.internal instead. Using it didnt work, and then I tried generating a proxy tcp but that also doesn't work.
Password:
pg_restore: error: could not translate host name "switchyard.proxy.rlwy.net:37627" to address: Name or service not known```
when i tried using the http generate a domain it also doesn't work:
```azeem@x9:~/alphazed$ pg_restore -U postgres -h postgres-production-0acf.up.railway.app -p 5432 -W -F t -d railway zedtopialatest.dump
Password:
pg_restore: error: connection to server at "postgres-production-0acf.up.railway.app" (35.212.94.98), port 5432 failed: Connection timed out
Is the server running on that host and accepting TCP/IP connections?```265 Replies
a year ago
n/a
a year ago
You got the port wrong, use the one the TCP proxy provided
pg_restore -U postgres -h tcp://switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump
Password:
pg_restore: error: could not translate host name "tcp://switchyard.proxy.rlwy.net" to address: Name or service not knowni tried your command verbatim copied and pasted and that didn't work either. i'm open to any suggestions.
a year ago
One sec, updating my postgres version so I can try it myself
if you'd like to see everything that I tried, you can see in my terminal session how I followed the tutorial and successfully ran the heroku pgdump but then when i followed the instructions to restore a pgdump on Railway it doesn't work.
I tried every method of connecting to my instance.
intead of the pghost which gives the first attempt, i then tried the database url, proxies, tcp proxies, and even deleted and redid the tcp proxy to no avail.
Password:
azeem@x9:~/alphazed$ pg_restore -U postgres -h postgres.railway.internal -p 5432 -W -F t -d railway zedtopialatest.dump
Password:
pg_restore: error: could not translate host name "postgres.railway.internal" to address: Name or service not known
azeem@x9:~/alphazed$ pg_restore -U postgres -h shinkansen.proxy.rlwy.net:51734 -p 5432 -W -F t -d railway zedtopialatest.dump
Password:
pg_restore: error: could not translate host name "shinkansen.proxy.rlwy.net:51734" to address: Name or service not known
azeem@x9:~/alphazed$ pg_restore -U postgres -h postgresql://postgres:MBefEhIeAMUZAvVswHQcDzuAHFnqEABP@shinkansen.proxy.rlwy.net:51734/railway -p 5432 -W -F t -d railway zedtopialatest.dump
Password:
pg_restore: error: could not translate host name "postgresql://postgres:MBefEhIeAMUZAvVswHQcDzuAHFnqEABP@shinkansen.proxy.rlwy.net:51734/railway" to address: Name or service not known
azeem@x9:~/alphazed$ pg_restore -U postgres -h postgres-production-0acf.up.railway.app -p 5432 -W -F t -d railway zedtopialatest.dump
Password:
pg_restore: error: connection to server at "postgres-production-0acf.up.railway.app" (35.212.94.98), port 5432 failed: Connection timed out
Is the server running on that host and accepting TCP/IP connections?
azeem@x9:~/alphazed$ pg_restore -U postgres -h switchyard.proxy.rlwy.net:37627 -p 5432 -W -F t -d railway zedtopialatest.dump
Password:
pg_restore: error: could not translate host name "switchyard.proxy.rlwy.net:37627" to address: Name or service not known
azeem@x9:~/alphazed$ pg_restore -U postgres -h tcp://switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump
Password:
pg_restore: error: could not translate host name "tcp://switchyard.proxy.rlwy.net" to address: Name or service not knowni changed the command and fixed the syntax from your suggestion to get pg_restore -U postgres -h switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump
it ran but it also had restore errors....
and when i logged in and tried to run my app or login with a pre-existing user it doesn't load
and everytime i try to click anything on my app i get a retry later error message which is infuriating.
a year ago
That's because of some internal schemas and triggers that heroku uses on their dbs
a year ago
a year ago
They have a resolution there but is a very annoying fix
a year ago
Some people on stackoverflow made it work by using the params --clean --no-acl --no-owner --section=pre-data --section=data
a year ago
But it is weird because the heroku specific command in the guide you linked supposedly does that
i tried the link you shared from heroku and it said:
azeem@x9:~/alphazed$ pg_restore -l zedtopialatest.dump > unfiltered.list
pg_restore: [archiver] unsupported version (1.15) in file headeri just tried the new params you shared and it says:
pg_restore -U postgres -h switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump --clean --no-acl --no-owner --section=pre-data --section=data
Password:
pg_restore: error: could not execute query: ERROR: cannot drop table public.users because other objects depend on itthe output is still running. would i have to delete or reset my database on railway and then try to restore again each time?
i had to make sure the version of pg dump and pg restore were all the same as my psql, and equal to or newer than the heroku version
you can use this update the readme. i see my updated data on the new site now. Thanks for helping and finding that heroku documentation
so make sure the versions are the same, dump using the tutorial and restore using the lists method with the syntax me and you built.
i got a majority of data to load now however there are still issues.
i just got Unknown primary key for table comments in model Comment.
when i dropped the db and tried to redo the procedure with a clean db to verify i don't see the new information populate either
none of the restores have worked after attempting to drop to restore from clean slate.
has anyone ever actually successfully restored a backup from heroku on railway in 2025?
a year ago
Did you try with a fresh deployed postgres instance or did you use the same one as before?
a year ago
Could u please try deleting that postgres instance, deploying a new one and running the restore command (the last one that worked) again?
a year ago
So we can take it from there
i deleted the postgres instance, and all that shows now is my github repo and a fish-volume
i added a new postgres and it automatically added a new volume, so i deleted the old fish volume of the old postgres, it's provisioning right now
clean instance, now just ran the command with the new host address, pg_restore -U postgres -h yamanote.proxy.rlwy.net -p 19052 -W -F t -d railway --use-list filtered.list --no-owner zedtopialatest.dump and it ran with no errors but no data was migrated
so i login with the ssh and try to run db:create or db:migrate since it shows there are no tables yet, thus it had nowhere to put the data i tried to restore.
azeem@x9:~/alphazed$ pg_restore -U postgres -h yamanote.proxy.rlwy.net -p 19052 -W -F t -d railway --use-list filtered.list --no-owner zedtopialatest.dump
Password:
azeem@x9:~/alphazed$ railway ssh
> Select a service alphazed
✓ Connected to interactive shell root@a3c835109450:/app# rails db:create
We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml.
To resolve this issue:
- Did you create the database for this app, or delete it? You may need to create your database.
- Has the database name changed? Check your database.yml config has the correct database name.
To create your database, run:
bin/rails db:create
Couldn't create 'railway' database. Please check your configuration.
rails aborted!
ActiveRecord::NoDatabaseError: We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml. (ActiveRecord::NoDatabaseError)
To resolve this issue:
- Did you create the database for this app, or delete it? You may need to create your database.
- Has the database name changed? Check your database.yml config has the correct database name.
To create your database, run:
bin/rails db:create
Caused by:
PG::ConnectionBad: connection to server at "fd12:46a5:fbde:0:a000:3a:f120:7f6e", port 5432 failed: FATAL: password authentication failed for user "postgres" (PG::ConnectionBad)
connection to server at "fd12:46a5:fbde:0:a000:3a:f120:7f6e", port 5432 failed: FATAL: password authentication failed for user "postgres"
Tasks: TOP => db:create
(See full trace by running task with --trace)
root@a3c835109450:/app#
```'
a year ago
Can you use a db client to connect and check to see if the railway table actually exists?
Please check your database configuration to ensure the username/password are valid.
):```
do i have to delete the whole project or something? why is this platform so shitty?
a year ago
No, just the postgres service, does it says that it is running fine?
a year ago
This wasn't supposed to be so hard, Railway spins up a plain postgres instance and serves it (not a custom one like Heroku)
a year ago
I wonder if we can dump it without the extra data from Heroku
the only time it successfully imported some data it omitted keys on comments etc
a year ago
Heroku uses Amazon Aurora for DBs and I can assure you I exported and imported multiples dbs from that service to Railway
a year ago
Let's go from the beggining, let's start with the dump
a year ago
Dump it again, use this command:
pg_dump DB_URL --no-acl --no-owner --no-privileges --clean --if-exists > dumptorailway.sql
a year ago
Then, before we proceed let's check if your Railway postgres instance is running fine, what's showing in the logs?
a year ago
(I updated the command above, I forgot about the --no-acl)
i can't even try dumping it again yet because the new repo doesn't import variables from heroky
typing import variables into the ctrl K box doesn't work unless you're at the variables screen. they should have just added a button this ctrl K interface is garbage. i just redid the whole thing now i have to delete the database url and replace it with railways
a year ago
Don't use ssh for such things
anyways i ssh'ed and ran rails db:migrate so we have a new clean repo and db with migrations but empty
a year ago
I think you're skipping things a bit
a year ago
Wasn't our focus to migrate your postgres db first?
a year ago
yeah, from heroku to railway
so anyways i can't run a ruby on rails application without running rails db:migrate after the first install.
a year ago
And the data?
a year ago
Primary keys are fine?
the data we want to restore now is the question do we dump again or try the old command again?
a year ago
We dump only the data.
so last time i dumped i did pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump
a year ago
If we already have the tables and structure, we should only dump the data.
--data-only \
--no-owner \
--no-privileges \
--no-acl \
--schema=public \
> public_data_only.sql```
I added the --data-only and the --schema=public
a year ago
and for the restore we gonna use psql instead of pg_restore, that dump returned a .sql file as you can see
-h host-here \
-p port-here \
-U postgres \
-d railway \
-W \
-f public_data_only.sqlfill in the host-here and port-here since you redeployed your db
i dumped using azeem@x9:~/alphazed$ pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump --data-only --schema=public Password:
a year ago
did it error?
a year ago
it's supposed to be a .sql file lol
a year ago
just change .dump to .sql
a year ago
U also forgot the rest of the flags but ok I guess
a year ago
-p 5432 \
-U udajirh8klodnn \
--data-only \
--no-owner \
--no-privileges \
--no-acl \
--schema=public \
> public_data_only.sql```
azeem@x9:~/alphazed$ pg_dump cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com
-p 5432
--data-only
--no-owner
--no-privileges
--no-acl
--schema=public \
public_data_only.sql
pg_dump: error: too many command-line arguments (first is " ")
pg_dump: hint: Try "pg_dump --help" for more information.
a year ago
edited it
a year ago
I deleted the -h and user
a year ago
by mistake
i did
pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.sql --data-only --schema=public --no-owner --no-privileges --no-acl
Password: a year ago
Yeah, try it
a year ago
how's the deployed DB on railway looking like?
a year ago
ok
a year ago
psql
i already know this syntax isnt gonna work:
psql \
-h host-here \
-p port-here \
-U postgres \
-d railway \
-W \
-f public_data_only.sqland the PGhost variable is postgres.railway.internal
a year ago
no it isn't
a year ago
it's the tcp proxy
a year ago
psql
-h gondola.proxy.rlwy.net
-p 40138
-U postgres
-d railway
-W
-f public_data_only.sql
azeem@x9:~/alphazed$ pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.sql --data-only --schema=public --no-owner --no-privileges --no-acl
Password:
azeem@x9:~/alphazed$ psql \
-h gondola.proxy.rlwy.net \
-p 40138 \
-U postgres \
-d railway \
-W \
-f public_data_only.sql
Password:
azeem@x9:~/alphazed$ a year ago
ok let's add verbose
a year ago
-h gondola.proxy.rlwy.net \
-p 40138 \
-U postgres \
-d railway \
-W \
-v ON_ERROR_STOP=1 \
-e \
-f public_data_only.sql```
azeem@x9:~/alphazed$ psql \
-h gondola.proxy.rlwy.net \
-p 40138 \
-U postgres \
-d railway \
-W \
-v ON_ERROR_STOP=1 \
-e \
-f public_data_only.sql
Password:
azeem@x9:~/alphazed$ a year ago
Use a db client to look at data
a year ago
also, can you open that .sql file and see if there's any insert statement?
a year ago
if it ran without errors then the issue is with our dump
azeem@x9:~/alphazed$ psql
-h gondola.proxy.rlwy.net
-p 40138
-U postgres
-d railway
-W
-f zedtopialatest.sql
a year ago
oh yeah true, your filename is different
got a whole bunch of
Query buffer reset (cleared).
psql:zedtopialatest.sql:10237: error: invalid command \neverything
psql:zedtopialatest.sql:10238: error: invalid command \N
Query buffer reset (cleared).
psql:zedtopialatest.sql:10240: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10241: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10242: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10243: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10245: error: invalid command \n
psql:zedtopialatest.sql:10246: error: invalid command \N
psql:zedtopialatest.sql:10247: error: invalid command \N
Query buffer reset (cleared).
psql:zedtopialatest.sql:10248: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10249: error: invalid command \n
psql:zedtopialatest.sql:10250: error: invalid command \N
psql:zedtopialatest.sql:10251: error: invalid command \N
Query buffer reset (cleared).
psql:zedtopialatest.sql:10252: error: invalid command \n
psql:zedtopialatest.sql:10253: ERROR: syntax error at or near "7856"
LINE 1: 7856 “Whosoever of you sees an evil, let him change it with ...
^
psql:zedtopialatest.sql:10253: ERROR: syntax error at or near "and"
LINE 1: and if he is not able to do so, then [let him change it] wit...
^
psql:zedtopialatest.sql:10902: ERROR: syntax error at or near "and"
LINE 1: and if he is not able to do so, then with his heart — and th...
^also alot of ``` ^
psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "base64"
LINE 1: base64,iVBORw0KGgoAAAANSUhEUgAABG0AAAIuCAIAAACKEGJoAAAAAXNSR...
^
psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "width"
LINE 1: width: 50%;
^
psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "">I clicked the like button because I like the feature. I clicked the like button because I like the feature. I clicked the like button because I like the feature. I clicked the like button because I like the feature. ...
^
psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "">Here are the sources that were linked:Here are the sources that were linked:
a year ago
Ok, i've found some docs
a year ago
let's dump this again...
for your ease of use in giving a new command here's the old one pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump
a year ago
pg_dump -Fp --no-acl --no-owner --data-only -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > herokudump.dumpa year ago
amazing.
a year ago
pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W latest.dump
a year ago
pg restore, its a binary not a .sql file
a year ago
oops
a year ago
I put in the heroku creds
a year ago
use the railway one
pg_restore --verbose --clean --no-acl --no-owner -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W herokudump.dump ?
a year ago
use the railway creds
a year ago
pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump
a year ago
how's it goin?
pg_restore: error: input file is too short (read 0, expected 5)
azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump
pg_restore: error: input file appears to be a text format dump. Please use psql.a year ago
ok let's use psql then
a year ago
psql --verbose -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W -f herokudump.dump
a year ago
btw is the file called latest?
a year ago
"latest.dump"
psql
-h gondola.proxy.rlwy.net
-p 40138
-U postgres
-d railway
-W
-f herokudump.dump
a year ago
then run this again with herokudump.dump
a year ago
if it errors then we try the psql
a year ago
ok then try the psql one
ok gonna try
psql \
-h gondola.proxy.rlwy.net \
-p 40138 \
-U postgres \
-d railway \
-W \
-f herokudump.dumpa year ago
forgot the verbose tag but ok.
a year ago
you used latest.dump here instead of herokudump.dump
a year ago
did u run this command again but with herokudump.dump?
psql \
-h gondola.proxy.rlwy.net \
-p 40138 \
-U postgres \
-d railway \
-W \
-f herokudump.dump```
a year ago
yeah, and it errored out
a year ago
I want you to run this one since when u ran it you used the wrong file
a year ago
yeah, my mistake
a year ago
This is the command the heroku docs tell us to run pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump
a year ago
I copy pasted it
a year ago
and forgot to change latest.dump to our actual dump name which is herokudump.dump
a year ago
That's why it throwed an error saying "error: input file is too short (read 0, expected 5)"
a year ago
Can you try it again but with the correct name file?
a year ago
pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump
azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump
pg_restore: error: input file appears to be a text format dump. Please use psql.when we first got some progress we used the heroku docs you shared and updated the syntax from the guide that was broken
first we did a dump pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump then we would do a list and edit: pg_restore -l zedtopialatest.dump and then sed -e '/EVENT TRIGGER/ s/./;&/' unfiltered.list > filtered.list finally pg_restore -U postgres -h gondola.proxy.rlwy.net -p 40138 -W -F t -d railway --use-list filtered.list --no-owner zedtopialatest.dump
the problem is the last time this was tried it didn't load all the comments or primary keys
a year ago
Ok, here's my last try.
Ur gonna delete that postgresql instance and spin up a new one.
You aren't going to do a db:migrate until we are done with the restore
You gonna dump everything except for the Heroku exclusive internal schemas, according to the Heroku docs here (https://devcenter.heroku.com/articles/heroku-postgres-import-export) on the Manual Dump section.
pg_dump -Fp --no-acl --no-owner -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > latest.dump
You gonna dump it into the railway db.
pg_restore --verbose --clean --no-acl --no-owner -h TCP_PROXY_HERE -p PROXY_PORT_HERE -U postgres -d railway -W latest.dump
What's different from last time? Last time we used the --data-only flag.
a year ago
Command 2 will probably throw some warnings that can be safely ignored
a year ago
amazing.
a year ago
what error?
so i ran #2 as pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W latest.dump
Password:
azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W latest.dump
pg_restore: error: input file appears to be a text format dump. Please use psql.
azeem@x9:~/alphazed$ pg_restore: error: input file appears to be a text format dump. Please use psql.```
a year ago
psql -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W -f latest.dump
it ran and didn't have any errors in the response,
THE DATA LOADED and the endpoints where i got errors prior seem to load. I haven't run into any errors yet!
a year ago
Ok! You will probably notice some extra tables there
a year ago
We imported the heroku internal tables there too
a year ago
We imported everything, actually
a year ago
I'm happy we got it working! Sorry if this was a bit confusing.
a year ago
At least we did it.
a year ago
<:kekw:788259314607325204>
a year ago
no worries
a year ago
now it's up to you to delete the extra tables and triggers heroku had
a year ago
and run that db:migrate
a year ago
(preferrably on your pre-deploy command or together with your start command, not in ssh)
because in ruby on rails the code needs to exist and the tables must be initalized for me to add a new field or row, modify tables etc
iight. this is a post deploy thing. i guess the guys who designed the platform dont get it
a year ago
then use it on your start command
a year ago
solution for anyone else:
start with a fresh postgres instance no tables no migrations nothing. dump your old one (my old creds you can replace with your own) and then use the psql to restore
- pg_dump -Fp --no-acl --no-owner -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > latest.dump
2. psql -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W -f latest.dump
Status changed to Solved chandrika • about 1 year ago
a year ago
!s


