Migrating DB from Heroku to Railway, Tutorial Out of Date on Railway Side, Need Help

azeemh
PRO

a month ago

i was following this tutorial on migrating from heroku to railway and I've downloaded my pgdump but the interface on railway doesnt provide a link i can access to restore the dump so I can finally move my data from heroku to railway and then move domain etc go live on railway. https://blog.railway.com/p/postgre-backup The tutorial says to use the PGHOST name but the tutorial expects a host link such as containers-us-west-15.railway.app however now PGHOST gives postgres.railway.internal instead. Using it didnt work, and then I tried generating a proxy tcp but that also doesn't work.

azeem@x9:~/alphazed$ pg_restore -U postgres -h switchyard.proxy.rlwy.net:37627 -p 5432 -W -F t -d railway zedtopialatest.dump Password: pg_restore: error: could not translate host name "switchyard.proxy.rlwy.net:37627" to address: Name or service not known

when i tried using the http generate a domain it also doesn't work:

azeem@x9:~/alphazed$ pg_restore -U postgres -h postgres-production-0acf.up.railway.app -p 5432 -W -F t -d railway zedtopialatest.dump Password: pg_restore: error: connection to server at "postgres-production-0acf.up.railway.app" (35.212.94.98), port 5432 failed: Connection timed out Is the server running on that host and accepting TCP/IP connections?

Solved$10 Bounty

1 Replies

a month ago

n/a


azeemh
PRO

a month ago

Thanks Medim i couldnt find the project id anyways


azeemh
PRO

a month ago

i thnk i got it 6e426d89-591b-42ce-b8dc-9ac489140a01


a month ago

You got the port wrong, use the one the TCP proxy provided
pg_restore -U postgres -h tcp://[switchyard.proxy.rlwy.net](switchyard.proxy.rlwy.net) -p 37627 -W -F t -d railway zedtopialatest.dump


azeemh
PRO

a month ago

i don't know because the port doesn't show as labeled in the UI


azeemh
PRO

a month ago

1381704267093639200


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ pg_restore -U postgres -h tcp://switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump Password: pg_restore: error: could not translate host name "tcp://switchyard.proxy.rlwy.net" to address: Name or service not known


azeemh
PRO

a month ago

i tried your command verbatim copied and pasted and that didn't work either. i'm open to any suggestions.


a month ago

One sec, updating my postgres version so I can try it myself


azeemh
PRO

a month ago

if you'd like to see everything that I tried, you can see in my terminal session how I followed the tutorial and successfully ran the heroku pgdump but then when i followed the instructions to restore a pgdump on Railway it doesn't work.

I tried every method of connecting to my instance.

intead of the pghost which gives the first attempt, i then tried the database url, proxies, tcp proxies, and even deleted and redid the tcp proxy to no avail.


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump Password: azeem@x9:~/alphazed$ pg_restore -U postgres -h postgres.railway.internal -p 5432 -W -F t -d railway zedtopialatest.dump Password: pg_restore: error: could not translate host name "postgres.railway.internal" to address: Name or service not known azeem@x9:~/alphazed$ pg_restore -U postgres -h shinkansen.proxy.rlwy.net:51734 -p 5432 -W -F t -d railway zedtopialatest.dump Password: pg_restore: error: could not translate host name "shinkansen.proxy.rlwy.net:51734" to address: Name or service not known azeem@x9:~/alphazed$ pg_restore -U postgres -h postgresql://postgres:MBefEhIeAMUZAvVswHQcDzuAHFnqEABP@shinkansen.proxy.rlwy.net:51734/railway -p 5432 -W -F t -d railway zedtopialatest.dump Password: pg_restore: error: could not translate host name "postgresql://postgres:MBefEhIeAMUZAvVswHQcDzuAHFnqEABP@shinkansen.proxy.rlwy.net:51734/railway" to address: Name or service not known azeem@x9:~/alphazed$ pg_restore -U postgres -h postgres-production-0acf.up.railway.app -p 5432 -W -F t -d railway zedtopialatest.dump Password: pg_restore: error: connection to server at "postgres-production-0acf.up.railway.app" (35.212.94.98), port 5432 failed: Connection timed out Is the server running on that host and accepting TCP/IP connections? azeem@x9:~/alphazed$ pg_restore -U postgres -h switchyard.proxy.rlwy.net:37627 -p 5432 -W -F t -d railway zedtopialatest.dump Password: pg_restore: error: could not translate host name "switchyard.proxy.rlwy.net:37627" to address: Name or service not known azeem@x9:~/alphazed$ pg_restore -U postgres -h tcp://switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump Password: pg_restore: error: could not translate host name "tcp://switchyard.proxy.rlwy.net" to address: Name or service not known


azeemh
PRO

a month ago

i changed the command and fixed the syntax from your suggestion to get pg_restore -U postgres -h [switchyard.proxy.rlwy.net](switchyard.proxy.rlwy.net) -p 37627 -W -F t -d railway zedtopialatest.dump


azeemh
PRO

a month ago

it ran but it also had restore errors….

1381708548353953792


azeemh
PRO

a month ago

and when i logged in and tried to run my app or login with a pre-existing user it doesn't load


azeemh
PRO

a month ago

most importantly. NONE OF MY USERS OR THEIR DATA WAS RESTORED.


azeemh
PRO

a month ago

None of the data seems to have been sucessfully migrated.


azeemh
PRO

a month ago

if this can't work then I can't use this platform


azeemh
PRO

a month ago

I hope there's a way to migrate data otherwise we're screwed.


azeemh
PRO

a month ago

the data doesn't restore at all. nothing shows up.


azeemh
PRO

a month ago

and everytime i try to click anything on my app i get a retry later error message which is infuriating.


a month ago

That's because of some internal schemas and triggers that heroku uses on their dbs



a month ago

They have a resolution there but is a very annoying fix


a month ago

Some people on stackoverflow made it work by using the params --clean --no-acl --no-owner --section=pre-data --section=data


a month ago

But it is weird because the heroku specific command in the guide you linked supposedly does that


azeemh
PRO

a month ago

i tried the link you shared from heroku and it said:

azeem@x9:~/alphazed$ pg_restore -l zedtopialatest.dump > unfiltered.list
pg_restore: [archiver] unsupported version (1.15) in file header

azeemh
PRO

a month ago

i just tried the new params you shared and it says:

pg_restore -U postgres -h switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump --clean --no-acl --no-owner --section=pre-data --section=data
Password: 
pg_restore: error: could not execute query: ERROR:  cannot drop table public.users because other objects depend on it

the output is still running. would i have to delete or reset my database on railway and then try to restore again each time?


azeemh
PRO

a month ago

here's what it said:

```
````

1381722085658726410


azeemh
PRO

a month ago

i think i was able to get it to work using the information you shared


azeemh
PRO

a month ago

i had to make sure the version of pg dump and pg restore were all the same as my psql, and equal to or newer than the heroku version

1381738088660992180


azeemh
PRO

a month ago

you can use this update the readme. i see my updated data on the new site now. Thanks for helping and finding that heroku documentation


azeemh
PRO

a month ago


azeemh
PRO

a month ago

so make sure the versions are the same, dump using the tutorial and restore using the lists method with the syntax me and you built.


azeemh
PRO

a month ago

i got a majority of data to load now however there are still issues.

i just got Unknown primary key for table comments in model Comment.


azeemh
PRO

a month ago

so did this not copy over the primary keys?


azeemh
PRO

a month ago

when i dropped the db and tried to redo the procedure with a clean db to verify i don't see the new information populate either


azeemh
PRO

a month ago

none of the restores have worked after attempting to drop to restore from clean slate.


azeemh
PRO

a month ago

after the first attempt went thru not a single restore has worked since.


azeemh
PRO

a month ago

has anyone ever actually successfully restored a backup from heroku on railway in 2025?


a month ago

Did you try with a fresh deployed postgres instance or did you use the same one as before?


azeemh
PRO

a month ago

i did a redeploy and that did not work.


a month ago

Could u please try deleting that postgres instance, deploying a new one and running the restore command (the last one that worked) again?


a month ago

So we can take it from there


azeemh
PRO

a month ago

ok


azeemh
PRO

a month ago

i deleted the postgres instance, and all that shows now is my github repo and a fish-volume


azeemh
PRO

a month ago

i added a new postgres and it automatically added a new volume, so i deleted the old fish volume of the old postgres, it's provisioning right now


azeemh
PRO

a month ago

clean instance, now just ran the command with the new host address, pg_restore -U postgres -h [yamanote.proxy.rlwy.net](yamanote.proxy.rlwy.net) -p 19052 -W -F t -d railway --use-list filtered.list --no-owner zedtopialatest.dump and it ran with no errors but no data was migrated


azeemh
PRO

a month ago

so i login with the ssh and try to run db:create or db:migrate since it shows there are no tables yet, thus it had nowhere to put the data i tried to restore.


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ pg_restore -U postgres -h yamanote.proxy.rlwy.net -p 19052 -W -F t -d railway --use-list filtered.list --no-owner zedtopialatest.dump
Password: 
azeem@x9:~/alphazed$ railway ssh
> Select a service alphazed
  ✓ Connected to interactive shell                                                                                                                                     root@a3c835109450:/app# rails db:create 
We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml.

To resolve this issue:

- Did you create the database for this app, or delete it? You may need to create your database.
- Has the database name changed? Check your database.yml config has the correct database name.

To create your database, run:

        bin/rails db:create
Couldn't create 'railway' database. Please check your configuration.
rails aborted!
ActiveRecord::NoDatabaseError: We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml. (ActiveRecord::NoDatabaseError)

To resolve this issue:

- Did you create the database for this app, or delete it? You may need to create your database.
- Has the database name changed? Check your database.yml config has the correct database name.

To create your database, run:

        bin/rails db:create


Caused by:
PG::ConnectionBad: connection to server at "fd12:46a5:fbde:0:a000:3a:f120:7f6e", port 5432 failed: FATAL:  password authentication failed for user "postgres" (PG::ConnectionBad)
connection to server at "fd12:46a5:fbde:0:a000:3a:f120:7f6e", port 5432 failed: FATAL:  password authentication failed for user "postgres"

Tasks: TOP => db:create
(See full trace by running task with --trace)
root@a3c835109450:/app# 

'


a month ago

Can you use a db client to connect and check to see if the railway table actually exists?


azeemh
PRO

a month ago

NO TABLES EXIST AND IT DOESNT EVEN CONNECT


azeemh
PRO

a month ago

literally the repo can't connect to the new database


azeemh
PRO

a month ago

```[1af59f1a-2c61-41af-8fa5-b0967548a4d3] ActionView::Template::Error (There is an issue connecting to your database with your username/password, username: postgres.

Please check your database configuration to ensure the username/password are valid.

):```


azeemh
PRO

a month ago

do i have to delete the whole project or something? why is this platform so shitty?


a month ago

No, just the postgres service, does it says that it is running fine?


azeemh
PRO

a month ago

i deleted the whole project already because this is bullshit


a month ago

This wasn't supposed to be so hard, Railway spins up a plain postgres instance and serves it (not a custom one like Heroku)


azeemh
PRO

a month ago

what you say and the reality are two very different things


azeemh
PRO

a month ago

none of your tutorials are up to date


a month ago

I wonder if we can dump it without the extra data from Heroku


azeemh
PRO

a month ago

the only time it successfully imported some data it omitted keys on comments etc


a month ago

Heroku uses Amazon Aurora for DBs and I can assure you I exported and imported multiples dbs from that service to Railway


azeemh
PRO

a month ago

lol then why doesn't it work today?


a month ago

Let's go from the beggining, let's start with the dump


a month ago

Dump it again, use this command:
pg_dump DB_URL --no-acl --no-owner --no-privileges --clean --if-exists > dumptorailway.sql


a month ago

Then, before we proceed let's check if your Railway postgres instance is running fine, what's showing in the logs?


a month ago

(I updated the command above, I forgot about the --no-acl)


azeemh
PRO

a month ago

ctrl K doesn't show the import variables from heroku interface either.


azeemh
PRO

a month ago

i can't even try dumping it again yet because the new repo doesn't import variables from heroky


azeemh
PRO

a month ago

heroku*


azeemh
PRO

a month ago

typing import variables into the ctrl K box doesn't work unless you're at the variables screen. they should have just added a button this ctrl K interface is garbage. i just redid the whole thing now i have to delete the database url and replace it with railways


azeemh
PRO

a month ago

when i railway ssh it doesn't show the new project so i can't run a db:migrate


a month ago

Don't use ssh for such things


azeemh
PRO

a month ago

lmfao you haven't added a way to run a migration post deploy



azeemh
PRO

a month ago

i unlinked from the old project and linked to the new one


azeemh
PRO

a month ago

yes i said POST deploy, not pre


azeemh
PRO

a month ago

you install the new code and new migrations from the repo then run them.


azeemh
PRO

a month ago

and then if you want to run a one off rake task etc you have to ssh as well


azeemh
PRO

a month ago

anyways i ssh'ed and ran rails db:migrate so we have a new clean repo and db with migrations but empty


a month ago

I think you're skipping things a bit


azeemh
PRO

a month ago

what did i skip?


azeemh
PRO

a month ago

i made a whole new project and redid everything


a month ago

Wasn't our focus to migrate your postgres db first?


azeemh
PRO

a month ago

no it was to restore my postgres db


a month ago

yeah, from heroku to railway


azeemh
PRO

a month ago

well those two words mean different things and this is tech support bro


azeemh
PRO

a month ago

i used to have this job at IBM


azeemh
PRO

a month ago

so anyways i can't run a ruby on rails application without running rails db:migrate after the first install.


azeemh
PRO

a month ago

i did that. now my user table exists etc.


a month ago

And the data?


a month ago

Primary keys are fine?


azeemh
PRO

a month ago

the data we want to restore now is the question do we dump again or try the old command again?


a month ago

We dump only the data.


azeemh
PRO

a month ago

so last time i dumped i did pg_dump -h [cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com](cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com) -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump


azeemh
PRO

a month ago

how do you want to modify this?


a month ago

If we already have the tables and structure, we should only dump the data.

```pgdump DBURL \
--data-only \
--no-owner \
--no-privileges \
--no-acl \
--schema=public \

publicdataonly.sql```

I added the --data-only and the --schema=public


azeemh
PRO

a month ago

ok


a month ago

and for the restore we gonna use psql instead of pg_restore, that dump returned a .sql file as you can see
psql \ -h host-here \ -p port-here \ -U postgres \ -d railway \ -W \ -f public_data_only.sql

fill in the host-here and port-here since you redeployed your db


azeemh
PRO

a month ago

i dumped using azeem@x9:~/alphazed$ pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump --data-only --schema=public Password:


a month ago

did it error?


azeemh
PRO

a month ago

nope


a month ago

it's supposed to be a .sql file lol


azeemh
PRO

a month ago

oh lol should i redo?


a month ago

just change .dump to .sql


a month ago

U also forgot the rest of the flags but ok I guess


azeemh
PRO

a month ago

can you just type the command correctly so i can copy it


azeemh
PRO

a month ago

i'm tried


azeemh
PRO

a month ago

tired*


a month ago

```pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com \
-p 5432 \
-U udajirh8klodnn \
--data-only \
--no-owner \
--no-privileges \
--no-acl \
--schema=public \

publicdataonly.sql```


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ pg_dump cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com \
-p 5432 \
--data-only \
--no-owner \
--no-privileges \
--no-acl \
--schema=public \

publicdataonly.sql
pgdump: error: too many command-line arguments (first is " ") pgdump: hint: Try "pg_dump --help" for more information.


a month ago

edited it


a month ago

I deleted the -h and user


a month ago

by mistake


azeemh
PRO

a month ago

i did

pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.sql --data-only --schema=public --no-owner --no-privileges --no-acl
Password: 

azeemh
PRO

a month ago

it ran no errors


azeemh
PRO

a month ago

so now do we just restore?


a month ago

Yeah, try it


a month ago

how's the deployed DB on railway looking like?


azeemh
PRO

a month ago

it has all my tables but it's empty


azeemh
PRO

a month ago

so exactly a new repo with a migrate ready to run


a month ago

ok


azeemh
PRO

a month ago

i didn't run the restore or psql yet


azeemh
PRO

a month ago

so to confirm, should i try the restore or the psql?


a month ago

psql


azeemh
PRO

a month ago

i already know this syntax isnt gonna work:

psql \
  -h host-here \
  -p port-here \
  -U postgres \
  -d railway \
  -W \
  -f public_data_only.sql

and the PGhost variable is postgres.railway.internal


a month ago

no it isn't


a month ago

it's the tcp proxy


azeemh
PRO

a month ago

your platform gives me postgres.railway.internal when i copy PGHOST


azeemh
PRO

a month ago

i know that this syntax is wrong and we need tcpip link


azeemh
PRO

a month ago

the link for this new postgres is gondola.proxy.rlwy.net:40138


a month ago

psql \
-h gondola.proxy.rlwy.net \
-p 40138 \
-U postgres \
-d railway \
-W \
-f publicdataonly.sql


azeemh
PRO

a month ago

ok i ran it


azeemh
PRO

a month ago

no errors


azeemh
PRO

a month ago

let's see if it worked or actually pulled data


azeemh
PRO

a month ago

the tables are still empty


azeemh
PRO

a month ago

we successfully ran the commands but the data isn't there


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.sql --data-only --schema=public --no-owner --no-privileges --no-acl
Password: 
azeem@x9:~/alphazed$ psql \
  -h gondola.proxy.rlwy.net \
  -p 40138 \
  -U postgres \
  -d railway \
  -W \
  -f public_data_only.sql
Password: 
azeem@x9:~/alphazed$ 

a month ago

ok let's add verbose


a month ago

psql \ -h gondola.proxy.rlwy.net \ -p 40138 \ -U postgres \ -d railway \ -W \ -v ON_ERROR_STOP=1 \ -e \ -f public_data_only.sql


azeemh
PRO

a month ago

ran the verbose no issues, but no data is showing up either.


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ psql \
  -h gondola.proxy.rlwy.net \
  -p 40138 \
  -U postgres \
  -d railway \
  -W \
  -v ON_ERROR_STOP=1 \
  -e \
  -f public_data_only.sql
Password: 
azeem@x9:~/alphazed$ 

azeemh
PRO

a month ago

1381783931891748900


a month ago

Use a db client to look at data


a month ago

also, can you open that .sql file and see if there's any insert statement?


a month ago

if it ran without errors then the issue is with our dump


azeemh
PRO

a month ago

nah filename error


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ psql \
-h gondola.proxy.rlwy.net \
-p 40138 \
-U postgres \
-d railway \
-W \
-f zedtopialatest.sql


azeemh
PRO

a month ago

this is running


azeemh
PRO

a month ago

but there are errors


azeemh
PRO

a month ago

i'm waiting and i'll send the complete response


a month ago

oh yeah true, your filename is different


azeemh
PRO

a month ago

i'm getting a shitton of syntax errors


azeemh
PRO

a month ago

got a whole bunch of

Query buffer reset (cleared).
psql:zedtopialatest.sql:10237: error: invalid command \neverything
psql:zedtopialatest.sql:10238: error: invalid command \N
Query buffer reset (cleared).
psql:zedtopialatest.sql:10240: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10241: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10242: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10243: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10245: error: invalid command \n
psql:zedtopialatest.sql:10246: error: invalid command \N
psql:zedtopialatest.sql:10247: error: invalid command \N
Query buffer reset (cleared).
psql:zedtopialatest.sql:10248: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10249: error: invalid command \n
psql:zedtopialatest.sql:10250: error: invalid command \N
psql:zedtopialatest.sql:10251: error: invalid command \N
Query buffer reset (cleared).
psql:zedtopialatest.sql:10252: error: invalid command \n
psql:zedtopialatest.sql:10253: ERROR:  syntax error at or near "7856"
LINE 1: 7856 “Whosoever of you sees an evil, let him change it with ...
        ^
psql:zedtopialatest.sql:10253: ERROR:  syntax error at or near "and"
LINE 1: and if he is not able to do so, then [let him change it] wit...
        ^
psql:zedtopialatest.sql:10902: ERROR:  syntax error at or near "and"
LINE 1: and if he is not able to do so, then with his heart — and th...
        ^

azeemh
PRO

a month ago

also alot of ``` ^
psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "base64"
LINE 1: base64,iVBORw0KGgoAAAANSUhEUgAABG0AAAIuCAIAAACKEGJoAAAAAXNSR…
^
psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "width"
LINE 1: width: 50%;
^
psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "">

I clicked the like button because I like the feature. I clicked the like button because I like the feature. I clicked the like button because I like the feature. I clicked the like button because I like the feature. … ^ psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "">

Here are the sources that were linked:Here are the sources that were linked:


azeemh
PRO

a month ago

and no data is in the table:

1381786223324168200


a month ago

Ok, i've found some docs


a month ago

let's dump this again…


azeemh
PRO

a month ago

lol ok


azeemh
PRO

a month ago

for your ease of use in giving a new command here's the old one pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump


azeemh
PRO

a month ago

lmk the new one


a month ago

pg_dump -Fp --no-acl --no-owner --data-only -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > herokudump.dump

azeemh
PRO

a month ago

successfully ran no errors


azeemh
PRO

a month ago

pg_dump -Fp --no-acl --no-owner --data-only -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > herokudump.dump Password:


azeemh
PRO

a month ago

how do you want to restore?


a month ago

amazing.


azeemh
PRO

a month ago

should i try pg_restore or psql?


a month ago

pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W latest.dump


a month ago

pg restore, its a binary not a .sql file


a month ago

oops


a month ago

I put in the heroku creds


a month ago

use the railway one


azeemh
PRO

a month ago

pg_restore --verbose --clean --no-acl --no-owner -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W herokudump.dump ?


a month ago

use the railway creds


azeemh
PRO

a month ago

oh


azeemh
PRO

a month ago

yea


azeemh
PRO

a month ago

that too


azeemh
PRO

a month ago

so different cred and filename


a month ago

pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump


a month ago

how's it goin?


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W latest.dump pg_restore: error: input file is too short (read 0, expected 5) azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump pg_restore: error: input file appears to be a text format dump. Please use psql.


a month ago

ok let's use psql then


a month ago

psql --verbose -h [gondola.proxy.rlwy.net](gondola.proxy.rlwy.net) -p 40138 -U postgres -d railway -W -f herokudump.dump


a month ago

btw is the file called latest?


a month ago

"latest.dump"


azeemh
PRO

a month ago

nah you put Herokudump.dump


azeemh
PRO

a month ago

sorry all lowercase


azeemh
PRO

a month ago

psql \
-h gondola.proxy.rlwy.net \
-p 40138 \
-U postgres \
-d railway \
-W \
-f herokudump.dump


a month ago

then run this again with herokudump.dump


azeemh
PRO

a month ago

ok


a month ago

if it errors then we try the psql


azeemh
PRO

a month ago

i did already i told you


azeemh
PRO

a month ago

8:19


a month ago

ok then try the psql one


azeemh
PRO

a month ago

ok gonna try

psql \
    -h gondola.proxy.rlwy.net \
    -p 40138 \
    -U postgres \
    -d railway \
    -W \
    -f herokudump.dump

a month ago

forgot the verbose tag but ok.


azeemh
PRO

a month ago

it's giving a ton of syntax errors again like the last time


a month ago

you used latest.dump here instead of herokudump.dump


a month ago

did u run this command again but with herokudump.dump?


azeemh
PRO

a month ago

medim i ran the command i just pasted


azeemh
PRO

a month ago

psql \ -h gondola.proxy.rlwy.net \ -p 40138 \ -U postgres \ -d railway \ -W \ -f herokudump.dump


a month ago

yeah, and it errored out


a month ago

I want you to run this one since when u ran it you used the wrong file


azeemh
PRO

a month ago

you gave me a command with the wrong file last time


a month ago

yeah, my mistake


azeemh
PRO

a month ago

and bad syntax


azeemh
PRO

a month ago

just take your time and give me something correct


a month ago

This is the command the heroku docs tell us to run pg_restore --verbose --clean --no-acl --no-owner -h [gondola.proxy.rlwy.net](gondola.proxy.rlwy.net) -p 40138 -U postgres -d railway -W herokudump.dump


a month ago

I copy pasted it


a month ago

and forgot to change latest.dump to our actual dump name which is herokudump.dump


a month ago

That's why it throwed an error saying "error: input file is too short (read 0, expected 5)"


a month ago

Can you try it again but with the correct name file?


azeemh
PRO

a month ago

please just retype a correct command and i'll run that


a month ago

pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump


azeemh
PRO

a month ago

thanks


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump
pg_restore: error: input file appears to be a text format dump. Please use psql.

azeemh
PRO

a month ago

using that new dump format gives us text format dumps that give errors


azeemh
PRO

a month ago

and they all require psql in the text format


azeemh
PRO

a month ago

when we first got some progress we used the heroku docs you shared and updated the syntax from the guide that was broken

first we did a dump pg_dump -h [cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com](cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com) -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump then we would do a list and edit: pg_restore -l zedtopialatest.dump and then sed -e '/EVENT TRIGGER/ s/./;&/' unfiltered.list > filtered.list finally pg_restore -U postgres -h [gondola.proxy.rlwy.net](gondola.proxy.rlwy.net) -p 40138 -W -F t -d railway --use-list filtered.list --no-owner zedtopialatest.dump


azeemh
PRO

a month ago

the problem is the last time this was tried it didn't load all the comments or primary keys


a month ago

Ok, here's my last try.
Ur gonna delete that postgresql instance and spin up a new one.

You aren't going to do a db:migrate until we are done with the restore

You gonna dump everything except for the Heroku exclusive internal schemas, according to the Heroku docs here (https://devcenter.heroku.com/articles/heroku-postgres-import-export) on the Manual Dump section.

  1. pg_dump -Fp --no-acl --no-owner -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > latest.dump

  2. You gonna dump it into the railway db.
    pg_restore --verbose --clean --no-acl --no-owner -h TCP_PROXY_HERE -p PROXY_PORT_HERE -U postgres -d railway -W latest.dump

What's different from last time? Last time we used the --data-only flag.


azeemh
PRO

a month ago

ok. setup a new database


azeemh
PRO

a month ago

running the dump command copied and pasted from 1


a month ago

Command 2 will probably throw some warnings that can be safely ignored


azeemh
PRO

a month ago

i ran command 1 no issues


a month ago

amazing.


azeemh
PRO

a month ago

command 2 failed


azeemh
PRO

a month ago

gondola.proxy.rlwy.net:13146


azeemh
PRO

a month ago

is my proxy


a month ago

what error?


azeemh
PRO

a month ago

so i ran #2 as pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W latest.dump


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ pg_dump -Fp --no-acl --no-owner -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > latest.dump Password: azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W latest.dump pg_restore: error: input file appears to be a text format dump. Please use psql. azeem@x9:~/alphazed$


azeemh
PRO

a month ago

IT FAILED BRO


azeemh
PRO

a month ago

azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W latest.dump pg_restore: error: input file appears to be a text format dump. Please use psql.


a month ago

psql -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W -f latest.dump


azeemh
PRO

a month ago

it ran and didn't have any errors in the response,

1381799472052568174


azeemh
PRO

a month ago

let's check if the data loaded


azeemh
PRO

a month ago

THE DATA LOADED and the endpoints where i got errors prior seem to load. I haven't run into any errors yet!


azeemh
PRO

a month ago

I def wanna buy you a drink or a beer or whatever your thing is.


a month ago

Ok! You will probably notice some extra tables there


azeemh
PRO

a month ago

bless up thank you


a month ago

We imported the heroku internal tables there too


a month ago

We imported everything, actually


azeemh
PRO

a month ago

i'll go thru it all but i think thats cool


azeemh
PRO

a month ago

everything works


a month ago

I'm happy we got it working! Sorry if this was a bit confusing.


a month ago

At least we did it.


a month ago

<:kekw:788259314607325204>


azeemh
PRO

a month ago

i'll put it thru it's paces and update but for today this was excellent


azeemh
PRO

a month ago

YES this is a victory and you deserve a thanks


azeemh
PRO

a month ago

also a raise for putting up with all the bs


a month ago

no worries


azeemh
PRO

a month ago

thanks so much Medim


a month ago

now it's up to you to delete the extra tables and triggers heroku had


a month ago

and run that db:migrate


a month ago

(preferrably on your pre-deploy command or together with your start command, not in ssh)


azeemh
PRO

a month ago

is there a way to say run db:migrate post deploy


azeemh
PRO

a month ago

because in ruby on rails the code needs to exist and the tables must be initalized for me to add a new field or row, modify tables etc



azeemh
PRO

a month ago

iight. this is a post deploy thing. i guess the guys who designed the platform dont get it


azeemh
PRO

a month ago

but thanks for helping me


azeemh
PRO

a month ago

i really appreciate everything you did


a month ago

then use it on your start command


azeemh
PRO

a month ago

solution for anyone else:

start with a fresh postgres instance no tables no migrations nothing. dump your old one (my old creds you can replace with your own) and then use the psql to restore

  1. pg_dump -Fp --no-acl --no-owner -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > latest.dump

2. psql -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W -f latest.dump


Status changed to Solved chandrika about 1 month ago


a month ago

!s


Migrating DB from Heroku to Railway, Tutorial Out of Date on Railway Side, Need Help - Railway Help Station