Migrating DB from Heroku to Railway, Tutorial Out of Date on Railway Side, Need Help
azeemh
PROOP

a year ago

i was following this tutorial on migrating from heroku to railway and I've downloaded my pgdump but the interface on railway doesnt provide a link i can access to restore the dump so I can finally move my data from heroku to railway and then move domain etc go live on railway. https://blog.railway.com/p/postgre-backup The tutorial says to use the PGHOST name but the tutorial expects a host link such as containers-us-west-15.railway.app however now PGHOST gives postgres.railway.internal instead. Using it didnt work, and then I tried generating a proxy tcp but that also doesn't work.

Password: 
pg_restore: error: could not translate host name "switchyard.proxy.rlwy.net:37627" to address: Name or service not known```

when i tried using the http generate a domain it also doesn't work: 

```azeem@x9:~/alphazed$ pg_restore -U postgres -h postgres-production-0acf.up.railway.app -p 5432 -W -F t -d railway zedtopialatest.dump
Password: 
pg_restore: error: connection to server at "postgres-production-0acf.up.railway.app" (35.212.94.98), port 5432 failed: Connection timed out
    Is the server running on that host and accepting TCP/IP connections?```
Solved$10 Bounty

265 Replies

a year ago

n/a


azeemh
PROOP

a year ago

Thanks Medim i couldnt find the project id anyways


azeemh
PROOP

a year ago

i thnk i got it 6e426d89-591b-42ce-b8dc-9ac489140a01


a year ago

You got the port wrong, use the one the TCP proxy provided

pg_restore -U postgres -h tcp://switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump


azeemh
PROOP

a year ago

i don't know because the port doesn't show as labeled in the UI


azeemh
PROOP

a year ago

1381704267093639228


azeemh
PROOP

a year ago

Password: 
pg_restore: error: could not translate host name "tcp://switchyard.proxy.rlwy.net" to address: Name or service not known

azeemh
PROOP

a year ago

i tried your command verbatim copied and pasted and that didn't work either. i'm open to any suggestions.


a year ago

One sec, updating my postgres version so I can try it myself


azeemh
PROOP

a year ago

if you'd like to see everything that I tried, you can see in my terminal session how I followed the tutorial and successfully ran the heroku pgdump but then when i followed the instructions to restore a pgdump on Railway it doesn't work.

I tried every method of connecting to my instance.

intead of the pghost which gives the first attempt, i then tried the database url, proxies, tcp proxies, and even deleted and redid the tcp proxy to no avail.


azeemh
PROOP

a year ago

Password: 
azeem@x9:~/alphazed$ pg_restore -U postgres -h postgres.railway.internal -p 5432 -W -F t -d railway zedtopialatest.dump
Password: 
pg_restore: error: could not translate host name "postgres.railway.internal" to address: Name or service not known
azeem@x9:~/alphazed$ pg_restore -U postgres -h shinkansen.proxy.rlwy.net:51734 -p 5432 -W -F t -d railway zedtopialatest.dump
Password: 
pg_restore: error: could not translate host name "shinkansen.proxy.rlwy.net:51734" to address: Name or service not known
azeem@x9:~/alphazed$ pg_restore -U postgres -h postgresql://postgres:MBefEhIeAMUZAvVswHQcDzuAHFnqEABP@shinkansen.proxy.rlwy.net:51734/railway -p 5432 -W -F t -d railway zedtopialatest.dump
Password: 
pg_restore: error: could not translate host name "postgresql://postgres:MBefEhIeAMUZAvVswHQcDzuAHFnqEABP@shinkansen.proxy.rlwy.net:51734/railway" to address: Name or service not known
azeem@x9:~/alphazed$ pg_restore -U postgres -h postgres-production-0acf.up.railway.app -p 5432 -W -F t -d railway zedtopialatest.dump
Password: 
pg_restore: error: connection to server at "postgres-production-0acf.up.railway.app" (35.212.94.98), port 5432 failed: Connection timed out
    Is the server running on that host and accepting TCP/IP connections?
azeem@x9:~/alphazed$ pg_restore -U postgres -h switchyard.proxy.rlwy.net:37627 -p 5432 -W -F t -d railway zedtopialatest.dump
Password: 
pg_restore: error: could not translate host name "switchyard.proxy.rlwy.net:37627" to address: Name or service not known
azeem@x9:~/alphazed$ pg_restore -U postgres -h tcp://switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump
Password: 
pg_restore: error: could not translate host name "tcp://switchyard.proxy.rlwy.net" to address: Name or service not known

azeemh
PROOP

a year ago

i changed the command and fixed the syntax from your suggestion to get pg_restore -U postgres -h switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump


azeemh
PROOP

a year ago

it ran but it also had restore errors....

1381708548353953792


azeemh
PROOP

a year ago

and when i logged in and tried to run my app or login with a pre-existing user it doesn't load


azeemh
PROOP

a year ago

most importantly. NONE OF MY USERS OR THEIR DATA WAS RESTORED.


azeemh
PROOP

a year ago

None of the data seems to have been sucessfully migrated.


azeemh
PROOP

a year ago

if this can't work then I can't use this platform


azeemh
PROOP

a year ago

I hope there's a way to migrate data otherwise we're screwed.


azeemh
PROOP

a year ago

the data doesn't restore at all. nothing shows up.


azeemh
PROOP

a year ago

and everytime i try to click anything on my app i get a retry later error message which is infuriating.


a year ago

That's because of some internal schemas and triggers that heroku uses on their dbs



a year ago

They have a resolution there but is a very annoying fix


a year ago

Some people on stackoverflow made it work by using the params --clean --no-acl --no-owner --section=pre-data --section=data


a year ago

But it is weird because the heroku specific command in the guide you linked supposedly does that


azeemh
PROOP

a year ago

i tried the link you shared from heroku and it said:

azeem@x9:~/alphazed$ pg_restore -l zedtopialatest.dump > unfiltered.list
pg_restore: [archiver] unsupported version (1.15) in file header

azeemh
PROOP

a year ago

i just tried the new params you shared and it says:

pg_restore -U postgres -h switchyard.proxy.rlwy.net -p 37627 -W -F t -d railway zedtopialatest.dump --clean --no-acl --no-owner --section=pre-data --section=data
Password: 
pg_restore: error: could not execute query: ERROR:  cannot drop table public.users because other objects depend on it

the output is still running. would i have to delete or reset my database on railway and then try to restore again each time?


azeemh
PROOP

a year ago

here's what it said:

1381722085658726410


azeemh
PROOP

a year ago

i think i was able to get it to work using the information you shared


azeemh
PROOP

a year ago

i had to make sure the version of pg dump and pg restore were all the same as my psql, and equal to or newer than the heroku version

1381738088660992180


azeemh
PROOP

a year ago

you can use this update the readme. i see my updated data on the new site now. Thanks for helping and finding that heroku documentation


azeemh
PROOP

a year ago


azeemh
PROOP

a year ago

so make sure the versions are the same, dump using the tutorial and restore using the lists method with the syntax me and you built.


azeemh
PROOP

a year ago

i got a majority of data to load now however there are still issues.

i just got Unknown primary key for table comments in model Comment.


azeemh
PROOP

a year ago

so did this not copy over the primary keys?


azeemh
PROOP

a year ago

when i dropped the db and tried to redo the procedure with a clean db to verify i don't see the new information populate either


azeemh
PROOP

a year ago

none of the restores have worked after attempting to drop to restore from clean slate.


azeemh
PROOP

a year ago

after the first attempt went thru not a single restore has worked since.


azeemh
PROOP

a year ago

has anyone ever actually successfully restored a backup from heroku on railway in 2025?


a year ago

Did you try with a fresh deployed postgres instance or did you use the same one as before?


azeemh
PROOP

a year ago

i did a redeploy and that did not work.


a year ago

Could u please try deleting that postgres instance, deploying a new one and running the restore command (the last one that worked) again?


a year ago

So we can take it from there


azeemh
PROOP

a year ago

ok


azeemh
PROOP

a year ago

i deleted the postgres instance, and all that shows now is my github repo and a fish-volume


azeemh
PROOP

a year ago

i added a new postgres and it automatically added a new volume, so i deleted the old fish volume of the old postgres, it's provisioning right now


azeemh
PROOP

a year ago

clean instance, now just ran the command with the new host address, pg_restore -U postgres -h yamanote.proxy.rlwy.net -p 19052 -W -F t -d railway --use-list filtered.list --no-owner zedtopialatest.dump and it ran with no errors but no data was migrated


azeemh
PROOP

a year ago

so i login with the ssh and try to run db:create or db:migrate since it shows there are no tables yet, thus it had nowhere to put the data i tried to restore.


azeemh
PROOP

a year ago

azeem@x9:~/alphazed$ pg_restore -U postgres -h yamanote.proxy.rlwy.net -p 19052 -W -F t -d railway --use-list filtered.list --no-owner zedtopialatest.dump
Password: 
azeem@x9:~/alphazed$ railway ssh
> Select a service alphazed
  ✓ Connected to interactive shell                                                                                                                                     root@a3c835109450:/app# rails db:create 
We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml.

To resolve this issue:

- Did you create the database for this app, or delete it? You may need to create your database.
- Has the database name changed? Check your database.yml config has the correct database name.

To create your database, run:

        bin/rails db:create
Couldn't create 'railway' database. Please check your configuration.
rails aborted!
ActiveRecord::NoDatabaseError: We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml. (ActiveRecord::NoDatabaseError)

To resolve this issue:

- Did you create the database for this app, or delete it? You may need to create your database.
- Has the database name changed? Check your database.yml config has the correct database name.

To create your database, run:

        bin/rails db:create


Caused by:
PG::ConnectionBad: connection to server at "fd12:46a5:fbde:0:a000:3a:f120:7f6e", port 5432 failed: FATAL:  password authentication failed for user "postgres" (PG::ConnectionBad)
connection to server at "fd12:46a5:fbde:0:a000:3a:f120:7f6e", port 5432 failed: FATAL:  password authentication failed for user "postgres"

Tasks: TOP => db:create
(See full trace by running task with --trace)
root@a3c835109450:/app# 
```'


a year ago

Can you use a db client to connect and check to see if the railway table actually exists?


azeemh
PROOP

a year ago

NO TABLES EXIST AND IT DOESNT EVEN CONNECT


azeemh
PROOP

a year ago

literally the repo can't connect to the new database


azeemh
PROOP

a year ago


 

Please check your database configuration to ensure the username/password are valid.

):```


azeemh
PROOP

a year ago

do i have to delete the whole project or something? why is this platform so shitty?


a year ago

No, just the postgres service, does it says that it is running fine?


azeemh
PROOP

a year ago

i deleted the whole project already because this is bullshit


a year ago

This wasn't supposed to be so hard, Railway spins up a plain postgres instance and serves it (not a custom one like Heroku)


azeemh
PROOP

a year ago

what you say and the reality are two very different things


azeemh
PROOP

a year ago

none of your tutorials are up to date


a year ago

I wonder if we can dump it without the extra data from Heroku


azeemh
PROOP

a year ago

the only time it successfully imported some data it omitted keys on comments etc


a year ago

Heroku uses Amazon Aurora for DBs and I can assure you I exported and imported multiples dbs from that service to Railway


azeemh
PROOP

a year ago

lol then why doesn't it work today?


a year ago

Let's go from the beggining, let's start with the dump


a year ago

Dump it again, use this command:

pg_dump DB_URL --no-acl --no-owner --no-privileges --clean --if-exists > dumptorailway.sql


a year ago

Then, before we proceed let's check if your Railway postgres instance is running fine, what's showing in the logs?


a year ago

(I updated the command above, I forgot about the --no-acl)


azeemh
PROOP

a year ago

ctrl K doesn't show the import variables from heroku interface either.


azeemh
PROOP

a year ago

i can't even try dumping it again yet because the new repo doesn't import variables from heroky


azeemh
PROOP

a year ago

heroku*


azeemh
PROOP

a year ago

typing import variables into the ctrl K box doesn't work unless you're at the variables screen. they should have just added a button this ctrl K interface is garbage. i just redid the whole thing now i have to delete the database url and replace it with railways


azeemh
PROOP

a year ago

when i railway ssh it doesn't show the new project so i can't run a db:migrate


a year ago

Don't use ssh for such things


azeemh
PROOP

a year ago

lmfao you haven't added a way to run a migration post deploy



azeemh
PROOP

a year ago

i unlinked from the old project and linked to the new one


azeemh
PROOP

a year ago

yes i said POST deploy, not pre


azeemh
PROOP

a year ago

you install the new code and new migrations from the repo then run them.


azeemh
PROOP

a year ago

and then if you want to run a one off rake task etc you have to ssh as well


azeemh
PROOP

a year ago

anyways i ssh'ed and ran rails db:migrate so we have a new clean repo and db with migrations but empty


a year ago

I think you're skipping things a bit


azeemh
PROOP

a year ago

what did i skip?


azeemh
PROOP

a year ago

i made a whole new project and redid everything


a year ago

Wasn't our focus to migrate your postgres db first?


azeemh
PROOP

a year ago

no it was to restore my postgres db


a year ago

yeah, from heroku to railway


azeemh
PROOP

a year ago

well those two words mean different things and this is tech support bro


azeemh
PROOP

a year ago

i used to have this job at IBM


azeemh
PROOP

a year ago

so anyways i can't run a ruby on rails application without running rails db:migrate after the first install.


azeemh
PROOP

a year ago

i did that. now my user table exists etc.


a year ago

And the data?


a year ago

Primary keys are fine?


azeemh
PROOP

a year ago

the data we want to restore now is the question do we dump again or try the old command again?


a year ago

We dump only the data.


azeemh
PROOP

a year ago

so last time i dumped i did pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump


azeemh
PROOP

a year ago

how do you want to modify this?


a year ago

If we already have the tables and structure, we should only dump the data.

  --data-only \
  --no-owner \
  --no-privileges \
  --no-acl \
  --schema=public \ 
> public_data_only.sql```

I added the --data-only and the --schema=public


azeemh
PROOP

a year ago

ok


a year ago

and for the restore we gonna use psql instead of pg_restore, that dump returned a .sql file as you can see

  -h host-here \
  -p port-here \
  -U postgres \
  -d railway \
  -W \
  -f public_data_only.sql

fill in the host-here and port-here since you redeployed your db


azeemh
PROOP

a year ago

i dumped using azeem@x9:~/alphazed$ pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump --data-only --schema=public Password:


a year ago

did it error?


azeemh
PROOP

a year ago

nope


a year ago

it's supposed to be a .sql file lol


azeemh
PROOP

a year ago

oh lol should i redo?


a year ago

just change .dump to .sql


a year ago

U also forgot the rest of the flags but ok I guess


azeemh
PROOP

a year ago

can you just type the command correctly so i can copy it


azeemh
PROOP

a year ago

i'm tried


azeemh
PROOP

a year ago

tired*


a year ago

  -p 5432 \
  -U udajirh8klodnn \
  --data-only \
  --no-owner \
  --no-privileges \
  --no-acl \
  --schema=public \ 
> public_data_only.sql```


azeemh
PROOP

a year ago

azeem@x9:~/alphazed$ pg_dump cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com

-p 5432

--data-only

--no-owner

--no-privileges

--no-acl

--schema=public \

public_data_only.sql

pg_dump: error: too many command-line arguments (first is " ")

pg_dump: hint: Try "pg_dump --help" for more information.


a year ago

edited it


a year ago

I deleted the -h and user


a year ago

by mistake


azeemh
PROOP

a year ago

i did

pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.sql --data-only --schema=public --no-owner --no-privileges --no-acl
Password: 

azeemh
PROOP

a year ago

it ran no errors


azeemh
PROOP

a year ago

so now do we just restore?


a year ago

Yeah, try it


a year ago

how's the deployed DB on railway looking like?


azeemh
PROOP

a year ago

it has all my tables but it's empty


azeemh
PROOP

a year ago

so exactly a new repo with a migrate ready to run


a year ago

ok


azeemh
PROOP

a year ago

i didn't run the restore or psql yet


azeemh
PROOP

a year ago

so to confirm, should i try the restore or the psql?


a year ago

psql


azeemh
PROOP

a year ago

i already know this syntax isnt gonna work:

psql \
  -h host-here \
  -p port-here \
  -U postgres \
  -d railway \
  -W \
  -f public_data_only.sql

and the PGhost variable is postgres.railway.internal


a year ago

no it isn't


a year ago

it's the tcp proxy


azeemh
PROOP

a year ago

your platform gives me postgres.railway.internal when i copy PGHOST


azeemh
PROOP

a year ago

i know that this syntax is wrong and we need tcpip link


azeemh
PROOP

a year ago

the link for this new postgres is gondola.proxy.rlwy.net:40138


a year ago

psql

-h gondola.proxy.rlwy.net

-p 40138

-U postgres

-d railway

-W

-f public_data_only.sql


azeemh
PROOP

a year ago

ok i ran it


azeemh
PROOP

a year ago

no errors


azeemh
PROOP

a year ago

let's see if it worked or actually pulled data


azeemh
PROOP

a year ago

the tables are still empty


azeemh
PROOP

a year ago

we successfully ran the commands but the data isn't there


azeemh
PROOP

a year ago

azeem@x9:~/alphazed$ pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.sql --data-only --schema=public --no-owner --no-privileges --no-acl
Password: 
azeem@x9:~/alphazed$ psql \
  -h gondola.proxy.rlwy.net \
  -p 40138 \
  -U postgres \
  -d railway \
  -W \
  -f public_data_only.sql
Password: 
azeem@x9:~/alphazed$ 

a year ago

ok let's add verbose


a year ago

  -h gondola.proxy.rlwy.net \
  -p 40138 \
  -U postgres \
  -d railway \
  -W \
  -v ON_ERROR_STOP=1 \
  -e \
  -f public_data_only.sql```


azeemh
PROOP

a year ago

ran the verbose no issues, but no data is showing up either.


azeemh
PROOP

a year ago

azeem@x9:~/alphazed$ psql \
  -h gondola.proxy.rlwy.net \
  -p 40138 \
  -U postgres \
  -d railway \
  -W \
  -v ON_ERROR_STOP=1 \
  -e \
  -f public_data_only.sql
Password: 
azeem@x9:~/alphazed$ 

azeemh
PROOP

a year ago

1381783931891748946


a year ago

Use a db client to look at data


a year ago

also, can you open that .sql file and see if there's any insert statement?


a year ago

if it ran without errors then the issue is with our dump


azeemh
PROOP

a year ago

nah filename error


azeemh
PROOP

a year ago

azeem@x9:~/alphazed$ psql

-h gondola.proxy.rlwy.net

-p 40138

-U postgres

-d railway

-W

-f zedtopialatest.sql


azeemh
PROOP

a year ago

this is running


azeemh
PROOP

a year ago

but there are errors


azeemh
PROOP

a year ago

i'm waiting and i'll send the complete response


a year ago

oh yeah true, your filename is different


azeemh
PROOP

a year ago

i'm getting a shitton of syntax errors


azeemh
PROOP

a year ago

got a whole bunch of

Query buffer reset (cleared).
psql:zedtopialatest.sql:10237: error: invalid command \neverything
psql:zedtopialatest.sql:10238: error: invalid command \N
Query buffer reset (cleared).
psql:zedtopialatest.sql:10240: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10241: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10242: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10243: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10245: error: invalid command \n
psql:zedtopialatest.sql:10246: error: invalid command \N
psql:zedtopialatest.sql:10247: error: invalid command \N
Query buffer reset (cleared).
psql:zedtopialatest.sql:10248: error: invalid command \n
Query buffer reset (cleared).
psql:zedtopialatest.sql:10249: error: invalid command \n
psql:zedtopialatest.sql:10250: error: invalid command \N
psql:zedtopialatest.sql:10251: error: invalid command \N
Query buffer reset (cleared).
psql:zedtopialatest.sql:10252: error: invalid command \n
psql:zedtopialatest.sql:10253: ERROR:  syntax error at or near "7856"
LINE 1: 7856 “Whosoever of you sees an evil, let him change it with ...
        ^
psql:zedtopialatest.sql:10253: ERROR:  syntax error at or near "and"
LINE 1: and if he is not able to do so, then [let him change it] wit...
        ^
psql:zedtopialatest.sql:10902: ERROR:  syntax error at or near "and"
LINE 1: and if he is not able to do so, then with his heart — and th...
        ^

azeemh
PROOP

a year ago

also alot of ``` ^

psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "base64"

LINE 1: base64,iVBORw0KGgoAAAANSUhEUgAABG0AAAIuCAIAAACKEGJoAAAAAXNSR...

^

psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "width"

LINE 1: width: 50%;

^

psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "">I clicked the like button because I like the feature. I clicked the like button because I like the feature. I clicked the like button because I like the feature. I clicked the like button because I like the feature. ...

^

psql:zedtopialatest.sql:2431: ERROR: syntax error at or near "">Here are the sources that were linked:Here are the sources that were linked:


azeemh
PROOP

a year ago

and no data is in the table:

1381786223324168232


a year ago

Ok, i've found some docs


a year ago

let's dump this again...


azeemh
PROOP

a year ago

lol ok


azeemh
PROOP

a year ago

for your ease of use in giving a new command here's the old one pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump


azeemh
PROOP

a year ago

lmk the new one


a year ago

pg_dump -Fp --no-acl --no-owner --data-only -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > herokudump.dump

azeemh
PROOP

a year ago

successfully ran no errors


azeemh
PROOP

a year ago

Password: 

azeemh
PROOP

a year ago

how do you want to restore?


a year ago

amazing.


azeemh
PROOP

a year ago

should i try pg_restore or psql?


a year ago

pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W latest.dump


a year ago

pg restore, its a binary not a .sql file


a year ago

oops


a year ago

I put in the heroku creds


a year ago

use the railway one


azeemh
PROOP

a year ago

pg_restore --verbose --clean --no-acl --no-owner -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W herokudump.dump ?


a year ago

use the railway creds


azeemh
PROOP

a year ago

oh


azeemh
PROOP

a year ago

yea


azeemh
PROOP

a year ago

that too


azeemh
PROOP

a year ago

so different cred and filename


a year ago

pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump


a year ago

how's it goin?


azeemh
PROOP

a year ago

pg_restore: error: input file is too short (read 0, expected 5)
azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump
pg_restore: error: input file appears to be a text format dump. Please use psql.

a year ago

ok let's use psql then


a year ago

psql --verbose -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W -f herokudump.dump


a year ago

btw is the file called latest?


a year ago

"latest.dump"


azeemh
PROOP

a year ago

nah you put Herokudump.dump


azeemh
PROOP

a year ago

sorry all lowercase


azeemh
PROOP

a year ago

psql

-h gondola.proxy.rlwy.net

-p 40138

-U postgres

-d railway

-W

-f herokudump.dump


a year ago

then run this again with herokudump.dump


azeemh
PROOP

a year ago

ok


a year ago

if it errors then we try the psql


azeemh
PROOP

a year ago

i did already i told you


azeemh
PROOP

a year ago

8:19


a year ago

ok then try the psql one


azeemh
PROOP

a year ago

ok gonna try

psql \
    -h gondola.proxy.rlwy.net \
    -p 40138 \
    -U postgres \
    -d railway \
    -W \
    -f herokudump.dump

a year ago

forgot the verbose tag but ok.


azeemh
PROOP

a year ago

it's giving a ton of syntax errors again like the last time


a year ago

you used latest.dump here instead of herokudump.dump


a year ago

did u run this command again but with herokudump.dump?


azeemh
PROOP

a year ago

medim i ran the command i just pasted


azeemh
PROOP

a year ago

psql \
    -h gondola.proxy.rlwy.net \
    -p 40138 \
    -U postgres \
    -d railway \
    -W \
    -f herokudump.dump```


a year ago

yeah, and it errored out


a year ago

I want you to run this one since when u ran it you used the wrong file


azeemh
PROOP

a year ago

you gave me a command with the wrong file last time


a year ago

yeah, my mistake


azeemh
PROOP

a year ago

and bad syntax


azeemh
PROOP

a year ago

just take your time and give me something correct


a year ago

This is the command the heroku docs tell us to run pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump


a year ago

I copy pasted it


a year ago

and forgot to change latest.dump to our actual dump name which is herokudump.dump


a year ago

That's why it throwed an error saying "error: input file is too short (read 0, expected 5)"


a year ago

Can you try it again but with the correct name file?


azeemh
PROOP

a year ago

please just retype a correct command and i'll run that


a year ago

pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump


azeemh
PROOP

a year ago

thanks


azeemh
PROOP

a year ago

azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 40138 -U postgres -d railway -W herokudump.dump
pg_restore: error: input file appears to be a text format dump. Please use psql.

azeemh
PROOP

a year ago

using that new dump format gives us text format dumps that give errors


azeemh
PROOP

a year ago

and they all require psql in the text format


azeemh
PROOP

a year ago

when we first got some progress we used the heroku docs you shared and updated the syntax from the guide that was broken

first we did a dump pg_dump -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -d dealh3bcbmv4hg -U udajirh8klodnn -p 5432 -W -F t > zedtopialatest.dump then we would do a list and edit: pg_restore -l zedtopialatest.dump and then sed -e '/EVENT TRIGGER/ s/./;&/' unfiltered.list > filtered.list finally pg_restore -U postgres -h gondola.proxy.rlwy.net -p 40138 -W -F t -d railway --use-list filtered.list --no-owner zedtopialatest.dump


azeemh
PROOP

a year ago

the problem is the last time this was tried it didn't load all the comments or primary keys


a year ago

Ok, here's my last try.

Ur gonna delete that postgresql instance and spin up a new one.

You aren't going to do a db:migrate until we are done with the restore

You gonna dump everything except for the Heroku exclusive internal schemas, according to the Heroku docs here (https://devcenter.heroku.com/articles/heroku-postgres-import-export) on the Manual Dump section.

pg_dump -Fp --no-acl --no-owner -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > latest.dump

You gonna dump it into the railway db.

pg_restore --verbose --clean --no-acl --no-owner -h TCP_PROXY_HERE -p PROXY_PORT_HERE -U postgres -d railway -W latest.dump

What's different from last time? Last time we used the --data-only flag.


azeemh
PROOP

a year ago

ok. setup a new database


azeemh
PROOP

a year ago

running the dump command copied and pasted from 1


a year ago

Command 2 will probably throw some warnings that can be safely ignored


azeemh
PROOP

a year ago

i ran command 1 no issues


a year ago

amazing.


azeemh
PROOP

a year ago

command 2 failed


azeemh
PROOP

a year ago

gondola.proxy.rlwy.net:13146


azeemh
PROOP

a year ago

is my proxy


a year ago

what error?


azeemh
PROOP

a year ago

so i ran #2 as pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W latest.dump


azeemh
PROOP

a year ago

Password: 
azeem@x9:~/alphazed$ pg_restore --verbose --clean --no-acl --no-owner -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W latest.dump
pg_restore: error: input file appears to be a text format dump. Please use psql.
azeem@x9:~/alphazed$ 

azeemh
PROOP

a year ago

IT FAILED BRO


azeemh
PROOP

a year ago

pg_restore: error: input file appears to be a text format dump. Please use psql.```


a year ago

psql -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W -f latest.dump


azeemh
PROOP

a year ago

it ran and didn't have any errors in the response,

1381799472052568174


azeemh
PROOP

a year ago

let's check if the data loaded


azeemh
PROOP

a year ago

THE DATA LOADED and the endpoints where i got errors prior seem to load. I haven't run into any errors yet!


azeemh
PROOP

a year ago

I def wanna buy you a drink or a beer or whatever your thing is.


a year ago

Ok! You will probably notice some extra tables there


azeemh
PROOP

a year ago

bless up thank you


a year ago

We imported the heroku internal tables there too


a year ago

We imported everything, actually


azeemh
PROOP

a year ago

i'll go thru it all but i think thats cool


azeemh
PROOP

a year ago

everything works


a year ago

I'm happy we got it working! Sorry if this was a bit confusing.


a year ago

At least we did it.


a year ago

<:kekw:788259314607325204>


azeemh
PROOP

a year ago

i'll put it thru it's paces and update but for today this was excellent


azeemh
PROOP

a year ago

YES this is a victory and you deserve a thanks


azeemh
PROOP

a year ago

also a raise for putting up with all the bs


a year ago

no worries


azeemh
PROOP

a year ago

thanks so much Medim


a year ago

now it's up to you to delete the extra tables and triggers heroku had


a year ago

and run that db:migrate


a year ago

(preferrably on your pre-deploy command or together with your start command, not in ssh)


azeemh
PROOP

a year ago

is there a way to say run db:migrate post deploy


azeemh
PROOP

a year ago

because in ruby on rails the code needs to exist and the tables must be initalized for me to add a new field or row, modify tables etc



azeemh
PROOP

a year ago

iight. this is a post deploy thing. i guess the guys who designed the platform dont get it


azeemh
PROOP

a year ago

but thanks for helping me


azeemh
PROOP

a year ago

i really appreciate everything you did


a year ago

then use it on your start command


azeemh
PROOP

a year ago

solution for anyone else:

start with a fresh postgres instance no tables no migrations nothing. dump your old one (my old creds you can replace with your own) and then use the psql to restore

  1. pg_dump -Fp --no-acl --no-owner -h cc6tobl4vo5pjf.cluster-czrs8kj4isg7.us-east-1.rds.amazonaws.com -p 5432 -U udajirh8klodnn -d dealh3bcbmv4hg -W > latest.dump

2. psql -h gondola.proxy.rlwy.net -p 13146 -U postgres -d railway -W -f latest.dump


Status changed to Solved chandrika about 1 year ago


a year ago

!s


Welcome!

Sign in to your Railway account to join the conversation.

Loading...