2 years ago
I know that MongoDB Replica Sets are not officially supported. However, with the Railway current capabilities in theory could be approached.
I am following the official steps and I feel to be stuck on first steps and I would like to understand if there is anything I can do alternatively.
With the start command support for docker, you can pass "--replSet rs0" to the mongod command. However, you need to enforce security with a key for the replicas communication and shared between three minimum required mongo replicas.
I was trying to store the key into an env variable, and then tweak the start command of the service with something like this:
[docker-entrypoint.sh](docker-entrypoint.sh) mongod --replSet rs0 --keyFile <(echo "${AUTH_SECRET}" | base64) --ipv6 --bind_ip ::,0.0.0.0
However, I think this is not working and I think the reason is that start command can't execute " <(echo ${VAR})" "$(echo ${VAR})" dynamic evaluations.
Am I correct on this limitation? Is there any chance to ask for support dynamic evals on start commands?
Alternatively, could you think in an alternative way to achieve this?
Of course, I was thinking in possibly create a forked mongo db image and include a way to fetch securely the secret, generate the file and be ready to link it on the start command just by the route. But I was wondering if there is an even easier way to achive this without going out Railsway context.
Thank you
0 Replies
2 years ago
(also use triple backticks for multi line code blocks)
I never tried to make one. I will check out the docs to see how. Thank you 😁
2 years ago
this would be a great addition if you could make it work in template form!
the only thing I am not sure how to do it is to automatically run the rs.initiate command only once
and after all nodes are created. Maybe templates are not ready for this use case
2 years ago
have a service in the template that does that initialisation
2 years ago
I'm confident you could make it work after familiarising yourself with the template creation process
2 years ago
Let us know if there’s something you’re missing! Init scripts are something I’ve thought about lightly but haven’t found a great solution to
2 years ago
i second init scripts, ive had a few uses for them, ended up sticking it in a start command
I have a question, do you know if shared variables are included on the template? They don't seem to appear once I try to convert a project that has shared envs to a template
2 years ago
they arent
2 years ago
but you can set a variable on one service and then reference it from other services
2 years ago
a real example of this would be my typebot template
where the builder and viewer both need a whole lot of the same variables, i set most of the variables on the builder service and reference them on the viewer service
2 years ago
sounds good!
i set most of the variables on the builder service
When you say this, how the communication between the service and railway happens to be able to set the envs on the service run? Do I have to import an API for that?
2 years ago
nope, please have a read of the docs page i linked and look through the variables on the builder and viewer service of the typebot template
I implemented a project where I created the three mongo nodes and a service that initializes those nodes using a github repository. I created the template from the project. Several questions:
The service is a Github repository with a main.sh command to wait the nodes and do the initialization once they ready. I am asked to clone the project whenever I deploy from the template. I guess I should use a docker image instead to avoid the clone and be more staightforward config?
I can't seem to generate properly the secrets with
${{ secret(128, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789+/') }}
as the docs suggest to be possible. When the project is deployed the value the raw value is utilized without password evaluation. What am I doing wrong?Is it possible to parametrize or allow to edit the db source (mongo docker image version) when deploy a project from the template? It would be good for the user to be able to select the proper DB version they expect to use.
2 years ago
if you could make it work with all docker images that would be highly preferable
maybe it doesn't support 128 length?
not yet unfortunately
I am trying to achieve the same thing. The service keeps failing with the following error "MongoDB - ReplicaSet - Failed to refresh key cache"
Yes, I got that as well on my new project to generate the template. The only difference has been that now I use the latest mongo 7 version. Not sure if this is related or what. I didn't have more time to reattempt on this. I am also thinking that it may be related with some kind of race condition on setting up the nodes and the rs initialization process. Not sure, still testing.
Regarding the template, it is good to be able to launch automatically all nodes and the initialization process, but I am still getting wrong the secret on using this on the secret env the way is mentioned on the docs ${{ secret(32, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789+/') }}
.
Hello, I'm new to this. I don't really understand the third step, the configuration of the primary node. Where should this be put? Should this be on the Redis? I am missing something. Context: Trying to use Prisma on this db, and it is requiring me a replica set.
a year ago
heres the docs on shared variables
Ah sorry, i miscounted :)) Fourth step the " rs.initiate" one. ( I didn't notice the duplicate 3. )
a year ago
welp i have no idea about that one then, sorry
a year ago
youd have to wait for an answer from nacho
That needs to be run using mongosh connecting to your primary instance. This is the step that ensures proper configuration of replica set hosts.
https://www.mongodb.com/docs/manual/reference/method/rs.initiate/
a year ago
Can this method be applied to a running database that has data in it?
In the docs they explain how to do it, https://www.mongodb.com/docs/manual/tutorial/convert-standalone-to-replica-set/
a year ago
I'll try this, My project has gone big and I need changeStreams to monitor the collections for invalid activity and error correction.
How did you connect to the primary node with mongosh? When I use the startup command you posted, I can't connect to the database through a public network. I can't find how to connect to the primary node through the private network to execute the rs.initiate() command.
Also, all three of the nodes spam this Aggregate command executor error
in the console, does your setup do that as well?
To connect using mongosh you normally need to do:
mongosh "mongodb://user:pwd@host:port/database" --authenticationDatabase admin
In the case of Railway, yes, you can't connect using the private IP. But some months back a way to expose a public IP pointing to TCP service (eg Mongo) was implemented. In fact, as part of adding a new mongo node that IP is exposed automatically with a port, if don't you have it you can do it yourself. In the picture I attach, you can check where you can find the public address to use it for the mongosh command.
Aggregate command executor error
This error will go away once the replica setup is configured properly.
a year ago
Won't we need a vpn to connect to the private network?
a year ago
This isn't working. MongoServerSelectionError: connection to 35.213.165.89:47424 closed
a year ago
I am getting the "Aggregate command executor error" and not able to connect
a year ago
plus in the rs.initialize my hostname doesn't change to mongo-two or three
a year ago
Should I change the service variable HOST to this?
a year ago
Alright I was able to follow through this somehow
a year ago
This is the last log that I get in my deploy logs
a year ago
Received first command on ingress connection since session start or auth handshake
a year ago
Is this a good thing?
I just see connection accepted/ended and auth succeeding logs on my end. Have you already reached that?
I have to say that I attempted to replicate my initial set up to make a template, but I run into many issues, the fact that for making the template all nodes are initialized, configured and served at the same time likely being the reason of all the issues. And I didn't have time to see them and clarify why it fails.
Maybe I was lucky on my initial attempt, but it worked straight with the steps described above.
I want to provide also more context about my set up: I use Mongo 6.0.11 version in my app via the docker official image tag.
In the template thing I tried with latest 7.x version where I got the problems. Not sure if related but just mentioning.
a year ago
Lucky you, I lost some of my data on my first attempt.
Happy to see that Railway just delivered the official support of MongoDB Replica Set, https://railway.app/template/ha-mongo
I have been using MongoDB replicas for long already, and this template should overcome many of the problems when I tried to automate. 🥳
9 months ago
awesome, we would love to hear your feedback!