4 months ago
This is for my stage instances ONLY.
For the stylipedia-producer, I cannot auth to the replica set. The logs shows its failing and I have tried several things. The AI keeps doing the same thing. I am not sure if its a password issue. I just had to create a whole new replica set and the user is admin for some reason instead of mongo like in my developer instance? I have no idea why things are failing, it its a password issue, then we need to we told we have to save database or hard drive pass words. Since I had no data I needed to save, I wanted this to be a clean start.
2 Replies
4 months ago
This thread has been opened as a public bounty so the community can help solve it. The thread and any further activity are now visible to everyone.
Status changed to Open Railway • 4 months ago
4 months ago
Thanks, but I did that several times and none of it has worked. I just deleted it and tried to sync from my development enviorment, but now that is stuck and the deployments are broken. This happened only after trying to give the replica set volumes. Now, even when I go to another browser, the purple deploy button shows and never works.
2 months ago
There are two separate problems here, and the first one explains why nothing you tried made any difference.
Why changing the password never worked
MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD are only read when the data directory is empty. That is standard MongoDB image behaviour, not a Railway quirk. Once the replica set has initialised once, editing those variables and redeploying changes nothing at all: the image sees an existing data directory, skips initialisation entirely, and the old credentials stay in force. So your app keeps failing auth with what looks like the correct password, you change it again, redeploy again, and get the identical failure. That matches "the AI keeps doing the same thing" exactly.
Two ways out. Clean start, which is simplest since you have no data to keep: delete the volume itself, not just the service, because a new empty volume lets initialisation run again and the variables finally take effect. Or keep the data: connect with the credentials that actually exist and run db.changeUserPassword("admin", "newpassword") in the admin database.
This is also why adding volumes made things worse rather than better. Mounting a volume onto a path that already had data hides the existing data directory, so you end up in a half-initialised state that depends on whether the mount landed on /data/db or somewhere else.
Why the user is admin and not mongo
Not an error, and not something that broke. The MongoDB Replica Set template and the single-instance MongoDB template are different templates with different defaults for the root username. Nothing carried over from your developer instance, and nothing needs fixing here. Rather than writing credentials by hand, use the service's own generated MONGO_URL variable reference so username, password and hostname always match whatever was actually provisioned.
Two connection-string details that cause authentication failed with correct credentials
First, authSource=admin. The root user lives in the admin database. If your URI names a different default database and omits authSource, the driver authenticates against that database instead, the user is not there, and you get a failure indistinguishable from a wrong password.
Second, percent-encode the password. Generated passwords often contain @, /, : or %, all of which terminate or corrupt fields in a MongoDB URI. This produces auth failures that survive every password reset.
The replica set gotcha, if you connect from outside Railway
A replica set client connects to the seed host, asks for the topology, and is told the member hostnames, which on Railway are the internal railway.internal names. From outside the project those do not resolve, so the driver discovers the set and then cannot reach any member, even though the seed connection through the TCP proxy succeeded. From a service inside the same project and environment, use the internal hostname with replicaSet=rs0 and authSource=admin. From your laptop or any external tool, add directConnection=true to skip topology discovery and talk only to the host you dialled; you lose automatic failover, which is fine for administration and debugging.
The stuck purple Deploy button
That is a staged changeset that cannot be applied. Deleting a service while other services still reference its variables leaves dangling references, and syncing an environment from development copies references to services that do not exist in staging. You did both. The changeset stays staged and the button does nothing in any browser, because the blocked state is server-side rather than a UI bug.
Discard the staged changes instead of trying to apply them, then check every remaining service in staging for variables still pointing at the deleted replica set, and remove or repoint them before staging anything new.
Suggested order
- Discard the staged changes so the environment can deploy at all.
- Remove variable references to the deleted service.
- Delete the old volume for a genuinely clean start.
- Redeploy the replica set and let it initialise with fresh credentials.
- Point the app at the generated MONGO_URL reference, adding authSource=admin, plus directConnection=true if you are connecting from outside Railway.