cff55b2e-ba77-4df5-9e61-cb0fe7bbead4

can you explain what "dns clustering with the internal network" means to you?

Sure think, so Erlang has a feature called clustering where if you run multiple nodes of your application, and they can discover each other, they connect and are able to do things like transfer jobs, share pubsub etc.

But to get that setup you need some way for nodes to discover each other. Ways to do that are:

• a static list (obviously not railway compatible)

• An orchestrator like k8s that can provide a list of nodes, example: https://www.gigalixir.com/docs/cluster

• Some way to broadcast/gossip discovery, in my case I tried dns resolution, since we have the app.internal names we can use to communicate in the app pricate network, an example: https://fly.io/docs/elixir/the-basics/clustering/

why wouldn't a static list of internal domains work?

static list would be ips

why wouldn't domains work?

I am not sure? It is very difficult to accertain from logs alone what is going wrong in each test

But if there is a way to get the internal ipv6 address that might be another thing I can try?

Like for example the fly.io example has

export DNS_CLUSTER_QUERY="${FLY_APP_NAME}.internal"
export RELEASE_DISTRIBUTION="name"
export RELEASE_NODE="${FLY_APP_NAME}-${FLY_IMAGE_REF##*-}@${FLY_PRIVATE_IP}"

I did find mentions of a replica_id envvariable, but using that as the name for each node doesn't seem to work

yes of course you simply do a AAAA DNS lookup for a given private domain, but respectively it sounds like you may not fully understand how the erland side of things works, that means implementing it on railway is going to be very challenging, especially since I don't know how the erlang side of things works either

I mean I could try to get an interactive console on my nodes and try to figure out how the railway internal network is set up but I thought I'd ask if anyone has already dealt with this?

And things like undocumented env variables that you can only find through discord 😄 I was hoping to maybe figure out a way to get each replicas ip through an env variable, as that would help tremedously too

the only way would be to do a dns lookup

but if it helps, the private network is setup like a LAN network, no firewalls, proxys or anything, just helpful dns names

Okay that is good to know! That sounds like if I figure out a way to get each nodes own ipv6 I can make it work

Keep in mind that the ip address of a given will change every time that service is redeployed

getting the ip address is the easy part lol

Redeploy is not a problem, the clustering is handling all that automatically, I just need to basically have every node know its own ip first and then its all good 😄 might just have to dump the env to see if I can find what I need there

again, do a DNS lookup

I'm not sure what you expect to find in the environment variables

https://utilities.up.railway.app/dns-lookup?value=utilities.railway.internal&type=aaaa