a month ago
Hi! We're setting up a self-hosted monitoring stack (Prometheus + Grafana) inside the same Railway project as our API, scraping over the private network to avoid egress.
Setup:
- NestJS API service running 4 replicas in production, exposing GET /metrics (prom-client)
- Prometheus deployed as a separate service in the same project/environment
- Scrape target: ${{api.RAILWAY_PRIVATE_DOMAIN}}:3000 over private networking
The problem: with 4 replicas behind a single private domain, a plain static scrape target would hit a different replica on each scrape interval, mixing per-process metrics (heap, GC, event loop, counters) into one useless series.
Questions:
- When a service has N replicas, does a DNS lookup on its RAILWAY_PRIVATE_DOMAIN return one AAAA record per replica, or a single record that's load-balanced at connection time?
- If it returns all replicas: is Prometheus dns_sd_configs (type AAAA) a supported/recommended pattern on Railway? Anything to know about DNS TTL / propagation after redeploys or scaling events, so targets refresh correctly?
- If it does NOT return all replicas: is there any supported way to enumerate or address individual replicas over the private network (per-replica hostnames, headless-service-style DNS, etc.)?
- Is there an official recommended pattern for scraping per-replica metrics on Railway?
Thanks! Happy to share our final setup back with the community once it works.
2 Replies
Status changed to Awaiting Railway Response Railway • about 1 month ago
a month ago
A DNS lookup on the service's private domain returns one record per replica, not a single load-balanced address. There is no per-query record count limit, so all 4 replicas will appear. For newer environments (created after October 16, 2025), you get both A and AAAA records; legacy environments return AAAA only.
Prometheus dns_sd_configs with the appropriate record type is the right pattern here and should work as expected. DNS records update as replicas come and go during redeploys or scaling changes, so Prometheus will pick up the new set on its next re-resolution interval.
Status changed to Awaiting User Response Railway • about 1 month ago
sam-a
A DNS lookup on the service's private domain returns one record per replica, not a single load-balanced address. There is no per-query record count limit, so all 4 replicas will appear. For newer environments (created after October 16, 2025), you get both A and AAAA records; legacy environments return AAAA only. Prometheus `dns_sd_configs` with the appropriate record type is the right pattern here and should work as expected. DNS records update as replicas come and go during redeploys or scaling changes, so Prometheus will pick up the new set on its next re-resolution interval.
a month ago
Thanks a lot Sam!
Status changed to Awaiting Railway Response Railway • about 1 month ago
Status changed to Solved Railway • about 1 month ago