a month ago
Hello Railway Support Team,
We are experiencing persistent outbound connectivity failures from Railway to Brazil’s official National Public Procurement Portal (PNCP).
Project: friendly-stillness
Environment: production
Primary service: licitasaas-worker
Also affected: licitasaas
Production region: europe-west4-drams3a
Destination: pncp.gov.br
Resolved IP: 189.9.176.245
Port: 443/TCP
BUSINESS IMPACT
Our application uses the public PNCP API to retrieve procurement notices and documents. The Railway containers cannot establish a TCP connection to PNCP, so document discovery fails before downloads or AI processing begin.
PRODUCTION RESULTS
Both licitasaas and licitasaas-worker report:
TypeError
cause: UND_ERR_CONNECT_TIMEOUT
elapsed time: approximately 10.5 seconds
A raw TCP test, without HTTP or application logic, also fails:
189.9.176.245:443 → timeout after 8 seconds
The worker repeatedly logs:
PNCP_GLOBAL_UNAVAILABLE: 2 consecutive failures; synchronization cycle interrupted
Occurrences were recorded on July 25, 2026 at approximately:
12:40, 12:56, 13:11, 13:26 and 13:41 UTC
CROSS-REGION TEST
We deployed an isolated temporary canary in US East:
us-east4-eqdc4a
The canary had no database, secrets, application logic or AI calls. It only performed DNS, TCP and HTTPS tests.
US East results:
- DNS resolved pncp.gov.br to 189.9.176.245
- TCP port 443 timed out after 8 seconds
- PNCP attachments endpoint: 3/3 connection timeouts
- PNCP search endpoint: 3/3 connection timeouts
- All HTTPS attempts failed with UND_ERR_CONNECT_TIMEOUT after approximately 10.5 seconds
Therefore, the issue is not limited to europe-west4. It affects at least EU West and US East.
EXTERNAL BASELINE
The same tests were performed outside Railway:
- DNS resolved to the same IP
- TCP port 443 connected in approximately 130 ms
- The attachments endpoint returned HTTP 200
- Five documents were returned
- Response time was below one second
This confirms the failure occurs while establishing the TCP connection from Railway, before TLS, HTTP or our application code.
PUBLIC ENDPOINTS TESTED
https://pncp.gov.br/api/pncp/v1/orgaos/07540925000174/compras/2026/25/arquivos
INFRASTRUCTURE HEALTH
All production services remain online. Deployments are successful, CPU and memory usage are low, the database is connected, and other integrations continue operating normally.
REQUEST
Please:
- Verify outbound TCP connectivity to 189.9.176.245:443.
- Check for packet drops, routing issues, firewall rules, ACLs or NAT problems.
- Confirm whether EU West and US East share egress infrastructure that could explain this behavior.
- Provide the outbound IP addresses or CIDR ranges used by our services in both regions.
- Check whether the destination may be blocking Railway egress addresses.
- Confirm whether Railway supports static outbound IPs for this use case.
- Recommend a supported mitigation.
Reproduction:
nc -vz -w 10 189.9.176.245 443
curl -v --connect-timeout 10 "https://pncp.gov.br/api/pncp/v1/orgaos/07540925000174/compras/2026/25/arquivos"
Our primary hypothesis is either a destination-side block affecting Railway outbound addresses or a shared routing/egress issue between Railway and 189.9.176.245.
Thank you.
1 Replies
a month ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • 27 days ago
a month ago
Hey, that's a really well put together report.
The fact that it fails identically from europe-west4 AND us-east4, but connects in ~130ms from your own machine, is basically the answer. Those two regions don't share egress, so a Railway-side routing/firewall issue would have to be a hell of a coincidence. What you're seeing is almost certainly PNCP dropping you.
Two details that point the same way:
- You get a silent SYN timeout, not a connection refused and not a 403. That's a firewall dropping packets, which is what gov.br properties tend to do to traffic they don't like. If Railway were blocking outbound you'd typically see a reject, and they don't filter outbound 443 anyway.
- Brazilian government portals (PNCP, Comprasnet, several others) are well known for blocking foreign cloud ASNs outright. Your working baseline was from a Brazilian or residential IP, right? That's the variable, not Railway.
Cheap way to confirm before anyone digs into infra: curl it from any non-Brazil cloud box you have lying around (DO, AWS, Fly, whatever). If that hangs too, it's confirmed and it has nothing to do with Railway. Then try the same from a São Paulo VPS and watch it return 200 instantly.
On your specific asks: as far as I know Railway doesn't offer static outbound IPs on standard plans, and honestly even if you got one it wouldn't help here since it'd still be a datacenter IP in a range that's likely already blocked. You'd just be swapping one blocked address for another.
The mitigation that actually works is putting an egress hop inside Brazil:
- Cheapest: a small VPS in São Paulo (AWS sa-east-1, Vultr, Magalu, Locaweb) running tinyproxy or squid, or just a tiny Node forwarder if you want to keep it dumb. Then point only pncp.gov.br traffic at it. Since you're on undici (that UND_ERR_CONNECT_TIMEOUT gives it away), ProxyAgent makes this like five lines.
- If you'd rather not run infra: any proxy provider with BR exit nodes (IPRoyal, Oxylabs, Bright Data) works, just more expensive per GB.
Keep the retry/backoff logic you already have, PNCP itself goes down fairly often even from inside Brazil.
It's also worth writing to PNCP asking them to allowlist your egress range, but I wouldn't build the roadmap around them answering.