We can see your futgenie-midas service and have been looking into the request path. There are no active or recent incidents affecting runtime latency in the timeframe you described (2026-06-02 21:00 UTC onward), and no throttling or capacity restrictions on your project. Your application logs and internal dependencies all look healthy from our side as well, consistent with what you found.

We are actively investigating the proxy and ingress layer for your service to determine what is introducing the delay before your handlers run. Could you confirm which region this service is deployed to? That will help us narrow the scope on our networking infrastructure. Also, is the issue still ongoing as of now, or has it resolved?

Hi, of course. It is deployed in EU West (Amsterdam, Netherlands). The issue is still ongoing. It isn't necessarily affecting users at the moment, but the metrics are still showing dramatically increased response times. Thank you!

I hope it's okay for me to tag on to this - I am having a similar issue with my harmonius-commitment project, but I am not tech savvy enough to provide the same level of data. I do not see any errors in the logs but my "response time" graph is peaking: https://i.imgur.com/DvN53gH.png. I did not use the app the last few days so I don't know if this would have occurred at the same time as OP's issues. However, it was working without issue before and without me having changed anything, now is basically unusable. Thanks for your time!

Just to add to this, I made a copy of our soon-to-be-live environment for a staging environment and after updating variables and settings, response times for straightforward activities (writing settings from frontend to database) is taking an excessively long time that I had to add in a 20,000 ms timeout just to keep it somewhat responsive. I thought maybe it was config settings until I saw this post.

Just to affirm:

Server logs show the handler finishing in 5ms; the 20s is before the request lands

PG 2–5ms, Redis connected, CPU ~0

Near-zero CPU rules out event-loop blocking

Persisted across our redeploys

Staging affected; production fine (same project)

The conclusion above matches the same conclusion we reached: inbound request path - proxy/edge/ingress/container network namespace is likely the cause, not code

It's worth noting we're on us-east4 (US East, Virginia, USA)

I believe this was related to an issue we saw today. Is it resolved?

Hi Jake. Everything seems to have gone back to normal now, thanks.

To follow up on this, the issue seems to have come back now. Response times back up to 30s

Tagging along, but same thing here. We had the very same issue, and it's back.

Hello, same for me here

the same for me issues

this started with us before one hour ago

the response time is getting worst

facing same issue

We're currently experiencing significant latency issues on our Singapore deployment, and this is impacting our business operations and user experience.

We observed a similar issue yesterday around 23:17 UTC. It appeared to improve for a short period afterward, possibly due to an infrastructure-side fix, but the problem has now returned and response times are once again very high.

Could you please investigate whether there are any ongoing issues affecting the Singapore region, networking, or underlying infrastructure? The recurring nature of this issue is causing disruption to our customers and daily operations.

We would appreciate any updates or assistance in resolving this as soon as possible.

Thank you.

Not resolved

any updates on this ticket ?

no update till now

The same issue started again

We're looking at it now with our engineers, will update shortly.

This issue also occurred yesterday around 23:17 UTC. After the Railway team pushed a fix, the service worked fine for around 5-6 hours, but then the problem returned.

Could you please confirm how long the current investigation/fix may take? This is causing serious disruption to our business operations, so we’d really appreciate a proper permanent fix rather than a temporary recovery.

Some of us are literally loosing users here, and the incident has been around for two days (as reported in different conversations).

Do you have an ETA, or at least could you open a new incident we can follow ?

Thank you.

Our engineers pushed what we believe to be a fix, can you confirm load times now?

same result

We still have the issue.

Hello, could you please run a traceroute or MTR to 69.46.46.46 and paste the output after running it for ~20 seconds?

It'd also be very helpful if you visited https://railway.com/.railway/cdn-trace and let me know what pop field is equal to, and where in the world you are located. Thank you.

okay

Hello, may I ask which domain or service this is occuring on? Is this happening to all of your users or only a subset?

v=hikari:0.1.2:1305a1583dc517b7e2603638b5710b21

r=production-hikari-dpsgp-1

pop=sin1

node=tr00

ts=1780585369197

hint=Railway is coming to a city near you

khizertikkadc.up.railway.app

Right now it's khizertikkadc.up.railway.app that's acting up, though earlier the issue was on khizertikkahal1.up.railway.app. Give me a minute — I'll check both and get back to you with the latest metrics

Okay, and may I ask if are you based near Singapore geographically?

Hi.

As far as we can tell, it happens to those domains at least:

platform.prod.bordogames.com

platform.devel.bordogames.com

p00ls-games-backend.prod.p00ls.io

p00ls-games-backend.devel.p00ls.io

bgg.prod.p00ls.io

bgg.devel.p00ls.io

Can't be 100% sure, but it seems to affect users in EU zone, not US (didn't test for Asia).

No, I'm in Pakistan — that's why I was looking at moving the Railway region closer to this side (Asia) instead of Europe.

Okay, would appreciate if you (or one of your effected users) could also:

• Run a traceroute or MTR to 69.46.46.46 for ~20 seconds and paste the response
• Go to https://railway.com/.railway/cdn-trace and let me know what the pop field is equal to
• Let me know where in the world you are located

Thank you!

okay

C:\Users\HP>tracert -d -h 20 -w 1000 69.46.46.46

Tracing route to 69.46.46.46 over a maximum of 20 hops

1 1 ms 1 ms 1 ms 192.168.1.1

2 7 ms 33 ms 18 ms 202.163.75.253

3 23 ms 4 ms 4 ms 10.15.224.134

4 49 ms 34 ms 99 ms 10.15.227.17

5 39 ms 48 ms 27 ms 10.15.19.229

6 44 ms 89 ms 59 ms 192.168.248.206

7 * 530 ms 44 ms 192.168.4.61

8 * * * Request timed out.

9 135 ms 129 ms 109 ms 27.111.230.50

10 115 ms 104 ms * 79.127.194.187

11 117 ms 103 ms 106 ms 69.46.46.46

Trace complete. location : pakistan punjab ,

https://railway.com/.railway/cdn-trace

v=hikari:0.1.2:1305a1583dc517b7e2603638b5710b21

r=production-hikari-dpsgp-2

pop=sin1

node=nzn2

ts=1780586150710

hint=Railway is coming to a city near you

Ok here you go : 

MTR output:

HOST: Loss% Snt Last Avg Best Wrst StDev

1.|-- 192.168.8.1 0.0% 10 2.2 4.4 2.2 9.2 2.7

2.|-- bordeaux1.regus.net 0.0% 10 3.7 5.7 2.5 15.8 3.9

3.|-- 93.40-14-84.ripe.coltfran 0.0% 10 16.7 18.1 14.5 22.3 2.6

4.|-- 193.114.160.163 70.0% 10 20.5 33.8 20.5 47.6 13.5

5.|-- prs-b3-link.ip.twelve99.n 0.0% 10 17.0 18.0 15.2 20.4 1.4

6.|-- datacamp-ic-382929.ip.twe 0.0% 10 14.6 17.1 14.4 20.3 1.9

7.|-- unn-185-156-45-116.cdn77. 20.0% 10 22.3 25.5 22.0 28.6 2.7

8.|-- vl221.par-itx5-dist-2.cdn 0.0% 10 24.0 25.3 21.7 28.2 1.7

9.|-- 69.46.46.46 0.0% 10 22.1 24.6 22.1 27.9 1.8

pop: cdg1

And as you may have guess : France

Are you also seeing latency to your application or has it only been reported by your users?

Yes, I'm experiencing the same latencies.

(It's worth what it's worth, but using a VPN to connect from the US solves 99% of the latency issue).

Yes, I'm seeing it directly too — not just user reports. Railway's own metrics show the spikes: p50 is normal (~350–480ms) but p99 jumps to 30s, with individual requests hitting 60–90s. My staff using the POS also experience it live — orders hang and they have to refresh and retry. So it's confirmed on both the metrics side and in real usage, especially during busy hours.

.

Attached is the 6-hour Response Time metric for khizertikkadc.up.railway.app As you can see, the median (p50) stays low/normal the whole time the app itself is fast — but p99 keeps spiking to 20–30 seconds at random points. It works fine for a while, then breaks again.

This is a live, busy restaurant POS system. A little while ago your team pushed a fix and it stabilized for a bit, but after that it started breaking again — you can see the spikes return later in the chart, during our busy dinner hours.

Update: Just now I also saw another 30-second spike on khizertikkahall1.up.railway.app (our Hall branch) — so it's happening on this service too, not just DC

This software has never given us this kind of trouble before — it's been running fine, but for the last two days it's been seriously struggling. Earlier the spikes were on two of our branches; now it's mainly this one (DC) left.

Please help us get this fixed — it's causing real problems for our customers and staff during peak hours. This is urgent.

Also, just to clarify, given the different open conversations on the matter, what we observe from traces is that headers are received, but then the body can take up to 30sc to arrive.

Our metrics show that our code is not blocked waiting for IO, or doing some heavy work in main thread (we have node and kotlin backends).

A typical trace really looks like «Parsing header» -> Nothing for 30sc -> «Processing the body»

Thank you, that is useful and we are still investigating the root cause.

we are seeing exactly the same thing. On our Node backends, the app's own service time is tiny (~86ms in our logs), but the full response can take up to 30 seconds because the body arrives long after the headers. Our traces show the same shape: headers parsed immediately → ~30s gap → body received. Our code is not blocked on IO or doing heavy main thread work either.

The fact that this is happening across both Node and Kotlin backends, for multiple customers, strongly suggests this is a Railway edge/proxy networking issue, not application code. The body bytes seem to be held/dripped somewhere in the proxy chain between the edge and the app.

@railway-team — can you please investigate the proxy/networking layer in the Singapore (sin1) region? This has been ongoing for two days and is seriously impacting our live restaurant POS during peak hours.

Hello all, we have made some changes to HTTP/2 behavior on our edge which should improve the experience for users with high RTT to your application. Please let me know if you're seeing any improvements! 🙏

Hello, thanks for the update. Looking at the last 1 hour of metrics, I haven’t seen any response time spikes above 2 seconds, and the 30+ second spikes we were experiencing earlier have not occurred during this period. From my side, it looks like the service has improved significantly. Hopefully it continues performing like this going forward. I’ll keep monitoring and let you know if anything changes. 🙏

Thank you very much for the update! I am sorry you ran into this, please let me know if you see anything change.

Thank you for looking into it so quickly.

So far, the service has been much more stable. The 20–30 second response time spikes we were seeing earlier have not occurred in the last hour, and response times have remained below 2 seconds.

We’ll continue monitoring, and I’ll definitely let you know if anything changes. Thanks again for your help and for making these improvements.

Still same for us

v=hikari:0.1.2:1305a1583dc517b7e2603638b5710b21

r=production-hikari-dpwaw-1

pop=waw1

node=j60f

ts=1780599915961

hint=Railway is coming to a city near you

Hey, it seems things are getting better, but as you can see, we still have some requests off the charts.

It seems to be on and off. For an hour or so it's back to normal and then it goes back to long response times.

Same here. Issue is back ☹️

Update for everyone on this thread — and apologies for the disruption over the past couple of days.

The root cause was an HTTP/2 behavior issue on our edge that introduced large delays in the inbound request path for users with higher round-trip time to their application (the "headers arrive, then a ~30s gap before the body" pattern several of you captured exactly). Our engineers shipped a fix to that edge behavior, and it's now confirmed holding on our side — the 30s+ response-time spikes should be gone across regions.

No action is needed from you. If you applied any temporary workarounds (e.g. routing around via VPN), you can revert them.

Thank you for the genuinely excellent diagnostics — the traceroutes, cdn-trace pop values, and the header/body timing traces made this much faster to pin down. Please keep an eye on your response-time metrics and let us know right away if you see anything else; we'll keep monitoring on our end as well.