a day ago
Summary:
Our production service (fortunecat-prod, www.fortunecat.co.kr) is experiencing intermittent severe latency spikes on simple, fast endpoints. Our internal handler processing consistently completes in milliseconds, but a large portion of requests take 1-9+ seconds, with no correlation to load, code complexity, or database query time. This is not reproducible locally.
What we've already tried on our side (none of which resolved the issue):
- Increased Node.js keepAliveTimeout to 75s, suspecting a race with edge proxy idle connection reuse.
- Disabled sleepApplication (serverless sleep) in railway.json to rule out cold starts.
- Added memory diagnostics to /api/health to check for a memory leak / OOM crash-loop, after an unexplained container restart at 00:38 KST with no corresponding deployment on our end.
- Fixed application-level bugs (unique constraint errors, chat timeout, token verification latency) - unrelated to this issue but ruled out as contributing factors.
Despite all of this, the stalls persist. The same code path, same endpoint, same load, produces wildly different response times.
Example from HTTP logs (Jul 5-6, 2026):
- GET /api/saju: 9-20ms typical, but 345ms / 625ms / 915ms / 1s / 2s observed
- GET /api/users/me: 21-33ms typical, but 2s / 4s observed
- GET /api/users/me/persona: 4-36ms typical, but 4s observed
- GET /api/gunghap: 15-30ms typical, but 7s observed
- 3 consecutive 500 errors on POST /api/unse/daily (now fixed, but may be related)
Our questions for your team:
- What is causing this intermittent latency? We've ruled out cold starts, application code, and database performance on our end - what else could explain this pattern?
- Is there a known issue with edge proxy / keepalive behavior on Railway that could cause this?
- Is there anything unusual in your infrastructure logs for our service during these stall windows (CPU/memory throttling, network issues, silent restarts)?
- What would you recommend we do next to resolve this?
Environment:
- Service: fortunecat-prod
- Domain: fortunecat.co.kr
- Region: Southeast Asia (Singapore)
- Replicas: 1
- Port: 8080
- Builder: Dockerfile
- Serverless sleep: Disabled
- Framework: Next.js (next start)
2 Replies
a day ago
This thread has been opened as a public bounty so the community can help solve it. The thread and any further activity are now visible to everyone.
Status changed to Open Railway • 1 day ago
a day ago
I tested this on my local PC server (hosted in Korea) and the response times are consistently fast. Here are the results from the network logs:
• persona – 200 OK, ~1.5 kB, avg 120–327 ms
• gunghap – 200 OK, ~34.2 kB, avg 357–713 ms
• me – 200 OK, ~1.4 kB, avg 114–130 ms
• inquiries – 200 OK, ~1.0 kB, avg 120–230 ms
• saju – 200 OK, ~3.7 kB, 258 msTotal: 46 requests / 427 kB transferred
Even with the server located in Korea, response speeds are holding up well. The heaviest endpoint is gunghap at 34.2 kB, but it's still within acceptable range. Overall performance looks stable. So I believe there may be an issue on the railway server side.
a day ago
Here are additional results from a Singapore-based server:
• Some requests showed intermittent slowdowns:
▪ persona – 2.60 s
▪ me – 1.44 s
▪ inquiries – 845 ms
• Other requests were fast:
▪ persona – 150–187 ms▪ gunghap – 159–266 ms
▪ me – 166–195 ms▪ inquiries – 136–137 ms
No code changes were made and no redeployment was done. The slowdowns occurred intermittently on their own, which strongly suggests this is an infrastructure-level issue on the server side, not a cold start.
Additionally, I've noticed a consistent pattern — performance is fine during the day, but after 9 PM (KST), the slowdowns occur again. This time-based pattern suggests possible server load or resource contention during peak hours.