Presigned GET returns NoSuchKey for objects that demonstrably exist
peterje
PROOP

6 days ago

This bucket has worked reliably for months; presigned-GET NoSuchKey failures only started appearing today, with no related change on our side. If anything changed in Tigris/Railway storage routing, regions, or presigning for this bucket/project recently, that's very likely the cause.

Bucket: exports-afivsou4hkjnbu0wp

Endpoint: https://t3.storageapi.dev · Region config: auto (presigned URLs are being signed for region sjc)

Access: S3-compatible via @aws-sdk/client-s3 v3.1077 (PutObject) + @aws-sdk/s3-request-presigner (GET)

Summary

We generate a PDF, PutObject it (returns HTTP 200), then hand a presigned GET URL to a client. As of today, fetching that presigned GET URL intermittently — and for some objects, persistently — returns NoSuchKey (404), even though the object is present. A HEAD/GET on the same key from our side succeeds and returns valid data, so the object was durably written and is not deleted (we use no lifecycle rules — and understand Railway buckets don't support them).

Concrete failing request

  • RequestId: 1782923034276471536
  • Key: pdf/1782923033854-89a69dc5-03ba-4097-87fe-5cd4873575ce.pdf
  • Response: <Error><Code>NoSuchKey</Code>...</Error>

Proof the object exists

  • Key: pdf/1782924540774-22ae4f7e-fc2b-4852-af60-9866574ad835.pdf
  • HEAD result: ETag "55e37cede7b4950c419e94eec44be5f3", Size 24096, Content-Type: application/pdf, LastModified: 2026-07-01T16:49:00Z
  • The 404 occurred at the same second as LastModified (read issued right after write). A correctly-signed GET now returns 200 from every region we tried (auto, sjc, fra, us-east-1).

What we've ruled out on our side

  • Not deletion/TTL (object present; no lifecycle).
  • Not a failed write (PutObject 200; object present with valid ETag/size).
  • Not URL style / region-in-signature (GET 200 from all regions).
  • Not SDK checksum behavior or Range requests (both 200/206 directly).
  • No corresponding deploy or code change on our side today.

Happy to provide fresh RequestIds/keys from ongoing occurrences.

Awaiting User Response

3 Replies

Status changed to Awaiting Railway Response Railway 6 days ago


sam-a
EMPLOYEE

6 days ago

Thanks for the detailed writeup, the ruled-out list saves us a lot of back and forth. You're right that this isn't your SDK or signing. So far it reads like a transient read-after-write hiccup on the storage side rather than a persistent fault, and the objects you flagged are now serving 200s from every region.

Before we dig into the storage layer, we want to confirm it's still happening and not a one-off blip from earlier today. Are you still seeing NoSuchKey on freshly written objects right now? If so, send a few fresh RequestId and key pairs from failures that persist (not ones that later recovered), and we'll correlate them against the backend.

Railway Team


Status changed to Awaiting User Response Railway 6 days ago


adamtowerz
PRO

6 days ago

https://status.tigrisdata.com/ has finally acknowledged an incident. Railway should probably do the same.


Status changed to Awaiting Railway Response Railway 6 days ago


5 days ago

The Tigris incident has been resolved. May I ask if you are still seeing this issue?


Status changed to Awaiting User Response Railway 5 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...