25 days ago
Our production dashboard's existing photo files are inaccessible. Please investigate the existing bucket's authorization/provisioning and preserve its objects. Do not delete, empty, or recreate the bucket as a workaround.
Diagnostic results from 26 October 2026 (KST):
- Bucket display name: hanip-photos; S3 bucket name: hanip-photos-hiojn7gzdy5n.
- Endpoint: https://t3.storageapi.dev (Singapore).
- Railway bucket metadata reports 74 objects / 7,991,369 bytes. The application has 37 photo records, each referencing an original and thumbnail.
- Direct authenticated GETs for all 37 original object keys returned HTTP 403 AccessDenied; none could be recovered. The database references are intact. We cannot establish whether the underlying objects are intact merely from the metadata count.
- We fetched the current credentials independently through the Railway API and CLI and confirmed they match. Both path-style and virtual-host-style S3 requests still fail.
- A credential reset also fails at the Railway control plane: BAD_USER_INPUT, "We couldn't rotate bucket credentials due to an unexpected error." Trace ID: 5093916392487416872 (earlier occurrence: 361976543922495658).
- The workspace was ACTIVE, not trialing, and no usage limit was configured when checked. We reviewed the similar community report resolved by removing a budget limit, but that does not match our observed configuration.
- The application has been redeployed, and the photo access failure is also reproducible outside the application with direct S3 requests.
We have read Railway's 22 October credential-reset recovery guide. This might relate to the 21 October incident, but the cause is not confirmed; obtaining the current credentials has not resolved it.
Requested help:
- Please check and repair the existing bucket's access/credential provisioning.
- Confirm whether the original objects remain intact and preserve them during investigation.
- If access cannot be restored promptly, provide a non-destructive export/recovery route for the existing objects.
We have preserved the database, object keys, and record/comment/photo relationships. Older photo files have no available independent local copy, so preserving the existing bucket is essential. New uploads now keep a persistent local copy as well.
Related email: sent to [email protected] on 26 October 2026 at 14:24 KST, subject "Production photo outage: hanip-photos AccessDenied; credential reset fails — preserve existing objects". No reply received as of our last inbox check on 27 October. This post is a follow-up through the official support channel.
No access keys, secrets, customer photos, or private application contents are included here. Detailed private diagnostics can be supplied through an appropriate support channel.
1 Reply
Status changed to Awaiting Railway Response Railway • 25 days ago
24 days ago
Sorry for the disruption to your photo dashboard and the delay getting back to you.
We’ve fixed the access issue affecting your existing hanip-photos bucket. There’s no need to delete or recreate it.
Status changed to Awaiting User Response Railway • 24 days ago
17 days ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • 17 days ago