Bad Experience with Raiway; Erroneous billing issue and services shutdown. No response from support team
mystical_whale_914
PROOP

a month ago

Railway decided to shutdown my production services because an estimated usage crossed my threshold. My billing cycle ends on 24th of the month and the actual usage was around 18% less than the set threshold. In spite of that Railway decided to shutdown the services. Railway support confirmed that it was based on some calculation that is not visible to users like me. I was able to recover all but one service. Railway support guided me with recovering the DB (start etcd services - which did not start with rest of the DB services). But I am unable to recover storage bucket. I have emailed the support team and according to Railway Agent, they closed that ticket without notifying me. I have emailed them a second time.

I am posting to check if someone knows how to recover a storage bucket and if someone can help me.

(No service is selected in this thread because Central Station help request is not showing my services)

Solved

7 Replies

Railway
BOT

a month ago

When the compute hard limit is hit, bucket access is suspended but existing data is preserved and not deleted. Access resumes once you raise or remove the hard limit, or when the next billing period starts. Your workspace is currently active and your billing cycle rolled over on June 24, so your bucket data should still be intact and accessible. If you are still seeing "Access Denied" errors on the bucket after having raised or removed the hard limit, please reply with the specific error you are seeing so we can narrow down the issue.


Status changed to Awaiting User Response Railway about 1 month ago


mystical_whale_914
PROOP

a month ago

image.png

Compute usage never crossed the limit on my dashboard. I have increased the threshold and current usage in dashboard is less than the threshold. It is showing access denied. Based on what you are saying above I must be able to access the bucket now. This is the worst experience I have had.

Attachments


Status changed to Awaiting Railway Response Railway about 1 month ago


mystical_whale_914
PROOP

a month ago

None of my services can access the assets in bucket using pre-signed url. If you have an automated way to shut things down, you should have a way of restarting the service when the issue is resolved.


Status changed to Awaiting Railway Response Railway about 1 month ago


mystical_whale_914
PROOP

a month ago

It started working when I increased the threshold or limit by another 30%. Overall I had to increase the limit by 60% for the services to start working. There are some internal calculations/bugs that are causing the billing discrepancies and in turn causing these issues for the customer.


Status changed to Awaiting Railway Response Railway about 1 month ago


sam-a
EMPLOYEE

a month ago

Glad the bucket is accessible again. The hard limit enforces against estimated usage for the full billing period, not just current usage, which is why the dashboard total can appear lower than the trigger point. We understand that gap was frustrating, especially with production workloads affected. You can review and adjust both the hard limit and email alert threshold from your Usage page at any time.

Please let us know if you run onto another issue.

Again, apologies for the impact this had on you.


Status changed to Awaiting User Response Railway about 1 month ago


mystical_whale_914
PROOP

a month ago

How can you enforce something on estimated usage?

What happens if I raise memory or resources on a service and that momentarily pushes the estimated usage above my set limit — does that suspend my services? For instance I can increase the memory on a service or DB for a day (for testing or running something memory intensive) which can throw the estimated usage above the threshold and cause suspension of services. It can happen in other ways as well like it did for me this week. This enforcement does not make sense. Additionally not all services come back up after increasing the limit.

Also, how can you suspend services based on a calculation that is not visible to the user. I have to wait for an alert email to come to know this (I was traveling internationally when it happened to me)


Status changed to Awaiting Railway Response Railway about 1 month ago


Railway
BOT

a month ago

The estimated projected usage is visible on your Usage page alongside current usage, so you can monitor it before it reaches the hard limit. Yes, temporarily scaling up a service's memory or CPU would increase the projected usage for the billing period, and if that projection crosses your hard limit, enforcement triggers. You can set a custom email alert at a lower dollar threshold for earlier warning. When services don't auto-recover after raising the limit, you can manually redeploy them from the deployment's 3-dot menu.


Status changed to Awaiting User Response Railway about 1 month ago


Railway
BOT

24 days ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 24 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...