2 months ago
This summer I enabled Cloud Agents as the new feature. That month my usage spiked, I checked my usage overview and what used to be 6$ per month is now jumped 5 fold to 35$ a month and growing. Only in the cost breakdown I see that "VM Memory" is growing every hour and accounts for the increase in costs.
A few days a go I found out that the cloud agents never went to sleep. The documentation suggest they do once I disconnect. I dont know what they mean by disconnect but I deleted them. I have no more sandboxes and no more cloud agents since a few days, however the VM memory minutes and the associated costs keep growing. There must be something wrong, or I have something running somewhere but I cannot find out where is lives.
Please help because I might have to terminate this account if this is not resolved.
6 Replies
a month ago
Adding numbers to this.
I deleted every cloud agent and sandbox on 17 August and checked the same day. The API confirmed it: zero agents, zero running sandboxes, every environment. VM Memory read 16,087 GB-min at 09:26 UTC.
Four days later it reads 19,351 GB-min and is still climbing. That is 3,264 GB-min billed against an empty account, USD 3.92 so far, about a dollar a day.
Two things made this hard to find. VM usage is attributed to no project, so the Usage-by-Project breakdown cannot show it; those rows sum to exactly the four container line items. And there is no VM measurement in the GraphQL usage query, so you cannot chart it or ask when it started. "railway usage --json" omits it too, USD 9.58 of a USD 35.36 subtotal on my account.
Checking takes two minutes: open Usage, note VM Memory in the Bill Breakdown, come back in an hour. If it moved and you have nothing running, you have the same problem.
The draw is 0.66 GB at 0.02 vCPU, so it is holding memory without doing any work. That looks like something asleep that stopped burning CPU and kept being metered for its allocated RAM, which the docs say should not happen. VM RAM bills at USD 50/GB/month against USD 10 for containers, so that idle machine costs me more than twice all seven of my real projects combined.
Ticket filed. Curious whether others see around 0.6 GB or something that scales with the agent size, because that would say which part of the teardown is failing.
a month ago
It would be nice if someone could respond before I burn through all my credits and my services get halted because I hit a usage limit. VM Memory ever increasing and I have no way to stop it
Attachments
a month ago
Hey there, sorry for the delay in getting back to you.
This is a bug on our side, I'm raising it internally and we're going to make sure that you receive a credit for your usage here.
I will update you as soon as I know more.
Status changed to Awaiting User Response Railway • about 1 month ago
a month ago
Thanks you for writing a reply, I will increase my limits in the meanwhile and await your response.
Status changed to Awaiting Railway Response Railway • about 1 month ago
a month ago
The Cloud Agent VM that was still running after your deletion has been stopped, and we've applied a $10 credit to your workspace to cover the erroneous VM charges it accrued. Your updated credit balance will be reflected on your billing page at https://railway.com/workspace/billing.
Sorry for the inconvenience here, you should be good to go moving forward!
Let me know if you still see billing ticking up or anything unexpected going on
Status changed to Awaiting User Response Railway • about 1 month ago
a month ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • about 1 month ago