503 Error when using Railway with Terraform

mw2000PRO

10 months ago

I'm running into a 503 - Service Unavailable error when trying to use terraform (specifically terraform plan) with railway using the railway API.

I'm using this community plugin to integrate terraform and railway - https://registry.terraform.io/providers/terraform-community-providers/railway/latest

My read on this is some kind of API Rate limit on the railway token. Our setup already works with 50 railway terraform resources, but gives us an error when we add 20 more.

Solved

9 Replies

mw2000PRO

10 months ago

21b7eb49-e84c-40c9-9278-73e17ff7b318


mw2000PRO

10 months ago

tagging @sina from our team for more visibility


10 months ago

!t


10 months ago

This thread has been escalated to the Railway team.

Status changed to Awaiting Railway Response brody 10 months ago


10 months ago

Can you send a payload of what the issue is? We'd like to look at the rate limiting response.

Not seeing anything otherwise that would suggest this is on our side. But would like to dig in deeper. Thanks.


mw2000PRO

10 months ago

This is what i can see on my end on terraform on running the plan

Attachments


10 months ago

We've merged what should be a fix to this issue, raising the rate limit.

Please let us know if this issue reoccurs.

Thanks.


mw2000PRO

10 months ago

This issue still occurs. using a fresh railway api token. It's fine for about 50 resources but the moment we add more resources to manage, the issue resurfaces

│ Error: Client Error

│ with railway_shared_variable.witness_core_production_op_sepolia_eth_rpc,

│ on main.tf line 40, in resource "railway_shared_variable" "witness_core_production_op_sepolia_eth_rpc":

│ 40: resource "railway_shared_variable" "witness_core_production_op_sepolia_eth_rpc" {

│ Unable to read shared variable, got error: returned error 503 Service

│ Unavailable: <html>

│ <head><title>503 Service Temporarily Unavailable</title></head>

│ <body>

│ <center><h1>503 Service Temporarily Unavailable</h1></center>

│ <hr><center>nginx/1.22.1</center>

│ </body>

│ </html>


10 months ago

Do you know which endpoint this is hitting? Because I've lifted it to allow 100 RPS bursts, but Terraform could be making tons of additional requests

Can you also send your IP?

Additionally, we don't maintain the Terraform provider. What's the usecase that you're using it?


10 months ago

Additionally I can totally track 100+ services no problem. Maybe this is an issue with one of the resources in the provider?

Which resources are you trying to add? I would probably file this against the GitHub issue and then we can have a look there as well

Attachments


mw2000PRO

10 months ago

@jake This is hitting the following endpoint: https://backboard.railway.app/graphql/v2(https://github.com/terraform-community-providers/terraform-provider-railway/blob/88190d08233dc212881ff4fe5ebf8fe40a676af6/internal/provider/provider.go#L90)

Resources I'm trying to add are - railway project, railway environment, railway service, along with custom domains, shared variables, and deployment triggers.

The use-case is to maintain the railway infra alongside other infra as IAC for the company.

My IP: 67.180.48.234


mw2000PRO

10 months ago

Which GH repo do you recommend flagging this issue on? The terraform provider?


10 months ago

Which GH repo do you recommend flagging this issue on? The terraform provider?

This would be the right place.


Status changed to Solved railway[bot] 10 months ago