17 days ago
We're experiencing a 403 Forbidden error with Varnish Error 54113 when making requests to PayPal's OAuth token endpoint (https://api-m.paypal.com/v1/oauth2/token) from our Vendure application hosted on Railway.
Key Details:
Same credentials work perfectly in Postman, curl, and other environments outside Railway
Error occurs consistently only when requests originate from Railway infrastructure
PayPal's Varnish cache server is blocking our requests with error code 54113
We've already implemented:
app.set('trust proxy', true)in Express configurationProper Basic Authentication with Base64 encoded credentials
Clean credentials (no whitespace)
Error Response:
403 Forbidden
Error 54113
Details: cache-ams21077-AMS
Varnish cache serverIs there any Railway-specific networking configuration (IP reputation, headers, IPv6/IPv4 routing) that could cause PayPal's WAF to block our requests? The fact that identical requests work from other networks suggests a Railway infrastructure-related issue.
Please advise if there are any known compatibility issues with PayPal's API or if Railway IPs might be flagged/blocked.
9 Replies
17 days ago
Hey there! We've found the following might help you get unblocked faster:
🧵 Urgent: Unable to Deploy Laravel PHP 8.2 Project Due to Docker Hub and Railpack Limitations
🧵 I need domain origin for outgoing requests to add to api whitelist
If you find the answer from one of these, please let us know by solving the thread!
17 days ago
This thread has been marked as public for community involvement, as it does not contain any sensitive or personal information. Any further activity in this thread will be visible to everyone.
Status changed to Open brody • 17 days ago
16 days ago
The issue seems to happen with Railway’s EU server. Enabling Static IP does not fix the issue, moving our frontend to a different region (like US East) fixes it !
15 days ago
PayPal blocks certain Railway EU datacenter IPs. cache-ams21077 means you're hitting their Amsterdam cache server - that's where the block happens.
Based on what luigi found, switching to us-west1 or us-east4 fixes it. Static IP won't help since Railway's IPs can be shared, and if someone else on that IP triggered PayPal's filters, you're blocked too.
If you need to stay in EU:
Route PayPal calls through a proxy service
- Or spin up a small VPS elsewhere ($6/mo DO droplet) just for PayPal OAuth requests
You can test if it's definitely IP-based by curling the endpoint from your Railway container vs locally - if local works and Railway doesn't, confirmed.
Region change is fastest fix though.
15 days ago
At the moment, we have resolved the error by hosting the PayPal confirmation on another server as a provisional measure. Railway MUST investigate this major issue IMMEDIATELY. This can cause significant economic losses, so this MUST be resolved
ciocca-dev
At the moment, we have resolved the error by hosting the PayPal confirmation on another server as a provisional measure. Railway MUST investigate this major issue IMMEDIATELY. This can cause significant economic losses, so this MUST be resolved
15 days ago
Unsure as to why you're pinning this on Railway. PayPal is returning 403, meaning the error is reaching PayPal's server. You should contact them regarding this issue.
Edit: What @caullenomdahl is correct. This is not Railway's responsibility, PayPal has for some reason blocked Railway.
12 days ago
How did you think this was our responsibility? Every other service is working fine. If PayPal has pined Railway's IP addresses as spam, there must be something wrong with Railway's infrastructure. We cannot contact PayPal on Railway behalf because this is a Railway infrastructure problem. As @caullenomdahl mentioned above, they also had to move to another server to resolve this issue
samgordon
Unsure as to why you're pinning this on Railway. PayPal is returning 403, meaning the error is reaching PayPal's server. You should contact them regarding this issue.Edit: What @caullenomdahl is correct. This is not Railway's responsibility, PayPal has for some reason blocked Railway.
9 days ago
Since Railway is providing the servers, it should also make sure its IPs don’t end up blacklisted anywhere, either by enforcing the correct use of services on the platform (the agreement we have to check during onboarding simply won’t do) or having regular checks with service providers to ban misbehaving users
luigigorlero
Since Railway is providing the servers, it should also make sure its IPs don’t end up blacklisted anywhere, either by enforcing the correct use of services on the platform (the agreement we have to check during onboarding simply won’t do) or having regular checks with service providers to ban misbehaving users
9 days ago
Railway has already likely removed the user causing the havoc, however it's not their responsibility to then go out and attempt to get the IP unblacklisted. There are thousands of potential services and Railway isn't monitoring them.
There is not something "wrong with Railway's infrastructure".
samgordon
Railway has already likely removed the user causing the havoc, however it's not their responsibility to then go out and attempt to get the IP unblacklisted. There are thousands of potential services and Railway isn't monitoring them.There is not something "wrong with Railway's infrastructure".
9 days ago
I respectfully disagree that this absolves Railway of responsibility in this situation.
Why This IS Railway's Responsibility
We cannot contact PayPal on Railway's behalf. PayPal won't unblock IPs based on requests from individual customers - they need to hear from the infrastructure provider directly. Railway has the business relationship and credibility to address this with PayPal, we don't.
Shared infrastructure = shared responsibility. When Railway provides shared IP addresses, maintaining the reputation of those IPs becomes part of the service we're paying for. If another Railway customer abused the service and got the IPs blacklisted, that's an internal Railway issue that shouldn't impact paying customers.
This is causing severe financial damage to our business. We're losing real revenue because Railway's EU infrastructure is blocked by a major payment provider. This isn't a minor inconvenience - it's actively harming our operations.
Why Suggested Solutions Don't Work for Us
Changing regions: Not feasible - our business operates in Europe, and moving to US regions would significantly impact latency and user experience
Dedicated IPs: While this might work, we shouldn't have to pay additional fees to work around Railway's IP reputation problems
External proxy: This adds unnecessary complexity, cost, and another point of failure to our production infrastructure
Our Experience
We have never received this kind of treatment from any hosting provider. Every other service we've used either proactively monitors IP reputation or takes responsibility for resolving blacklist issues when they occur.
Railway needs to either:
Contact PayPal directly to get your EU datacenter IPs unblocked
Implement better abuse prevention to keep your IPs off blacklists
Provide working dedicated IPs at no additional cost as compensation for this service failure
This is a critical infrastructure issue affecting multiple paying customers (has seen above). It requires Railway's direct intervention, not workarounds that shift the burden to users.