The linked service is attempting to communicate with this service https://railway.com/project/28f95e16-ecc7-4cb3-82e6-7b22d4f9eefe/service/5c1f125a-82bb-4767-b262-8d9be87a405b?environmentId=5855d21e-0aa3-4962-b7d2-8a6cd4031189

.

Lmk what you need from me, i'm here

Howdy!

Poked around and networking isnt looking like an issue here. Networking would prevent any form of connection. This is a 403 which means your auth is a bit funky. Looks like specifically the backend service is having a 403 on a specific route.

When did this first appear?

First started this mornign around 1230AM as per railway logs

https://railway.com/project/28f95e16-ecc7-4cb3-82e6-7b22d4f9eefe/logs?environmentId=5855d21e-0aa3-4962-b7d2-8a6cd4031189&filter=%40service%3A%22access-germain-backend%22+AND+403&timeFrame=7d

Could it be cloudflare sitting infront of the domain coming from the railway native request?

Because I can make the API call from local completely fine

Yep that was the issue

Come on guys

I just pushed a fix by moving to internal networking

but somethign is getting lost between our requests from the app and the core-02 domain

are you able to log on the send and receiving end whats being sent/received to see what differs?

I can get on a call if needed

All I did to fix was change the domain on the upstream (access backend) app to the railway internal and its working

which to me clearly indicates an issue at the public domain

which is being proxied by cloudflare

Nothing is different thats the only thing I changed

Glad you're unblocked! But we do not modify your request, I'm a bit confused as to why this happens but 4xx are historically application level faults. Not a ton I can do here 😅

I mean

What kind of an explanation is that

would you like me to point it back to the public URL and show that it breaks again lol

I can quite literally show you the issue in real time

What domain is 403'ing?

https://core-02.germainlabs.com/

Have you done something in Cloudflare to make it work? It doesn't 403 for me.

It doesn't 403 for me either from local

Its only from railway

Sounds like Cloudflare's proxy is blocking requests from Railway, something you would need to look into in your Cloudflare dashboard.

Yeah so cloudflare is flagging the railway IP

Our network engineer had a quick look at your traffic, and he believes Cloudflare thinks you are fuzzing, so that would explain it.

Either way, proxying via the private network is best practice, so something good has come of this!

Got it. We will stuck to internal networking then

Thank you as always

Appreciate you bringing it to networking

No problem, sorry for the scare!