Railway behind CloudFront CDN no longer working
Anonymous
PROOP

8 days ago

Hello, I have been using CloudFront CDN for a while on my service.

My pattern roughly looks like:

  1. Incoming User Request

  2. CloudFront (strips XFF header, to avoid spoofing)

  3. Railway Frontend Service (forwards XFF header)

  4. (Internal Network) Railway Backend Service (consumes XFF header)

  5. Check for Googlebot (rate limit pass)

  6. Response

This was working fine for me until June 4th. Since June 4th, evening UTC, I can see in my logs that no more Googlebots are detected. This also overlaps with Edge / CDN changes on the Railway side.

Is someone please able to help me explain what is happening here? What changes to the EDGE / CDN broke my logic, and what can I do to fix it?

EDIT: I roughly use these CloudFront configuration instructions: https://dev.to/isoppp/connecting-railway-to-cloudfront-6hb

Best Regards,

Jacques

Solved$20 Bounty

Pinned Solution

7 days ago

We don't strip any headers like that on the edge; the behavior should've been preserved. Could you let me know how you're checking for GoogleBot? The only header we now strip and re-write on the edge is X-Forwarded-For, so if you're using that header to check the client IP, then it will now be the CloudFront IP, not the GoogleBot IP. You can forward your own client IP header to Railway from CloudFront instead and use that to identify the real downstream IP.

That being said, we do now offer a free CDN which you can enable per-service which should be able to replace CloudFront entirely. If you have a use case for CloudFront which our CDN doesn't offer, I'd also be interested to hear it so we could consider building it into Railway directly.

2 Replies

Railway
BOT

8 days ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway 8 days ago


7 days ago

We don't strip any headers like that on the edge; the behavior should've been preserved. Could you let me know how you're checking for GoogleBot? The only header we now strip and re-write on the edge is X-Forwarded-For, so if you're using that header to check the client IP, then it will now be the CloudFront IP, not the GoogleBot IP. You can forward your own client IP header to Railway from CloudFront instead and use that to identify the real downstream IP.

That being said, we do now offer a free CDN which you can enable per-service which should be able to replace CloudFront entirely. If you have a use case for CloudFront which our CDN doesn't offer, I'd also be interested to hear it so we could consider building it into Railway directly.


phin

We don't strip any headers like that on the edge; the behavior should've been preserved. Could you let me know how you're checking for GoogleBot? The only header we now strip and re-write on the edge is `X-Forwarded-For`, so if you're using that header to check the client IP, then it will now be the CloudFront IP, not the GoogleBot IP. You can forward your own client IP header to Railway from CloudFront instead and use that to identify the real downstream IP. That being said, we do now offer a free CDN which you can enable per-service which should be able to replace CloudFront entirely. If you have a use case for CloudFront which our CDN doesn't offer, I'd also be interested to hear it so we could consider building it into Railway directly.

Anonymous
PROOP

7 days ago

Hi phin,

Thanks for this answer, I will accept it as a solution because it pointed me to the right direction.

I am also leaving more details below in case another user might find it helpful:

  • Railway changed edge behaviour around beginning of June to strip X-Forwarded-For, which means it is no longer containing the true client IP.
  • The fix is to add your own custom header by attaching a CloudFront Function to the Viewer Request of your behaviour
  • You want to set your header value in said function to event.viewer.ip
  • Then you can update your service to consume this new custom header instead of X-Forwarded-For (or overwrite X-Forwarded-For with a middleware depending on your framework)

@phin Is your CDN service also allowing to configure path-pattern based behaviours routing to different origins like CloudFront does? Or is it pure caching functionality for a single service?


Status changed to Solved dev 2 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...