8 days ago
Hello, I have been using CloudFront CDN for a while on my service.
My pattern roughly looks like:
-
Incoming User Request
-
CloudFront (strips XFF header, to avoid spoofing)
-
Railway Frontend Service (forwards XFF header)
-
(Internal Network) Railway Backend Service (consumes XFF header)
-
Check for Googlebot (rate limit pass)
-
Response
This was working fine for me until June 4th. Since June 4th, evening UTC, I can see in my logs that no more Googlebots are detected. This also overlaps with Edge / CDN changes on the Railway side.
Is someone please able to help me explain what is happening here? What changes to the EDGE / CDN broke my logic, and what can I do to fix it?
EDIT: I roughly use these CloudFront configuration instructions: https://dev.to/isoppp/connecting-railway-to-cloudfront-6hb
Best Regards,
Jacques
Pinned Solution
7 days ago
We don't strip any headers like that on the edge; the behavior should've been preserved. Could you let me know how you're checking for GoogleBot? The only header we now strip and re-write on the edge is X-Forwarded-For, so if you're using that header to check the client IP, then it will now be the CloudFront IP, not the GoogleBot IP. You can forward your own client IP header to Railway from CloudFront instead and use that to identify the real downstream IP.
That being said, we do now offer a free CDN which you can enable per-service which should be able to replace CloudFront entirely. If you have a use case for CloudFront which our CDN doesn't offer, I'd also be interested to hear it so we could consider building it into Railway directly.
2 Replies
8 days ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • 8 days ago
7 days ago
We don't strip any headers like that on the edge; the behavior should've been preserved. Could you let me know how you're checking for GoogleBot? The only header we now strip and re-write on the edge is X-Forwarded-For, so if you're using that header to check the client IP, then it will now be the CloudFront IP, not the GoogleBot IP. You can forward your own client IP header to Railway from CloudFront instead and use that to identify the real downstream IP.
That being said, we do now offer a free CDN which you can enable per-service which should be able to replace CloudFront entirely. If you have a use case for CloudFront which our CDN doesn't offer, I'd also be interested to hear it so we could consider building it into Railway directly.
phin
We don't strip any headers like that on the edge; the behavior should've been preserved. Could you let me know how you're checking for GoogleBot? The only header we now strip and re-write on the edge is `X-Forwarded-For`, so if you're using that header to check the client IP, then it will now be the CloudFront IP, not the GoogleBot IP. You can forward your own client IP header to Railway from CloudFront instead and use that to identify the real downstream IP. That being said, we do now offer a free CDN which you can enable per-service which should be able to replace CloudFront entirely. If you have a use case for CloudFront which our CDN doesn't offer, I'd also be interested to hear it so we could consider building it into Railway directly.
7 days ago
Hi phin,
Thanks for this answer, I will accept it as a solution because it pointed me to the right direction.
I am also leaving more details below in case another user might find it helpful:
- Railway changed edge behaviour around beginning of June to strip
X-Forwarded-For, which means it is no longer containing the true client IP. - The fix is to add your own custom header by attaching a CloudFront Function to the
Viewer Requestof your behaviour - You want to set your header value in said function to
event.viewer.ip - Then you can update your service to consume this new custom header instead of
X-Forwarded-For(or overwriteX-Forwarded-Forwith a middleware depending on your framework)
@phin Is your CDN service also allowing to configure path-pattern based behaviours routing to different origins like CloudFront does? Or is it pure caching functionality for a single service?
Status changed to Solved dev • 2 days ago