Edge CDN serves zstd to clients that don't accept it (broken Vary: Accept-Encoding)
jclaveau
PROOP

2 months ago

Given a cached zstd entry, the edge serves it to every request whose Accept-Encoding lacks zstd.

Safari/iOS sends gzip, deflate, br then gets the raw zstd frame and renders full-page binary garbage.

A single curl only looks intermittent because one client samples one PoP's cache state at one moment.

Mechanism: origin (Caddy) negotiates correctly and sets Vary: Accept-Encoding. The edge buckets Accept-Encoding too coarsely —gzip,deflate,br / br / identity / empty all collapse into the same bucket as zstd, and the bucket serves whichever variant warmed it first.

Captured (asset https://www.hiphiphip.app/assets/index-K-1UCDw4.js, x-cache: HIT):

gzip, deflate, br -> content-encoding: zstd ✗ client never sent zstd

gzip, deflate -> content-encoding: gzip ✓

br -> content-encoding: zstd ✗

identity / empty -> content-encoding: zstd ✗

gzip -> content-encoding: gzip ✓

Deterministic repro (fires the instant the edge has the key cached):

URL='https://www.hiphiphip.app/assets/index-K-1UCDw4.js?cb=UNIQUE'

curl -sI -H 'Accept-Encoding: zstd' "$URL" # warm bucket → zstd, cached

curl -sI -H 'Accept-Encoding: gzip, deflate, br' "$URL" # HIT → served zstd ← bug

Note: we currently see x-cache: MISS on every request to that PoP (cdg1) — looks like edge caching is cold/off for it right now;

was returning HITs earlier today. Did edge caching/compression change recently?

Impact: ~20% of global traffic (all Safari/iOS pre-26.3) hits full-page garbage on any zstd-cached asset/document.

Workaround live: we disabled zstd at origin (gzip only)

Project-Id: 65aff0db-6586-4be0-8420-b2e67ae4378d

Deployment with zstd enabled: c651d4e2-c21d-41a8-a12d-cc44005dd46e

image.png

Attachments

Closed

11 Replies

2 months ago

Thanks for the report, I'm looking into it.


2 months ago

This should be fixed fleet-wide. We now normalize valid Accept-Encoding values before sending the request upstream, to match our cacheable encoding buckets.


2 months ago

Thank you again for the detailed report!


jclaveau
PROOP

2 months ago

Thank you for your reactivity!


2 months ago

Are you confirming that this is fixed?


jclaveau
PROOP

2 months ago

I'll test again tomorrow, I need the feedback of the impacted user


2 months ago

Sounds good, thanks!


jclaveau
PROOP

2 months ago

Instead of waiting I tested the CDN change on a dedicated env: https://railway.com/project/65aff0db-6586-4be0-8420-b2e67ae4378d/service/8e8e4d50-be24-4b86-be1e-31b672fb1e55?environmentId=9d558642-6565-4b4f-96e7-4148e258b0c5

Here are the results I get:

  Accept-Encoding contains gzip OR br   → gzip   (all real browsers)
    gzip, deflate, br, zstd  (Chrome)   → gzip
    gzip, deflate, br        (Safari)   → gzip   ← gate PASS, no garbage
  Accept-Encoding without gzip/br       → zstd
    zstd / zstd, deflate                → zstd   (no real browser sends this)

So even browsers accepting zstd now get gzip if they accept it, making zstd support pointless.

Is it the behavior you expect?


2 months ago

Yes that's correct, we prioritize gzip in most cases which is the same behavior as other CDNs like Cloudflare


2 months ago

You may also notice that if the client only sends zstd (which would never be the case on a normal client), then we bypass the cache entirely. I have a task to better support clients that want to prioritize zstd, but the solution I implemented yesterday should fix the garbled content which caused the original issue.


jclaveau
PROOP

2 months ago

Yes, there is no issue anymore on legacy Iphones. I will stick to gzip in my codebase though. Is there some place i can track your zstd support?


Welcome!

Sign in to your Railway account to join the conversation.

Loading...