9 days ago
Hi — looking for help diagnosing a 403 that only seems to affect Facebook's link-preview scraper.
App: https://offtosleepstrong.com (Next.js, Railway-hosted)
Symptom: Facebook's Sharing Debugger consistently returns 403 when scraping https://offtosleepstrong.com — "URL returned a bad HTTP response code." Link previews on Facebook/Messenger therefore show only bare domain text, no OG image or title.
Ruled out so far:
-
App-side block. Curl tests with the exact FB UA strings (including meta-externalagent/1.1) all return 200 against apex and www, plus all marketing pages. Same result with Googlebot, Twitterbot, Slackbot, LinkedInBot, standard browser UAs.
-
Middleware block. Crawler-UA bypass in middleware ahead of any auth pipeline. Response headers confirm middleware passes through cleanly.
-
robots.txt block. User-agent: * / Allow: / with only private app paths disallowed.
-
Upstream proxy/CDN. DNS confirms apex and www both point directly to Railway (hduu8kl0.up.railway.app and 9gbgkvq1.up.railway.app, 66.33.22.x). No Cloudflare, no other CDN. Server header is railway-edge.
-
Stale FB cache. Multiple Scrape Again + browser refresh cycles over several hours still show 403 with fresh timestamps.
iMessage, Slack, LinkedIn, Twitter all generate clean link previews for the same URL. Block appears isolated to Facebook's scraper IP range.
Hypothesis: Railway's edge may apply IP-level rate-limiting or bot-protection rules that catch FB's scraper IPs specifically, returning 403 before requests reach the app.
Questions:
-
Does railway-edge maintain IP-level blocks or rate limits against known scraper / crawler IP ranges?
-
If yes, can you allowlist Facebook's published scraper IP ranges for our project, or guide me to a setting to opt out of edge bot protection?
-
If no, what additional debug info would help isolate this?
Happy to provide deploy timestamps, project ID, or anything else. Thanks!
1 Replies
9 days ago
This thread has been marked as public for community involvement, as it does not contain any sensitive or personal information. Any further activity in this thread will be visible to everyone.
Status changed to Open Railway • 9 days ago
9 days ago
Unfortunately, this isn't something Railway can fix. I've tried curling the URL with Meta's UA and I'm getting HTTP 200 codes. According to older threads, this error happens somewhere after the request leaves Railway. You'd need to raise this issue with Meta.