Are you using a custom or generated domain?

If custom, are you using Cloudflare's proxy?

Seen this before.
I'd suggest using Cloudflare's DNS and enabling their proxy. This will prevent bots/scrapers like these that try to find common exploits/vulnerabilities.

I tried both and it’s still happening.
Yes, the custom domain is behind Cloudflare DNS.
If it’s a public domain, then I think this is normal.
But I just deployed it 1–2 days ago and no one knows about it yet, which is weird.

Go to your domain page in Cloudflare dashboard, Security > Settings

Then turn on what you can

I think that should prevent these bots from trying to hit these routes.

The key ones are stuff like block AI bots, bot fight mode, and browser integrity check.

did you register the domain recently? the things theyre requesting are very common scans

I would just like to mention that you were using a public domain.

Sorry for not being clear. Yes, this is a custom domain and also a public domain.
By “public domain,” I mean it’s new and no one knew about it before.
So it doesn’t make sense for bots to attack it like this.

Bots are constantly scraping the internet.

Looking for leaked information, vulnerabilities, exploits, etc.

Thanks. I’ve enabled blocking for AI bots and will see the results soon.

My point is that this is a private project and it’s serverless, but the requests keep waking the server up every day. 😅

I’m just wondering, because other projects on the same domain (using subdomains and deployed on Railway) don’t have requests like this.

However, thanks everyone. I’ll continue investigating.

Well tbh, I'm not exactly sure how they even find domains like this in the first place.

I'm more or less making a general statement from what I know.

Yep, I agree. really appreciate this conversation. Thanks

Whenever you (or Cloudflare, Railway) provision a new TLS certificate for a domain, that is now public knowledge via the CT (certificate transparency) logs.

Searchable here https://crt.sh

So, I’d assume scrapers/bots use these logs to find new domains to exploit.