Wildcard domain stuck issuing TLS
stevenmichaelthomas
PROOP

2 years ago

I have an app that uses a wildcard domain (multi-tenant), but I've been unable to get things working in Railway. Root domain works great, but even with multiple retries (removing the wildcard domain and DNS records, re-adding, etc.) things seem stuck in the provisioning TLS certificate stage.

Any help is greatly appreciated - we are trying to move our app off of Vercel for many reasons, and this is the final stage to build the proof of concept and make a decision.

Thanks!

124 Replies

stevenmichaelthomas
PROOP

2 years ago

77a7cdf4-e635-4e13-aba1-aaca47f04c14


brody
EMPLOYEE

2 years ago

who's your DNS provider?


stevenmichaelthomas
PROOP

2 years ago

Cloudflare! i have turned off proxy for both cname records


stevenmichaelthomas
PROOP

2 years ago


stevenmichaelthomas
PROOP

2 years ago

app. is working perfectly. Seems to be an issue with wildcard. I've tried removing it and re-adding it - is there a way to manually provision things?


brody
EMPLOYEE

2 years ago

show me a screenshot of the railway domains please


stevenmichaelthomas
PROOP

2 years ago


brody
EMPLOYEE

2 years ago

what happens if you remove the wildcard domain from railway, and then add it back? (without touching dns in cloudflare)


stevenmichaelthomas
PROOP

2 years ago


stevenmichaelthomas
PROOP

2 years ago

^ So it's seeing the correct values in cloudflare


brody
EMPLOYEE

2 years ago

it technically shouldn't matter, those are random values and only serve as a means to resolve the correct ip


stevenmichaelthomas
PROOP

2 years ago

Got it. Yeah it seems this part is fine, but it gets firmly stuck on TLS issuing


stevenmichaelthomas
PROOP

2 years ago

Shall I update these values on Cloudflare and see what happens?


brody
EMPLOYEE

2 years ago

you may have hit the cert issuing limit


stevenmichaelthomas
PROOP

2 years ago

What's that? And is there a way around it? Or a way to resolve or debug?


brody
EMPLOYEE

2 years ago

i would have to flag the team


stevenmichaelthomas
PROOP

2 years ago

Ok! If you're able to help with this it would be hugely appreciated. We very ready to move our whole team over, but need to validate that this works as expected before we make the jump.


stevenmichaelthomas
PROOP

2 years ago

And it's a bit urgent for us because of performance issues on Vercel 😭


stevenmichaelthomas
PROOP

2 years ago

Thank you so much for your help!


brody
EMPLOYEE

2 years ago

may i ask if you are pro?


stevenmichaelthomas
PROOP

2 years ago

I am not yet, but as soon as we have this resolved we will migrate there


stevenmichaelthomas
PROOP

2 years ago

If moving to pro helps unblock, happy to do ti now


brody
EMPLOYEE

2 years ago

haha no thats not why i was asking, im not gonna ask you to upgrade while you are having issues.
just wanted to make sure you had the right discord badges is all 🙂


stevenmichaelthomas
PROOP

2 years ago

❤️ hahaha fair that would be a solid sales+support move 😂


stevenmichaelthomas
PROOP

2 years ago

But yes, we will be doing this shortly and moving the whole team over. Plz free us from Vercel!


stevenmichaelthomas
PROOP

2 years ago

Step change in performance already in our main app moving to Railway


brody
EMPLOYEE

2 years ago

glad to hear it, we welcome you to railway, and the community!


brody
EMPLOYEE

2 years ago

did you try end up trying this?


stevenmichaelthomas
PROOP

2 years ago

Yep! That's what lead to the above screenshot (I haven't changed anything since then)


brody
EMPLOYEE

2 years ago

okay thanks for confirming


stevenmichaelthomas
PROOP

2 years ago

np!


ray-chen
EMPLOYEE

2 years ago

Hey!

https://help.railway.app/questions/wildcard-domain-stuck-issuing-tls-93093f7e was this you?

It's been flagged to our infra eng - appears to be an issue on our end. It's the start of the day in North America so it'll get looked at pretty soon, apologies for the delay


stevenmichaelthomas
PROOP

2 years ago

That's me too! yes!


stevenmichaelthomas
PROOP

2 years ago

Thanks so much <@1060856209332260864> ! And all good. Things have been so awesome otherwise.


stevenmichaelthomas
PROOP

2 years ago

And good morning 🙂


jake
EMPLOYEE

2 years ago

Hi!


jake
EMPLOYEE

2 years ago

You around I'd love to help you fix this


jake
EMPLOYEE

2 years ago

I'm not able to pull dns records for it


brody
EMPLOYEE

2 years ago

I can dig it and get the correct results back


jake
EMPLOYEE

2 years ago

What's correct result in this case?


brody
EMPLOYEE

2 years ago

these


jake
EMPLOYEE

2 years ago

Both these look incorrect…

I'm getting Required value: [g7t2czuh.authorize.railwaydns.net](g7t2czuh.authorize.railwaydns.net)


jake
EMPLOYEE

2 years ago

(And, BTW the job to obtain it expired)


jake
EMPLOYEE

2 years ago

For some reason it only goes for 5 minutes


jake
EMPLOYEE

2 years ago

<@1159893063695605770> If you can retry issuing it super quickly that would be great


brody
EMPLOYEE

2 years ago

i think they r&r'd the domain in railway and didnt update the dns in cloudflare, but updating the cname in cloudflare shouldn’t be absolutely necessary since they are just random cnames, right?


brody
EMPLOYEE

2 years ago

that would explain all the "stuck issuing tls" help threads ive always seen


jake
EMPLOYEE

2 years ago

Yea


stevenmichaelthomas
PROOP

2 years ago

Hi!


stevenmichaelthomas
PROOP

2 years ago

I'm back again


stevenmichaelthomas
PROOP

2 years ago

What did you want me to check?


stevenmichaelthomas
PROOP

2 years ago

Current DNS:


stevenmichaelthomas
PROOP

2 years ago

Based on the latest values given in Railway, but it's still stuck.


stevenmichaelthomas
PROOP

2 years ago

Let me know if there's anything I can try


jake
EMPLOYEE

2 years ago

Can you delete the wildcard for me?


jake
EMPLOYEE

2 years ago

And retry


stevenmichaelthomas
PROOP

2 years ago

From where? In Cloudflare?


jake
EMPLOYEE

2 years ago

Both


stevenmichaelthomas
PROOP

2 years ago

Yep. Doing it now


jake
EMPLOYEE

2 years ago

Basically retry issuing it from scratch as if you'd never done it

So delete it in Cloudflare and Railway


jake
EMPLOYEE

2 years ago

Then, let DNS purge


jake
EMPLOYEE

2 years ago

Then, hit create in Railway


jake
EMPLOYEE

2 years ago

I'll walk alongside you and figure out wtf is wrong


stevenmichaelthomas
PROOP

2 years ago

Ok - back to this:


stevenmichaelthomas
PROOP

2 years ago

Ran a dig on both and the records are gone


jake
EMPLOYEE

2 years ago

Slick!


jake
EMPLOYEE

2 years ago

Okay, remove it on Railway


jake
EMPLOYEE

2 years ago

(Please)


stevenmichaelthomas
PROOP

2 years ago

Yep, done. Then re-added it in Railway, and just added records to cloudflare


stevenmichaelthomas
PROOP

2 years ago

dig now shows both entries


stevenmichaelthomas
PROOP

2 years ago

And we're back to this:


stevenmichaelthomas
PROOP

2 years ago


jake
EMPLOYEE

2 years ago

Okay! Lemme validate this…


stevenmichaelthomas
PROOP

2 years ago


stevenmichaelthomas
PROOP

2 years ago


jake
EMPLOYEE

2 years ago

Trying to pull the info sec


jake
EMPLOYEE

2 years ago

I see two txt records

solving challenge: *.[onorder.xyz](onorder.xyz): [*.[onorder.xyz](onorder.xyz)] authorization failed: HTTP 403 urn:ietf:params:acme:error:unauthorized - Incorrect TXT record \"7-glVtTwZT9qY9qVnOolZMaO8V9iv0pqqEmrnoFBeho\" (and 1 more) found at _acme-challenge.[onorder.xyz](onorder.xyz)


jake
EMPLOYEE

2 years ago

Can you delete the TXT records?


jake
EMPLOYEE

2 years ago

Oh, cause you have an app.[onorder.xyz](onorder.xyz) + a *.onorder.xyz


jake
EMPLOYEE

2 years ago

<@1159893063695605770> Sorry for tag it's time sensitive cause the workflow will retry


jake
EMPLOYEE

2 years ago

Can you tell me why your'e trying to do both here?


stevenmichaelthomas
PROOP

2 years ago


stevenmichaelthomas
PROOP

2 years ago

I wanted to get app. working since that's where we host our main app


stevenmichaelthomas
PROOP

2 years ago

and wildcard wasn't working


stevenmichaelthomas
PROOP

2 years ago

Should I remove that?


jake
EMPLOYEE

2 years ago

I think so ye


jake
EMPLOYEE

2 years ago

I think it's messing with the DNS but not 100% certain


stevenmichaelthomas
PROOP

2 years ago

Ok, done!


stevenmichaelthomas
PROOP

2 years ago

I think the issue existed before that, but no harm in trying!


jake
EMPLOYEE

2 years ago

Dunno where these guys are coming from



jake
EMPLOYEE

2 years ago

That's super weird


stevenmichaelthomas
PROOP

2 years ago

Yeah. Reading that thread to see if we can do anything


stevenmichaelthomas
PROOP

2 years ago


stevenmichaelthomas
PROOP

2 years ago

Ohhhh cloudflare has universal SSL


stevenmichaelthomas
PROOP

2 years ago

verified by TXT


jake
EMPLOYEE

2 years ago

Ah


jake
EMPLOYEE

2 years ago

Yea you'll have to turn that off


stevenmichaelthomas
PROOP

2 years ago

Try again?


stevenmichaelthomas
PROOP

2 years ago

This would be a great one to add to docs - I didn't turn this on, maybe new cloudflare default


jake
EMPLOYEE

2 years ago

If that's the issue defs


jake
EMPLOYEE

2 years ago

I'll try and dig the record


jake
EMPLOYEE

2 years ago

(The thing retries on a set schedule. Next one is in 8 minutes)


stevenmichaelthomas
PROOP

2 years ago


stevenmichaelthomas
PROOP

2 years ago

Looks better!


jake
EMPLOYEE

2 years ago

Ha


jake
EMPLOYEE

2 years ago

Beat me to it


jake
EMPLOYEE

2 years ago

!remind me to check back in 8 minutes


stevenmichaelthomas
PROOP

2 years ago

haha


stevenmichaelthomas
PROOP

2 years ago

awesome. Will check back shortly. Thanks


jake
EMPLOYEE

2 years ago

Marked as successful on our end!


jake
EMPLOYEE

2 years ago

Can you give it a poke and check on yours?


stevenmichaelthomas
PROOP

2 years ago

Oh damn!!!


stevenmichaelthomas
PROOP

2 years ago

it worked!


stevenmichaelthomas
PROOP

2 years ago

Thanks!


stevenmichaelthomas
PROOP

2 years ago

Great one to add to any docs and notes about cloudflare 🙂


stevenmichaelthomas
PROOP

2 years ago

But all makes sense!


stevenmichaelthomas
PROOP

2 years ago

Thank you so much!


brody
EMPLOYEE

2 years ago

super helpfull for me too, now i know where to look when a similar issue with wildcards happen, thanks cooper!


brody
EMPLOYEE

2 years ago

wow he's good


stevenmichaelthomas
PROOP

2 years ago

Just a real 👏 support experience all around


stevenmichaelthomas
PROOP

2 years ago

GIVE THESE PEOPLE A RAISE!


stevenmichaelthomas
PROOP

2 years ago

Hahah thanks so much. Team is taking a final look and we'll migrate everything over. Everything is working great.



Loading...