Deployed app responds randomly with 503s

obstkompost
PRO

a year ago

Hello everyone,

when I open the admin dashboard of a deployed KeyCloak instance, I get 503s instead of the resource for random web resources (images, JS files, html files,…).
In the browser's network console you can then see the Railway "Application failed to respond" page in the bodies of the 503.

I have to reload the page several times until all the resources needed for the dashboard could be loaded.

The problem has existed for quite a while. In the logs of the deployed KeyCloak instance there is no indication that anything is wrong (so no noticeable error logs or anything like that)

0 Replies

obstkompost
PRO

a year ago

f751037a-ba7a-42ab-8a24-90e99fcd12d3


a year ago

you had it working before right?


obstkompost
PRO

a year ago

I've had this problem for what feels like forever.
I think the 503s always appeared randomly for this app


a year ago

i take it you have not deployed keycloak from the template then?


obstkompost
PRO

a year ago

Yeah, i am using a "selfmade"-DockerImage


a year ago

okay so for context, i have seen this before with keycloak, sometimes its server sends malformed responses that envoy (railway's proxy) does not like, the solution was to run a caddy proxy in the same container in parallel and expose the caddy proxy only, caddy would internally proxy to keycloak, thus smoothing over the responses from keycloak's server (for lack of a better term)

You can see how i have implmented that here:


obstkompost
PRO

a year ago

Thank you very much!
I will try it and post the result here


a year ago

let me know if you have any issues!


obstkompost
PRO

a year ago

I can not believe it but it seems like it works now 😃
I reloaded the Dashboard 10 times and could not produce one 503er Error

Thank you very much Brody!

I have still some question, about the topic:

  • What kind of malformed responses are keycloak sending?
    Is there an open ticket at KeyCloak for this?
    If the responses do not meet the Http specification, then that needs to be corrected.
    Or is the railway proxy too sensitive here?


a year ago

What kind of malformed responses are keycloak sending?
i honestly have no clue

Is there an open ticket at KeyCloak for this?
i also have no clue

Or is the railway proxy too sensitive here?
this is more likely the case, envoy is getting removed in favour of railway's own home grown proxy anyway at some point


obstkompost
PRO

a year ago

Does it then make sense to address the issue with Railway so that they can configure their proxy accordingly?


a year ago

keycloak is the only thing i have seen have this issue, so no it doesnt really make sense for them to spend the time to fix this thing that likely only effects keycloak


obstkompost
PRO

a year ago

fair enough (:


a year ago

new proxy too, so also not worth the time when envoy is getting thrown out a window


a year ago

just curious too, where you using the latest version of keycloak?


obstkompost
PRO

a year ago

One more little question:
In the keycloak-Template-Repo there is this java.config:
https://github.com/leonardochappuis/keycloak-docker/blob/master/java.config
Do i need that too or why are this file there?


obstkompost
PRO

a year ago

Yeah i was using the latest Version of KeyCloak


a year ago

i think that was to remove a newer auth version for database communications that the database railway deployed at the time didnt support, but now railway deploys up to date databases so you likely dont need to do that


a year ago

you are now using multirun and caddy in the container too?


obstkompost
PRO

a year ago

right


a year ago

awsome!


obstkompost
PRO

a year ago

you can mark your answer as the solution


obstkompost
PRO

a year ago

Bought your more Coffee for the help!
You saved me many hours!


a year ago

thank you very much, I appreciate that!!


Deployed app responds randomly with 503s - Railway Help Station