There's nothing reported at this time, is there anything in your logs that would indicate that they have crashed?

I have an uptime monitor in place on my api that has indicated the api has been down for around 40 minutes.

I'm also seeing the below in the my logs for the API:

[Nest] 56 - 04/08/2024, 2:39:21 PM ERROR [ExceptionsHandler] Network error: Unexpected token < in JSON at position 0

Which seems to be an error that results from a Server error

I'm also seeing issues with my auth server not being accessible and my migrator crashing

For some context, everything was working yesterday

These issues would not be due to an outage with the platform.

The classic Unexpected token < in JSON at position 0 means the server / api responded with HTML instead of JSON.

Please provide the exact full length errors you are getting.

Here's what I see in my application

Please provide the exact full length errors you are getting.

In the mean time, have you tried re-deploying your services that have crashed?

Most of my services appear to be up but when I trying accessing them I get errors, the below seems to be the root of the issue:

ClientError: GraphQL Error (Code: 524): {"response":{"error":"\n\n\n\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

\n

\n

\n

\n A timeout occurred\n Error code 524\n

\n

\n Visit cloudflare.com/5xx-error-landing?utm_source=errorcode_524&utm_campaign=g.synclabs.so\"" rel="\"noopener">cloudflare.com

for more information.\n

\n

2024-04-08 14:52:38 UTC

\n

\n

\n

\n

\n \n

\n

\n \n \n \n \n

\n You\n

\n \n Browser\n \n

\n Working\n

\n\n

\n

\n cloudflare.com/5xx-error-landing?utm_source=errorcode_524&utm_campaign=g.synclabs.so\"" rel="\"noopener">\n \n \n

\n

\n Seattle\n

\n cloudflare.com/5xx-error-landing?utm_source=errorcode_524&utm_campaign=g.synclabs.so\"" rel="\"noopener">\n Cloudflare\n

\n

\n Working\n

\n\n

\n

\n \n \n \n \n

\n g.synclabs.so\n

\n \n Host\n \n

\n Error\n

\n\n

\n

\n \n\n

\n

\n

I'm assuming synclabs is a 3rd party api your apps talk to?

It going down has brought your own apps down it would seem.

synclabs is our own API

Which we deployed to railway along with all of our other apps

Any ideas why the API is down, is it related to Railway?

The API would be down because it has crashed, please check it's deployment logs. This would not be due in part to anything Railway related.

Whats throwing my off is our API looks to be up an running on Railway (I attached a screen shot).

The only log I see related to any issues is the one I first shared:

[Nest] 56 - 04/08/2024, 3:03:43 PM ERROR [ExceptionsHandler] Network error: Unexpected token < in JSON at position 0
@replica: 8a4250f2-7200-4551-a6e6-cbfb72a9aea9
View in Context

So you're saying this is an issue with my API failing? If thats the case, why did it build without any issues, and what could cause this build to work yesterday but not today?

I very much appreciate all the help with this!

Green doesn't always mean it is up and running with perfect health, it could even have a status of "Completed" meaning your code has exited with a non-error status code, or even simply soft locked.

Building an app and running an app are two vastly different operations and are not comparable in the slightest.

As previously mentioned, that error indicates that whatever API call the app has made has returned HTML (because of the angle bracket) Instead of JSON.

It's starting to seem like there might not be the most optimal error handling in the code for these apps. So please try to redeploy all your apps.

I'll try redeploying now, we'll work on improving the error handling as well.

Now that I think about it, the app that is returning this error hasn't had any updates recently. Could this be a problem with Cloudflare, our DNS service?

Nope, Not a problem with Railway, Cloudflare, or DNS, your app has crashed, locked up, or at least something along those lines.

I'm not getting the full picture here so I'm not able to pinpoint what domino has fallen first and why, for the answer to that you would need to carefully review your deployment logs for all effected services.