Encountered a random restart after using V2 runtime for a while

10 months ago

I was just randomly checking my deployment logs when I noticed a 'restart' out of nowhere. I checked sentry and there had been no errors, and I also checked my dev logs and the last command that was ran was at 7:02 PM (GMT+2) and that was just me running the ping command. I'm not sure if this is related to the runtime but it's the first ever time I've noticed it happen.

Solved

1 Replies

10 months ago

f8f1d4b6-1530-4b65-a444-67b8c86c5fca



10 months ago

!t


10 months ago

This thread has been escalated to the Railway team.

Status changed to Awaiting Railway Response angelo 10 months ago


10 months ago

does this service have a volume?


10 months ago

No


10 months ago

The only service I have that has a volume is the db


10 months ago

I can also provide my sentry logs if that's neccessary


10 months ago

my bad anyway, i didn't notice the v2 runtime specific logs until after I asked


10 months ago

These are the last sentry logs. Nothing crashed and it wouldn't make sense if it did because the last command was just !ping and it was 2 hours ago

1253427440307540000


10 months ago

@cody Can I push changes to github or should I not touch anything for now? I need to update my bun.lockb file because I forgot to update it in the last push


10 months ago

Did i rlly ping myself


10 months ago

Ok dyslexia thanks


10 months ago

CC. @Brody, this was meant for u


10 months ago

you can go ahead and push the changes, your app comes first


10 months ago

Alright, tho the 'backup' is up and running rn so it's not as important


10 months ago

I'm gonna watch the logs and see if any more restarts happen


10 months ago

This is bun right? I wouldn't be surprised if it just crashed outta nowhere for some reason


10 months ago

Yes, though this has never happened before, and I only noticed it happening after switching to the v2 runtime


10 months ago

I just realized that was Jake commenting, lol


10 months ago

yeah the bridge should really show the usernames


10 months ago

So far so good, I haven't seen anything restart randomly


10 months ago

I think that's just lucky, as far as I know they haven't looked into anything


10 months ago

I wonder if they do keep track of things like exit code so they could tell you if this was your app exiting or an issue on their side


10 months ago

That'd be nice if they did, because It would mean something is wrong when it shouldn't be


10 months ago

unfortunately there's nothing I can do here, we would have to wait to hear back from a team member


10 months ago

I ran it locally for like an hour and it didn't exit


10 months ago

I suspect it may be something related to bun and the lockb file. It hasn't happened ever since I updated that


10 months ago

Jake also said it could be a bun thing


10 months ago

It's a possibility so I can't rule anything out for now. For all I know it has nothing to do with the v2 runtime


10 months ago

it's very possible it is a bun thing


10 months ago

Just that the timing could've been wrong and it seemed like v2 was the issue 🥲


10 months ago

it was a fair assumption, but we can't rule out either until it's looked into further


10 months ago

Yeah


10 months ago

Hey @Brody. This is a little offtopic but I wanted to know if there's a way to view the docker configuration railway automatically comes up with. I'm having issues with my Dockerfile again and I'm really lost


10 months ago

not really, that's kinda abstracted away, but what issues are you having?



10 months ago

My dockerfile uses the latest version of bun, which would be 1.1.16, so I don't think that's the issue. It hasn't happened again, though yesterday I went back to the v1 runtime just to be safe


10 months ago

Also speaking of the dockerfile, I'm trying to go back to my old architecture, which means compiling the typescript code first and then running it (dist folder), so bun doesn't eat away my ram, and the issue is that I simply don't know how to do that since railway used to manage it for me. I tried like 20 different setups and ultimately each of them either failed either on build or on deployment 🫠


10 months ago

have you checked out buns docs on their recommendations for running in production?


10 months ago

Yeah, I'm currently trying to build one based off of their example. I tried doing so yesterday (cough at 2am) but that didn't turn out well


10 months ago

If I'm understanding docker docs and the guide correctly then it shouldn't be much work


10 months ago


10 months ago

in their example, the build command generates a myapp executable that should be well executed


pauldpsHOBBY

10 months ago

I run multiple bun apps on Railway and haven't noticed random restarts AFAIK, but one thing you could do is have your app capture SIGnals and output to console when one happens (something like "Received signal…"), so that you can determine if Railway is sending signals to your app to kill it


10 months ago

oh that's a great idea


10 months ago

and on that note, capturing signals will only work if your app is executed directly, running an npm command to then start your app will cause the signals to be lost


pauldpsHOBBY

10 months ago

I like capturing SIGINT (local Ctrl+C) and SIGTERM (sent by Railway on sleep/restart/etc)


pauldpsHOBBY

10 months ago

yeah that's true, I ended up doing a shell script to orchestrate my app start (the ones with things like database migration) and the shell script captures signals and outputs to console


10 months ago

but also follow buns guide I linked, multiple people have reported it has saved them memory


10 months ago

I used to do this too, but I removed it in the commit I made a month ago. I thought that It'd be to since I've never gotten any unexpected restarts or anything else that might lead to the app getting killed/crashing. I guess I can re implement it again but it hasn't happened again since I made this post and talked about it in <#727689277219012669>


10 months ago

I'm not concerned about saving memory that much, but I'd prefer it to run on 130mb on idle compared to the 250 it's running rn


10 months ago

Locally it runs on 200 but for some reason on railway it's 250


10 months ago

well, once again dockerfile didn't work, wohoo


10 months ago

dockerfiles always work, when written correctly 🙂


10 months ago

Well yeah. Mine works like 75%, then it just crashes saying it can't find dist


10 months ago

Should I open another thread to get a little help with it


10 months ago

I don't know too much about bun but sure!


10 months ago

It's the same as npm, has practically no difference. My issue is just with paths and everything as I don't understand how docker works. <#1254872133654548591>


10 months ago

On a side note


10 months ago

1255233222221234200


10 months ago

This happened


10 months ago

i think its safe to say that this is your app crashing


10 months ago

Yeah but these only happen on railway, not locally, which Is what makes me feel it has something to do with railway


10 months ago

And this is on the v2 runtime. In more than half a year of using the platform this has never happened lol


10 months ago

could be an issue with the new version of bun


10 months ago

have you implemented signal catching yet?


10 months ago

can you console log the HOSTNAME variable for me too?


10 months ago

No, I'm finding it difficult to balance work and development, and I've also been pretty sick lately 😅


10 months ago

get well soon 🙂


10 months ago

I can do that, yeah


10 months ago

You do have a point. I've only noticed "weird" things when using bun


10 months ago

I might make a separate branch that uses node to deploy on railway and see if that fixes it


10 months ago

please do


10 months ago

Bun is still relatively new so I understand it has its issues as well


10 months ago

because at this point it really is looking like an issue with your application (bun)


10 months ago

Bun seems like it's more trouble than it's worth lol


10 months ago

at this point, I agree


10 months ago

I'll have to change my dockerfile which will probably drive me nuts like the previous time. God forbid I'm doing that alone, I'll ask my friend for help 🥲


10 months ago

dockerfiles are fun when you get the hang of them


10 months ago

far better than dealing with nixpacks


10 months ago

True


10 months ago

I should probably install docker desktop so I can stop making 20 commits every time I update my file


10 months ago

I've been "testing" the changes on railway, and I realize that doing so was just causing me more problems


10 months ago

docker desktop - that's a good idea


10 months ago

I think that I can pretty much rule out that this is a railway issue. I'm 99% willing to bet it's bun because I just had a crash right now after running a command that gets some stats from bun


10 months ago

It happened without any context and this has never happened on node. It does happen a few times when I run locally as well


10 months ago

I spent quite a bit testing different scenarios and sure enough it just died


10 months ago

It's going back to node tomorrow, I don't want more random crashes and restarts


10 months ago

moving back to node is for the best


10 months ago

Yeah I'm not gonna bother with bun anymore lol. Maybe in the future when it gets a lot better and stable but definitely not for now


10 months ago

sounds good to me


railway[bot]

9 months ago

This thread has been marked as solved automatically due to a lack of recent activity. Please open a new thread if you require further assistance. Thank you!

Status changed to Solved railway[bot] 9 months ago