Encountered a random restart after using V2 runtime for a while

2 years ago

I was just randomly checking my deployment logs when I noticed a 'restart' out of nowhere. I checked sentry and there had been no errors, and I also checked my dev logs and the last command that was ran was at 7:02 PM (GMT+2) and that was just me running the ping command. I'm not sure if this is related to the runtime but it's the first ever time I've noticed it happen.

Solved

91 Replies

2 years ago

f8f1d4b6-1530-4b65-a444-67b8c86c5fca




This thread has been escalated to the Railway team.

Status changed to Awaiting Railway Response angelo-railway over 1 year ago


2 years ago

does this service have a volume?


2 years ago

No


2 years ago

The only service I have that has a volume is the db


2 years ago

I can also provide my sentry logs if that's neccessary


2 years ago

my bad anyway, i didn't notice the v2 runtime specific logs until after I asked


2 years ago

These are the last sentry logs. Nothing crashed and it wouldn't make sense if it did because the last command was just !ping and it was 2 hours ago

1253427440307540000


2 years ago

@cody Can I push changes to github or should I not touch anything for now? I need to update my bun.lockb file because I forgot to update it in the last push


2 years ago

Did i rlly ping myself


2 years ago

Ok dyslexia thanks


2 years ago

CC. @Brody, this was meant for u


2 years ago

you can go ahead and push the changes, your app comes first


2 years ago

Alright, tho the 'backup' is up and running rn so it's not as important


2 years ago

I'm gonna watch the logs and see if any more restarts happen


2 years ago

This is bun right? I wouldn't be surprised if it just crashed outta nowhere for some reason


2 years ago

Yes, though this has never happened before, and I only noticed it happening after switching to the v2 runtime


2 years ago

I just realized that was Jake commenting, lol


2 years ago

yeah the bridge should really show the usernames


2 years ago

So far so good, I haven't seen anything restart randomly


2 years ago

I think that's just lucky, as far as I know they haven't looked into anything


2 years ago

I wonder if they do keep track of things like exit code so they could tell you if this was your app exiting or an issue on their side


2 years ago

That'd be nice if they did, because It would mean something is wrong when it shouldn't be


2 years ago

unfortunately there's nothing I can do here, we would have to wait to hear back from a team member


2 years ago

I ran it locally for like an hour and it didn't exit


2 years ago

I suspect it may be something related to bun and the lockb file. It hasn't happened ever since I updated that


2 years ago

Jake also said it could be a bun thing


2 years ago

It's a possibility so I can't rule anything out for now. For all I know it has nothing to do with the v2 runtime


2 years ago

it's very possible it is a bun thing


2 years ago

Just that the timing could've been wrong and it seemed like v2 was the issue 🥲


2 years ago

it was a fair assumption, but we can't rule out either until it's looked into further


2 years ago

Yeah


2 years ago

Hey @Brody. This is a little offtopic but I wanted to know if there's a way to view the docker configuration railway automatically comes up with. I'm having issues with my Dockerfile again and I'm really lost


2 years ago

not really, that's kinda abstracted away, but what issues are you having?



2 years ago

My dockerfile uses the latest version of bun, which would be 1.1.16, so I don't think that's the issue. It hasn't happened again, though yesterday I went back to the v1 runtime just to be safe


2 years ago

Also speaking of the dockerfile, I'm trying to go back to my old architecture, which means compiling the typescript code first and then running it (dist folder), so bun doesn't eat away my ram, and the issue is that I simply don't know how to do that since railway used to manage it for me. I tried like 20 different setups and ultimately each of them either failed either on build or on deployment 🫠


2 years ago

have you checked out buns docs on their recommendations for running in production?


2 years ago

Yeah, I'm currently trying to build one based off of their example. I tried doing so yesterday (cough at 2am) but that didn't turn out well


2 years ago

If I'm understanding docker docs and the guide correctly then it shouldn't be much work


2 years ago


2 years ago

in their example, the build command generates a myapp executable that should be well executed


pauldps
HOBBY

2 years ago

I run multiple bun apps on Railway and haven't noticed random restarts AFAIK, but one thing you could do is have your app capture SIGnals and output to console when one happens (something like "Received signal…"), so that you can determine if Railway is sending signals to your app to kill it


2 years ago

oh that's a great idea


2 years ago

and on that note, capturing signals will only work if your app is executed directly, running an npm command to then start your app will cause the signals to be lost


pauldps
HOBBY

2 years ago

I like capturing SIGINT (local Ctrl+C) and SIGTERM (sent by Railway on sleep/restart/etc)


pauldps
HOBBY

2 years ago

yeah that's true, I ended up doing a shell script to orchestrate my app start (the ones with things like database migration) and the shell script captures signals and outputs to console


2 years ago

but also follow buns guide I linked, multiple people have reported it has saved them memory


2 years ago

I used to do this too, but I removed it in the commit I made a month ago. I thought that It'd be to since I've never gotten any unexpected restarts or anything else that might lead to the app getting killed/crashing. I guess I can re implement it again but it hasn't happened again since I made this post and talked about it in <#727689277219012669>


2 years ago

I'm not concerned about saving memory that much, but I'd prefer it to run on 130mb on idle compared to the 250 it's running rn


2 years ago

Locally it runs on 200 but for some reason on railway it's 250


2 years ago

well, once again dockerfile didn't work, wohoo


2 years ago

dockerfiles always work, when written correctly 🙂


2 years ago

Well yeah. Mine works like 75%, then it just crashes saying it can't find dist


2 years ago

Should I open another thread to get a little help with it


2 years ago

I don't know too much about bun but sure!


2 years ago

It's the same as npm, has practically no difference. My issue is just with paths and everything as I don't understand how docker works. <#1254872133654548591>


2 years ago

On a side note


2 years ago

1255233222221234200


2 years ago

This happened


2 years ago

i think its safe to say that this is your app crashing


2 years ago

Yeah but these only happen on railway, not locally, which Is what makes me feel it has something to do with railway


2 years ago

And this is on the v2 runtime. In more than half a year of using the platform this has never happened lol


2 years ago

could be an issue with the new version of bun


2 years ago

have you implemented signal catching yet?


2 years ago

can you console log the HOSTNAME variable for me too?


2 years ago

No, I'm finding it difficult to balance work and development, and I've also been pretty sick lately 😅


2 years ago

get well soon 🙂


2 years ago

I can do that, yeah


2 years ago

You do have a point. I've only noticed "weird" things when using bun


2 years ago

I might make a separate branch that uses node to deploy on railway and see if that fixes it


2 years ago

please do


2 years ago

Bun is still relatively new so I understand it has its issues as well


2 years ago

because at this point it really is looking like an issue with your application (bun)


2 years ago

Bun seems like it's more trouble than it's worth lol


2 years ago

at this point, I agree


2 years ago

I'll have to change my dockerfile which will probably drive me nuts like the previous time. God forbid I'm doing that alone, I'll ask my friend for help 🥲


2 years ago

dockerfiles are fun when you get the hang of them


2 years ago

far better than dealing with nixpacks


2 years ago

True


2 years ago

I should probably install docker desktop so I can stop making 20 commits every time I update my file


2 years ago

I've been "testing" the changes on railway, and I realize that doing so was just causing me more problems


2 years ago

docker desktop - that's a good idea


2 years ago

I think that I can pretty much rule out that this is a railway issue. I'm 99% willing to bet it's bun because I just had a crash right now after running a command that gets some stats from bun


2 years ago

It happened without any context and this has never happened on node. It does happen a few times when I run locally as well


2 years ago

I spent quite a bit testing different scenarios and sure enough it just died


2 years ago

It's going back to node tomorrow, I don't want more random crashes and restarts


2 years ago

moving back to node is for the best


2 years ago

Yeah I'm not gonna bother with bun anymore lol. Maybe in the future when it gets a lot better and stable but definitely not for now


2 years ago

sounds good to me


Railway
BOT

2 years ago

This thread has been marked as solved automatically due to a lack of recent activity. Please open a new thread if you require further assistance. Thank you!

Status changed to Solved Railway over 1 year ago


Loading...