Poor performance, and crashing services with no error - am I reaching hobby plan hardware limits?

5 months ago

Project ID: 55ec2528-2ba2-49ff-86e1-44e2194d1272

Hello. I recently added a few more services to my project, it now has 7 individual services. I have noticed that the performance has become worse recently, one of the frontend apps (storefront) has extremly slow response time, it is making requests that has a roundtrip through a nextjs backend in the service it self, then via an API to a more advanced backend, which is connected to a postgres database - so it is quite a few steps, but it used to be decently fast, and when i run the project locally, it also performs pretty fast. Now, since a few weeks, the full roundtrip from the frontend https requests takes between 10-20 seconds. It used to be about 0.5-2 seconds.

I also have a docker service (meilisearch) that recently started crashing, and I do not see any direct crash error in the runtime log - so I have a suspiscion that it is terminting due to hardware limitations.

Is it likely that I am reaching the hardware limits for my hobby user - and upgrading to pro, could be a solution?

Maybe some admin here have better insight in users reaching their limits? and could check for me, if I am reaching those limits? πŸ™‚

Solved

0 Replies

5 months ago

hello,

can you please make sure all your services are in the same non-metal region


5 months ago

as for meilisearch, seems like it got a bit greedy

1332767828243251200


5 months ago

Hey Brody! Thanks for checking in. πŸ™‚
Amazing, changing the regions fixed the roundtrip delay ⚑

How do I ensure my services are in the same region? This project was launched from a template, and by default the the services was not on same non-metal region.

Could you elaborate a little on the meilisearch greediness? I am using a standard docker image, I never had issues with it before today (but havent tested it in about 10 days). I have been using the :latest - but I have also tested getmeili/meilisearch:v1.11.3 - which I know didn't cause problems just a few weeks ago, and this image also crashes now (after some time, length seems a bit random).


5 months ago

New hobby deploys will buy default go to metal, this isn't something configurable at the moment.

oom_killed = out of memory killed, aka it used more than the allowed 8gb and was shut down


5 months ago

Aha, okay! I see all the docker services (postgres, minio, meiliseach, redis) are all deployed in Oregon (non-metal), while all the git services are deployed on the new inhouse metal servers. πŸ€” I received quite a few emails from people asking me why their webshop is slow - so for now I will tell them to manually change the regions. Do you know if there are any plans for changes to this, so manual steps wont to required?

phew 8 gb of memory use 🀒 that's very odd. it should not use that much


5 months ago

I will ticket a thing that says "don't deploy to metal if template contains services with volumes"


5 months ago

metal doesn't support volumes yet hence the services with volumes still being on gcp


5 months ago

you are using the private network for all interservice communications right?


5 months ago

OK! It's exciting with the metal servers. Will there be any metal servers in the EU? (GDPR compliance attractiveness)

No, not everything is private network yet unfortunately - but this is on my list of things I want to imrove. There's a internal connection between the postgres database and the backend. But redis, is currently over public, same goes for the api between the storefront app and the backend service - because the backend also hosts a dashboard web app - but I do have an idea for how to solve this, it's just work that I havent gotten to yet


5 months ago

why is redis public?

and yes hobby will get an EU and Asia region, though I'd have to say that if GDPR mattered, that's probably not a hobby project anymore?


5 months ago

No that's unrelated… just went through a whole proces trying to figure out if supabaser was GDPR compliant and had to involve 3rd party specialist, who ended up concluding that they weren't. And I saw that there recently was released a self managed supabase template here, on Railway, so that could be one way of running it for customers in EU that needs GDPR compliance - if Railways metal server do not share user data outside of the EU ✌️

Regarding redis, that's simply just because I havent had the time yet. switching to the private url broke the app, but I know there's some documentation somwhere gonig over exatcly how to use redis within the internalt network


5 months ago

this documentation?


5 months ago

ticket just created btw!


5 months ago

Yep that's the one!


4 months ago

Hello @rpuls - sorry for the long wait, but this has been fixed, if a template has volumes, all services will deploy to the same GCP region


4 months ago

No worries! I created a little screen recording guide, how to manually set same regions. And I did notice some days later that deployments were automatically on same region. ✌️thanks for following up though 😊


4 months ago

we only just made this change today


4 months ago

Really πŸ€”? I had deploys for… 2~ weeks where all services were same regions


4 months ago

yep just today


3 months ago

!s


Status changed to Solved brody β€’ 4 months ago


Poor performance, and crashing services with no error - am I reaching hobby plan hardware limits? - Railway Help Station