Issue with Redis Outbound
magicmike123
HOBBYOP

2 years ago

Hi, I have a doubt about Redis, Im using a Redis service for my API, both deployed here in Railway. Cache works fine right now for what im doing but theres an issue, its showing some serious (100 MB+ of outbound network) on a request with the cache, is that a bug?

Closed

222 Replies

magicmike123
HOBBYOP

2 years ago

N/A


magicmike123
HOBBYOP

2 years ago

N/A because im not sure which id to show, the backend id, or redis id?


magicmike123
HOBBYOP

2 years ago

And i just saw this: , does that mean im doing it wrong? Sorry im newbie with this stuff

1305170997477117995


2 years ago

you should check the variables on your app's service, not the Redis service


magicmike123
HOBBYOP

2 years ago

oh, okey, im going to check that now


magicmike123
HOBBYOP

2 years ago

mm I dont see any:

1305245589977432064


magicmike123
HOBBYOP

2 years ago

not sure if im doing it right


2 years ago


magicmike123
HOBBYOP

2 years ago

im confused right now I read that but im kinda same, im a noob at this


2 years ago

look at the picture


magicmike123
HOBBYOP

2 years ago

the picture i sent?


2 years ago

the picture in the docs page i sent


magicmike123
HOBBYOP

2 years ago

ah okey


magicmike123
HOBBYOP

2 years ago

1305247223860297748


2 years ago

^


magicmike123
HOBBYOP

2 years ago

im looking at it but i cant figure out, i mean what i understand is that some variables that are used in 2 places must be placed here, right


magicmike123
HOBBYOP

2 years ago

right


2 years ago

no, you have no need for shared variables


magicmike123
HOBBYOP

2 years ago

but this worked before for me at some point which is even more strange


magicmike123
HOBBYOP

2 years ago

ah right thats why it worked, cause they not needed then


magicmike123
HOBBYOP

2 years ago

1305247586428387359


magicmike123
HOBBYOP

2 years ago

1305247625880141824


magicmike123
HOBBYOP

2 years ago

I got these envs right now on my 2 services


magicmike123
HOBBYOP

2 years ago

wait when you say app service you mean like, my service-backend-project, right?


2 years ago

yes


2 years ago

send a link to the project please


magicmike123
HOBBYOP

2 years ago

here in my code i reference this

1305248055616077885


magicmike123
HOBBYOP

2 years ago

yes



2 years ago

youre good, its using the private url


magicmike123
HOBBYOP

2 years ago

and REDIS URL seems a string I dont even remember where I got the value from, maybe it should be reference variable?


magicmike123
HOBBYOP

2 years ago

aah okey so im good there


2 years ago

it already is a reference variable


magicmike123
HOBBYOP

2 years ago

okey ✅ so thats ok


magicmike123
HOBBYOP

2 years ago

but yeah outbound is still like, big, it wasnt at some point god knows what happened not me xd


2 years ago

i dont really see any outbound on redis


magicmike123
HOBBYOP

2 years ago

1305248899543339028


magicmike123
HOBBYOP

2 years ago

1305248943088730182


2 years ago

thats not the redis service


magicmike123
HOBBYOP

2 years ago

cause im just basically getting the data from redis on cache in theory


magicmike123
HOBBYOP

2 years ago

ah


magicmike123
HOBBYOP

2 years ago

wait can you see where it comes from? i got a login and user-settings but no way thats 130 mb


2 years ago

no i cant


magicmike123
HOBBYOP

2 years ago

maybe my cache is just not working now? but it works locally, how could it not work same on railway? strange issue


magicmike123
HOBBYOP

2 years ago

i think maybe i messed up railway redis


magicmike123
HOBBYOP

2 years ago

because locally i always do redis flush all


magicmike123
HOBBYOP

2 years ago

but i cant flsh all on redis railway service, is there a way to flush all?


magicmike123
HOBBYOP

2 years ago

maybe old data is caussing some issue idk


2 years ago

again, you are connecting to redis via the private network, so there would be no measurable network


magicmike123
HOBBYOP

2 years ago

right, so all that data is 100% not redis then... mmm what could it even be idk


2 years ago

its any outbound data leaving the private network, could be mongo, could be sending data to your frontend


2 years ago

side question, where is the frontend service?


magicmike123
HOBBYOP

2 years ago

yeah its here, also having an issue with it where it crashes on a certain page when it tries to fetch some data: https://railway.app/project/e1a3daa8-0161-42a8-9945-2ddf4a33a66d


2 years ago

why is it in a different project?


magicmike123
HOBBYOP

2 years ago

wait what? i just see theres no repo connected to it, wtf

1305250403096727692


magicmike123
HOBBYOP

2 years ago

cause i had to create new project when i was setting it up


magicmike123
HOBBYOP

2 years ago

or maybe its mistake and should been a service?


magicmike123
HOBBYOP

2 years ago

but how is it no repo connected


magicmike123
HOBBYOP

2 years ago

ah wait right


magicmike123
HOBBYOP

2 years ago

because i uploaded with the weird react method


magicmike123
HOBBYOP

2 years ago

OOH


2 years ago

via cli


magicmike123
HOBBYOP

2 years ago

then its not updating


magicmike123
HOBBYOP

2 years ago

like backend


magicmike123
HOBBYOP

2 years ago

f0ck


2 years ago

^


magicmike123
HOBBYOP

2 years ago

funk


magicmike123
HOBBYOP

2 years ago

right


magicmike123
HOBBYOP

2 years ago

hold on hold on


magicmike123
HOBBYOP

2 years ago

how can I make it so that it does like backend it auto updates on git commit? is possible? that must be the issue 99%


magicmike123
HOBBYOP

2 years ago

cause i did commit pushed and thought already frontend updated like backend but no


magicmike123
HOBBYOP

2 years ago

thats gotta be the issue


magicmike123
HOBBYOP

2 years ago

1st question: can i fix it where it is now then move it to the same project?


2 years ago

delete it and deploy it into the correct project this time


magicmike123
HOBBYOP

2 years ago

ah okey, lets do that


magicmike123
HOBBYOP

2 years ago

if i rename project, does it affect anything in terms of functionality or its just name change?

1305251204963500163


magicmike123
HOBBYOP

2 years ago

just so instead of backend project ah i change it to project ah and put all the 3 services together: backend, frontend, and redis


2 years ago

just a name change


magicmike123
HOBBYOP

2 years ago

okey i changed that name now


magicmike123
HOBBYOP

2 years ago

so for the react thingy is there a way to make it like, automatically build from github repo everytime or i gotta do the cli setup every time?


2 years ago

lets get the service into the correct project first


magicmike123
HOBBYOP

2 years ago

yeah ima delete the frontend project 1 sec


magicmike123
HOBBYOP

2 years ago

done


magicmike123
HOBBYOP

2 years ago

right, so first of all i create an empty service


magicmike123
HOBBYOP

2 years ago


magicmike123
HOBBYOP

2 years ago

oh theres a one click template option, but i never used template before


magicmike123
HOBBYOP

2 years ago

1305252411471958036


2 years ago

no, you already did that.

empty service -> name the service -> set variables -> deploy -> create domain -> attach github repo -> deploy


magicmike123
HOBBYOP

2 years ago

ah, okey


magicmike123
HOBBYOP

2 years ago

theres no variables in the frontend except the process.env.REDIS_URL I think being only exception. So for that I gotta use reference variable I believe


2 years ago

the frontend does not need redis


magicmike123
HOBBYOP

2 years ago

1305252909222596700


magicmike123
HOBBYOP

2 years ago

ah right


magicmike123
HOBBYOP

2 years ago

you are right


2 years ago

the frontend should need a backend api domain variable though


magicmike123
HOBBYOP

2 years ago

for now im just hardcoding the values but will change in future dont worry


magicmike123
HOBBYOP

2 years ago

okey so i cant see the domaint hingy


magicmike123
HOBBYOP

2 years ago

yeah i cant see it in settings


2 years ago

fixed


magicmike123
HOBBYOP

2 years ago

ah deploy


magicmike123
HOBBYOP

2 years ago

okey


magicmike123
HOBBYOP

2 years ago

shows me this

1305253451734843412


magicmike123
HOBBYOP

2 years ago

i click there to make a deployment yes?


magicmike123
HOBBYOP

2 years ago

idk


2 years ago

id go with the big deploy button


magicmike123
HOBBYOP

2 years ago

oke


magicmike123
HOBBYOP

2 years ago


2 years ago

no it does not


magicmike123
HOBBYOP

2 years ago

wait


magicmike123
HOBBYOP

2 years ago

ah no


magicmike123
HOBBYOP

2 years ago

the purple button you meant right


magicmike123
HOBBYOP

2 years ago

the deploy one, wasnt looking at it


magicmike123
HOBBYOP

2 years ago

1305254462377558016


magicmike123
HOBBYOP

2 years ago

it shows me this now


2 years ago

it seemed pretty noticeable to me lol


magicmike123
HOBBYOP

2 years ago

sorry im not used to the interface yet


magicmike123
HOBBYOP

2 years ago

i miss on obvious stuff


2 years ago

continue with the steps i gave you


magicmike123
HOBBYOP

2 years ago

i been user of railway for few years but actively went back just like few days ago


magicmike123
HOBBYOP

2 years ago

okey


magicmike123
HOBBYOP

2 years ago

okey i created domain, next is attach github repo


magicmike123
HOBBYOP

2 years ago

and next deploy, okey


magicmike123
HOBBYOP

2 years ago

okey its doing stuff its loading


magicmike123
HOBBYOP

2 years ago

YEEES


magicmike123
HOBBYOP

2 years ago

GOOD JOB!


magicmike123
HOBBYOP

2 years ago

it was this thing man, i can finally see my data again my stuff 🥹 nice!


magicmike123
HOBBYOP

2 years ago

lets see if it also fixes the outbound stuff, right? because now in theory it same project


magicmike123
HOBBYOP

2 years ago

okey so 1st request is this: 130mb, completely normal, data is not cached (i need to re-cache every hour btw its not a permanent cache)

1305255824234385441


magicmike123
HOBBYOP

2 years ago

so now i wait like 2 mins and reload the page again to re-fetch, in theory from cache and hopefully no more big outbound data like this 130 mb


2 years ago

no it wont, to avoid outbound traffic you need to use the private network and you cant use the private network for client side rendered websites, or 3rd party services such as mongo


magicmike123
HOBBYOP

2 years ago

wait...


magicmike123
HOBBYOP

2 years ago

😢


2 years ago

if i recall correctly ive said that a few times


magicmike123
HOBBYOP

2 years ago

you probably did but i did not get it, im too slow with all these stuff im new to it and my mind if like close to explode plus im coding so much these days learning stuff too im like exhausted mentally tbh


magicmike123
HOBBYOP

2 years ago

so theres no absolute way to cache the 130 mb response for the frontend theres no trick or workaround? my app literally wont be profitable if I dont find a way to do that idk what to do 😦


magicmike123
HOBBYOP

2 years ago

and i spent months on it omg idk


2 years ago

dont send 130mb worth of data to the frontend then


magicmike123
HOBBYOP

2 years ago

right...right, i just cant, it defeated my entire plan but theres no work around


2 years ago

though im just going off that you are saying, id first figure out what this outbound traffic is


magicmike123
HOBBYOP

2 years ago

must think of something


magicmike123
HOBBYOP

2 years ago

ah


magicmike123
HOBBYOP

2 years ago

yeah its the 120k items, but its weird because the size is about 65 MB


magicmike123
HOBBYOP

2 years ago

55 to 65 MB, no idea why it shows 130 MB, its like it fetching the data twice, but is not.


2 years ago

figure out where this outbound traffic is going


magicmike123
HOBBYOP

2 years ago

ima try


2 years ago

is your frontend actually downloading 130mb of data?


magicmike123
HOBBYOP

2 years ago

only 65


magicmike123
HOBBYOP

2 years ago

1305259366861246494


magicmike123
HOBBYOP

2 years ago

cant be a coincidence than 130 mb is twice 65


2 years ago

we are not double counting outbound


magicmike123
HOBBYOP

2 years ago

else {

console.log("through the else");

const userDesiredItemsFromCache = await retrieveArrayFromRedis();

res.json({
  items: userDesiredItemsFromCache,
  update: {
    nextMinute: timer.minute,
    nextSecond: timer.second,
  },
});

}


magicmike123
HOBBYOP

2 years ago

thats my cache code


magicmike123
HOBBYOP

2 years ago

maybe retrieveArray is doing it twice idk need to see that


magicmike123
HOBBYOP

2 years ago

doesnt seem like

1305260372961267792


2 years ago

are you console logging 65mb worth of data?


magicmike123
HOBBYOP

2 years ago

yeah


2 years ago

lets not do that


magicmike123
HOBBYOP

2 years ago

that could be the issue maybe?


2 years ago

no


magicmike123
HOBBYOP

2 years ago

ah


magicmike123
HOBBYOP

2 years ago

yeah i removed the console.logs


magicmike123
HOBBYOP

2 years ago

rest of code on the cached version are these:

1305261286287736832


magicmike123
HOBBYOP

2 years ago

theres an if below, but im going through the else on all cached (verified im going through it)


2 years ago

find a way to not send 65 mb worth of data to the frontend


magicmike123
HOBBYOP

2 years ago

yeah thats something i need to do, but lets ignore this for a moment now and try to see why it takes 130 mb


magicmike123
HOBBYOP

2 years ago

im not doing a double operation there and im just fetching a timer from mongoDB thats an object with 3 fields


magicmike123
HOBBYOP

2 years ago

connectedRealms is not even a fetch but its an object with like 100 fields, neither can be that


2 years ago

dont really know what to tell you, i have no way of looking at your network traffic


magicmike123
HOBBYOP

2 years ago

and getHighestTime returns about 300 KB


magicmike123
HOBBYOP

2 years ago

ah right


magicmike123
HOBBYOP

2 years ago

well no worries I will have to change all of this somehow and hopefully that fixes that issue as well, who knows


magicmike123
HOBBYOP

2 years ago

but yeah


magicmike123
HOBBYOP

2 years ago

brb dinner


magicmike123
HOBBYOP

2 years ago

Hi Brody, I got it way way lower now frontend response is about 2 MB, for some reason it will still shows 4 MB but i think thats fine.


magicmike123
HOBBYOP

2 years ago

So I want to try and calculate the cost of a user network usage based on this, you let me know if what i say makes sense


2 years ago

scroll to the bottom -


magicmike123
HOBBYOP

2 years ago

So a user makes request of 4 MB to my app, he will lets say, do it on average around 4 times per hour, so 16 MB per hour. I expect the user to (at most, lets put higher case) use this app for 8 hours a day, 25 days a month. so thats 3200 MB, 3,2GB


magicmike123
HOBBYOP

2 years ago

so that should be 0.32$ network usage


magicmike123
HOBBYOP

2 years ago

does this make sense?


magicmike123
HOBBYOP

2 years ago

yeah I know pricing its 0.1$ per 1 GB of network egress


magicmike123
HOBBYOP

2 years ago

just want to make sure I understand the thing with this simplified math I did for an estimate


magicmike123
HOBBYOP

2 years ago

ooh


2 years ago

scroll to the bottom please


magicmike123
HOBBYOP

2 years ago

I can make example there, nice


magicmike123
HOBBYOP

2 years ago

okey I do example there thanks ✅


2 years ago

switch to resource-based


magicmike123
HOBBYOP

2 years ago

yes


magicmike123
HOBBYOP

2 years ago

1305281305101205564


2 years ago

well you absolutely will not be using 0 cpu and 0 bytes of memory


2 years ago

and that's only 1 users worth of networking


magicmike123
HOBBYOP

2 years ago

oh yeah yeah


magicmike123
HOBBYOP

2 years ago

i meant just to calculate the network of 1 user, yes


magicmike123
HOBBYOP

2 years ago

cause then I have some doubt, RAM for example, it takes around 400 MB of RAM per app load, but like they just load the app and then they can see all data they dont need to reload often just once every hour for user, for new data to come in because external API updates every 1 hour. so maybe they are using the app 1/3 of all the entire time of a day, so that would be 400 MB of RAM divided between 3, so 133 MB RAM average cost per user, does my example make sense?


2 years ago

nope


magicmike123
HOBBYOP

2 years ago

oh, how do i calculate it then


2 years ago

put in your average cpu and memory usage


magicmike123
HOBBYOP

2 years ago

right its what i mean, the average it is the cost for RAM and CPU


2 years ago

so put in the average and it will tell you the cost


magicmike123
HOBBYOP

2 years ago

so if user peaks 400 MB on app load doesnt matter because lets say he does 1 app load per month then average would be super low, like 10 MB or whatever


magicmike123
HOBBYOP

2 years ago

right


2 years ago

I highly doubt your app could idle down to 10mb


2 years ago

especially with replicas


magicmike123
HOBBYOP

2 years ago

you think even less, or more?


2 years ago

far more, it's node


magicmike123
HOBBYOP

2 years ago

ah no no, forget the app, it will have a base RAM cost


magicmike123
HOBBYOP

2 years ago

but im thinking the added RAM usage per user right now


magicmike123
HOBBYOP

2 years ago

idle is probably 300 MB, but per user maybe on average it adds 10 MB, is what I mean,based on my above example here


2 years ago

then put that into the calculator


magicmike123
HOBBYOP

2 years ago

average is good because nobody uses app 24 hours, at most they will use 12 hours and 99% of people will use less


magicmike123
HOBBYOP

2 years ago

yeah


magicmike123
HOBBYOP

2 years ago

another question: lets imagine im hitting the 32 GB RAM limit, can I add 1 replica and then it scales about 64 GB of RAM? (I know its not exactly like that in theory, but in practice, in a very simplified way of putting it, its basically double hardware power, right?)


magicmike123
HOBBYOP

2 years ago

and another replica for 96 GB of RAM, etc etc...


2 years ago

it's still 32gb of ram per instance, just now that you have two running the total combined can be 64gb


magicmike123
HOBBYOP

2 years ago

yeah its what i mean, the total combined


2 years ago

then yes, but that may be subject to change


magicmike123
HOBBYOP

2 years ago

so theorical limit is crazy high because if I have 1 service + the 49 replicas available is 1600 GB of RAM


magicmike123
HOBBYOP

2 years ago

but dont worry cause my bank account is little


magicmike123
HOBBYOP

2 years ago

and i will have little server


magicmike123
HOBBYOP

2 years ago

im not rich xd


2 years ago

exactly, and what's why I say it's subject to change, that's far too much resources for a single user


magicmike123
HOBBYOP

2 years ago

but its good to know the limit, i think i will never find limit in this platform which is possitive to hear 😄


magicmike123
HOBBYOP

2 years ago

yeah it really is lol


Welcome!

Sign in to your Railway account to join the conversation.

Loading...