7 months ago
hey there! i'm facing a weird issue. i'll try my best to explain it. i've an application that's meant to run in the background 24/7 and processes some information. it opens a grpc request, processes that data, sleeps for a while, and does this in a loop. while processing the information, it also sends a few http requests to external servers to fetch some data.
i've tried running the code locally and it works fine — the grpc request is opened properly, the http requests are sent out properly, and overall the data is being processed as expected.
when i tried deploying this service on railway, the grpc request was opened properly, but the http requests weren't being sent out. i've attached a screenshot of a function that sends out http requests while processing the data.
i had added a few logs earlier (removed them for the sake of brevity), and from the logs, it looks like it doesn't get past the getTokenInfos function. it's a simple function that sends out a post request. i tried logging the response, but it wasn't visible in the railway logs. am i doing something wrong? i'm unable to figure out what's the exact issue because the logs don't tell anything i.e. no error logs.
41 Replies
that's the project id. you can try re-deploying the latest deployment and notice that it acts the records to positions table but not ixns table, which might be because of the above mentioned issue
7 months ago
You said that it sleeps after the grpc and http request. Are you waiting until the HTTP request is completed before sleeping?
7 months ago
Which library are you using for grcp and http? node-http?
im using this library - https://www.npmjs.com/package/@connectrpc/connect-node
7 months ago
So to clarify the grcp goes through but the http is what has 0 response hmmm
7 months ago
Are you reaching out to a server that has strict cors or access policy?
yup, correct. the server doesn't have any strict cors or access policy afaik. ive tried to run a simple curl request via sshing into the service to confirm that the sending a request manually doesn't have any issues
7 months ago
And you are using node-http or axios
im using ky library for sending http requests which uses fetch api under the hood
7 months ago
And the catch block is never being triggered correct?
7 months ago
On first start could you make a similar/dummy request with node-fetch? Would be interested if the ky library has some issues <:Hmmmmm:891565000068587551>
7 months ago
The first thing that comes to my mind is await failing because of sleeps. But as you said thats not an issue
yea, it's quite weird and unexpected behavior. ive faced it again with another service which is a script which just fetches a bunch of rows from a db and processes them in parallel. during processing, it sends requests to a few external api services (https://public-api.birdeye.so and https://mainnet.helius-rpc.com). it's a metal instance. it was just stuck for a while ~10 mins. there was no request timeout error in the logs as well (ive set the timeout to be 10s during experimentation). are those apis blocklisted or something?
7 months ago
if when you ssh or connect to the container you are able to make a request to those endpoints they arent locked. I wonder if during processing something is changing which is just odd?
7 months ago
yeah let me google around a bit
7 months ago
what node version are you using?
7 months ago
I'm reading up here too:
https://station.railway.com/questions/last-night-cet-random-fetch-failed-5d71a0b7
via ssh, it seems fine. that's what left me confused. the script is quite simple - just fetches some records from db, getting some metadata from external api services and updating the data in db.
7 months ago
I'm still a bit sketchy about the ky thing. Are you able to temporarily swap to using axios/plain fetch and see?
ill give it a shot but im not quite sure if that would have any effect because the code works fine locally.
7 months ago
Yeah I'm not sure that would fix it either but its something to rule out. If that doesnt work I think its on railway core itself. I also dont know if this would do anything but I'm curious if enabling static IPs would do anything.
Generally confusing and I'm sorry. Wish I had a better answer on this but it seems like you are doing everything right on your end.
7 months ago
Unless, you are using docker for deployments right?
7 months ago
Well nevermind that wouldnt be an issue
ah, fixed the issue. had to change the nodejs version. railway was using v18
7 months ago
Oh wonderful yeah I had asked about that lol
7 months ago
Hope I was able to help resolve this!
7 months ago
Even if it was just poking at what it wasnt 😂
7 months ago
Oh no worries lol
7 months ago
LTS is the best
yup, it is! i was just trying to check which things where different from my local environment and the railway deployment and final went down till node.js version and fixed it
7 months ago
Crazy that was the issue but good to know for the future. Will ref this thread if I see that issue again.
https://station.railway.com/questions/grpc-http-issue-168f8606
7 months ago
Feel free to close the thread <:picassoface:1281722225593946123>


