7 months ago
Hi,
My Node.js application seems to be getting killed and it feels like its getting killed by Railway. Are certain requirements that are necessary to be met to have a long running process? On the local the process runs for 20-25 minutes.
The process fetched data via API from Alpha Vantage, which takes time due to Alpha Vantage rate limitations. While the process is waiting for the data, no other process is waiting for the data and the container is terminated by Railway.
4 Replies
7 months ago
Hey there! We've found the following might help you get unblocked faster:
- 🧵 How to keep alive node js application?
- 🧵 Issuing TLS Certificate – Stuck
- 📚 Deploy Node.js & Express API with Autoscaling, Secrets, and Zero Downtime
If you find the answer from one of these, please let us know by solving the thread!
7 months ago
railway stops containers when there's no active work, so possibly fetching data via a 20-25 min API can look frozen but it isnt. you have to keep them alive by either running them in a worker service with its on queue, a cron trigger. something as simple as:
setInterval(() => console.log[hb] ${new Date().toISOString()}), 30_000);
or you can chunk the job from the API:
const jobs = [/* big list */];
const chunkSize = 50;
for (let i = 0; i < jobs.length; i += chunkSize) {
const chunk = jobs.slice(i, i + chunkSize);
await Promise.all(chunk.map(runJob)); // replace runJob with your function
await new Promise(r => setTimeout(r, 10_000)); // pause 10s between chunks
}
depending on how your script is written, it can look something like:
const fetch = require("node-fetch");
const API_KEY = process.env.ALPHA_VANTAGE_KEY;
const RATE_LIMIT = 5; // 5 requests per minute
const CHUNK = 5;
const sleep = ms => new Promise(r => setTimeout(r, ms));
(async () => {
for (let i = 0; i < SYMBOLS.length; i += CHUNK) {
const batch = SYMBOLS.slice(i, i + CHUNK);
await Promise.all(batch.map(async symbol => {
const url = https://www.alphavantage.co/query?function=GLOBAL_QUOTE&symbol=${symbol}&apikey=${API_KEY};
const res = await fetch(url);
console.log(symbol, await res.json());
}));
console.log("Waiting 10s for next batch...");
await sleep(10_000); // obey rate limit
}
})();
youll need to adapt this for your scripts, otherwise you can running it as a worker, separate from your actual web app, something like:
async function main() {
console.log("Worker started");
while (true) {
await doWork(); // your API or DB task
await new Promise(r => setTimeout(r, 10_000));
}
}
main().catch(console.error)
yeeet
railway stops containers when there's no active work, so possibly fetching data via a 20-25 min API can look _frozen_ but it isnt. you have to keep them alive by either running them in a worker service with its on queue, a cron trigger. something as simple as: `setInterval(() => console.log[hb] ${new Date().toISOString()}), 30_000);` or you can chunk the job from the API: `const jobs = [/* big list */];` `const chunkSize = 50;` `for (let i = 0; i < jobs.length; i += chunkSize) {` ` const chunk = jobs.slice(i, i + chunkSize);` ` await Promise.all(chunk.map(runJob)); // replace runJob with your function` ` await new Promise(r => setTimeout(r, 10_000)); // pause 10s between chunks` `}` depending on how your script is written, it can look something like: `const fetch = require("node-fetch");` `const API_KEY = process.env.ALPHA_VANTAGE_KEY;` `const RATE_LIMIT = 5; // 5 requests per minute` `const CHUNK = 5;` `const sleep = ms => new Promise(r => setTimeout(r, ms));` `(async () => {` ` for (let i = 0; i < SYMBOLS.length; i += CHUNK) {` ` const batch = SYMBOLS.slice(i, i + CHUNK);` ` await Promise.all(batch.map(async symbol => {` ` const url = https://www.alphavantage.co/query?function=GLOBAL_QUOTE&symbol=${symbol}&apikey=${API_KEY};` ` const res = await fetch(url);` ` console.log(symbol, await res.json());` ` }));` ` console.log("Waiting 10s for next batch...");` ` await sleep(10_000); // obey rate limit` ` }` `})();` youll need to adapt this for your scripts, otherwise you can running it as a worker, separate from your actual web app, something like: `async function main() {` ` console.log("Worker started");` ` while (true) {` ` await doWork(); // your API or DB task` ` await new Promise(r => setTimeout(r, 10_000));` ` }` `}` `main().catch(console.error)`
7 months ago
railway stops containers when there's no active work
No we do not.
7 months ago
As Brody mentioned, Railway does not terminate long-running processes, and doing so would contradict Railway's core philosophy. It's possible that your Node.js application is either running out of memory and crashing, or simply crashing without generating any error messages. Examining logs and metrics is the recommended approach in this situation.
