rust deploy uses 10x more ram than local dev

hey guys, i have a rust api that is using 4-8gb of ram suddenly,

now for rust that's unheard of considering it used to chill at 200mb +/-

when running cargo run --release --package x
locally i get out to a max of 1.5gb ram (this includes compiling)

yet on railway it ticks out to 8gb at peak (this includes compiling)

takes roughly 26 minutes to compile, build, deploy on railway

There's a total of 950 +/- crate dependencies

projectId = 7c6a1d9f-d3d2-4808-96b1-10c8df175503

No deploy that happened in that weird spike, nor am i able to reproduce the low memory usage anymore

179 Replies

2 years ago

dockerfile or nixpacks?


nixpacks


// nixpacks.toml
providers = ['rust']

[phases.build]
cmds = ['echo building xylex image', 'cargo build --package xylex']

[phases.test]
cmds = ["cargo test --package xylex"]
dependsOn = ["build"]

[start]
cmd = 'cargo run --package xylex'

[variables]
NIXPACKS_INSTALL_CACHE_DIRS = './nixpacks_cache'
NIXPACKS_BUILD_CACHE_DIRS = './nixpacks_cache'

2 years ago

you should not do cargo run as the start command, you should be building a release and then running the resulting binary


i had a cargo run --release


but i just changed it a few min ago


2 years ago

let's see your new nixpacks.toml file please


2 years ago

additionally, @salvage your input here would be greatly appreciated


// nixpacks.toml
providers = ['rust']

[phases.build]
cmds = ['echo building xylex image', 'cargo build --release --package xylex']

[phases.test]
cmds = ["cargo test --package xylex"]
dependsOn = ["build"]

[start]
cmd = './target/release/xylex'

[variables]
NIXPACKS_INSTALL_CACHE_DIRS = './nixpacks_cache'
NIXPACKS_BUILD_CACHE_DIRS = './nixpacks_cache'

2 years ago

yeah that looks appropriate now


yeah well i used to do it like this in the start but had tremendous amount of issues as you might remember


it would just brick and never deploy unless done via the cli


2 years ago

I do remember, did you also have those variables set? I don't see any purpose to them and can see them causing issues


yes


no


i hadn't set them yet


this is just me trying to get it working in a normal way


2 years ago

for good measure, remove them please


the nixpacks_cache ?


2 years ago

and then let me know if you run into further issues


2 years ago

yes, both those variables you have there


okay, gotcha 2 min


the target could be up to 20gb in some weird c folder hence i deferred it to the cache folder


2 years ago

I'm assuming the default cache stuff nixpacks does is sufficient


compiling rn


same issue as before


on railway and on nixpacks locally


local

1224469832414531600


2 years ago

just freezes there?


yes


2 years ago

god


i love railway


2 years ago

the rust provider in nixpacks honestly is shit


but rust


has been very painful to work with on railway


2 years ago

yeah i've moved to cargo-chef


2 years ago

in a dockerfile


i was reading about that yesterday


2 years ago

haha milo beat me to it


2 years ago

much better then nixpacks


2 years ago

agree with Milo here


2 years ago

also this issue looks unrelated to nixpacks


right okay, so you advise me to do the same


im not sure, it stalls local and on railway


2 years ago

nixpacks or


yes


2 years ago

okay nevermind


2 years ago

this is a nixpacks issue 😂



2 years ago

im not sure why it is making it stall that bad though


2 years ago

yes, I'm not a rust dev but have heard great things about cargo chef and when nixpacks is painful (almost always) dockerfiles are the way to go


we still chilling here


2 years ago

from yours truly


yeah it was the exact same issue months ago


somehow the cli fixed it


2 years ago

just use cargo-chef, see if that helps


omw


2 years ago

yep cargo chef in a dockerfile for sure


2 years ago

ping me if this happens again (or get brody to lol)



yeah as funnily enough we went to rust cus of low resource usage


whahaha


2 years ago

no you dont need to install it


2 years ago


you just make the recipe.json ?


i am


2 years ago

you do all that in the dockerfile



ah i see


2 years ago

jsut look at the example


2 years ago

this bit

1224470945012387800


should be able to copy most of that


2 years ago

yup



2 years ago

delete ur nixpacks.toml


2 years ago

make sure the Dockerfile is named Dockerfile


yes wait im showing u something




2 years ago

thats just nixpacks doing extra stuff


2 years ago

dont worry about that


2 years ago

please delete ur nixpacks,toml


2 years ago

you somehow needed up with two rust providers


2 years ago

but yeah, Dockerfile time


well i think it's a bug in NixPacks when cargo workspaces are enabled


as xylex_wss is not the main bin


it starts building a random workspace and the core bin


2 years ago

nixpacks and rust dont mix unfortunately


going to docker now ig


2 years ago

yup


fucking hate docker time consuming


2 years ago

not really


2 years ago

this will be easy


2 years ago

yeah you won't need any kind of complex Dockerfile


okay so i changed these from app to xylex

1224473010250186800


anything besides that ?


my cargo is called xylex


2 years ago

what is the binary u wanna


2 years ago

okay


2 years ago

cool


2 years ago

should be good


railway autodetects this yes


2 years ago

yes



2 years ago

let us know how it goes


2 years ago

and please cancel those old deployments


whahah yes i will



i am guessing one of these should've been app still


2 years ago

I assume that means you need to install openssl


bros too experienced


2 years ago

RUN apt update -y && apt install openssl libssl-dev -y


2 years ago

put that in the line before the ENTRYPOINT instruction


2 years ago

perfect


man idk where milo came from but they needa give him a raise


bro's got shortcuts to paste solutions


2 years ago

Milo is the rust professional around here


what a king


2 years ago

fr


2 years ago

just from my personal rust app dockerfile haha


are you in the railway dev team milo


2 years ago

he's not, just a community mod


i see, so it is personal projects/companies that u used rust in


2 years ago

yeah, some enterprise things as well


2 years ago

rust is really good and scalable in my experience


yes we switched in january it is amazing


2 years ago

haha



i am unsure if the ssl install was the right call


certificate verify failed"


2 years ago


2 years ago

what dependencies are u using


2 years ago

might need ca-certificates too? just a guess


shouldn't matter it deploys fine on ubuntu server


2 years ago

can you send ur cargo.toml


yes


2 years ago

I mean this is not the same at all though


2 years ago

are you calling reqwest directly


alright brace for impact guys


// cargo.toml 
[package]
name = "xylex"
version = "0.1.0"
edition = "2021"


[dependencies]
polars = {version = "0.35.0", features = ["lazy", "rolling_window"] }
reqwest = { version = "0.11.23", features = ["json", "blocking"] }
tokio = { version = "1.35.1", features = ["full", "macros", "rt-multi-thread"] }
chrono = { version = "0.4", features = ["serde"] }
serde  = { version = "1.0.196", features = ["derive"] }
supabase_rs = "0.2.2"
thiserror = "1"
rocket = "0.5.0"
serde_json = "1.0.111"
dotenv = "0.15.0"
mail-send = "0.4.6"
mail-builder = "0.3.1"
rsi = "0.1.2"
csv = "1.1"
arrow = "*"
serenity = { version = "0.12", default-features = false, features = ["client", "gateway", "rustls_backend", "model", "collector"] }
clokwerk = "0.4.0"
image = "0.24.8"
rusttype = "0.9.3"
rand = "0.8.5"
qrcode = "0.13"
async-trait = "0.1.77"
futures = "0.3.3"
imageproc = "0.23.0"
tokio-tungstenite = { version = "0.21.0", features = ["native-tls"] }
serde_yaml = "0.9.31"
lazy_static = "1.4.0"
trading_sessions = "0.1.2"
md-5 = "0.10.6"
hex-literal = "0.4.1"
fibonacci_retracement = "0.1.0"
futures-util = "0.3.30"
futures-channel = "0.3.30"
url = "2.5.0"
sha256 = "1.5.0"
strum = "0.26"
strum_macros = "0.26"
sled = "0.34.7"

xylex_spread_tracker = { path = "./xylex_spread_tracker" }
xylex_card_generator = { path = "./xylex_card_generator" }
xylex_hash_router = { path = "./xylex_hash_router" }
xylex_scheduler = { path = "./xylex_scheduler" }
xylex_discord = { path = "./xylex_discord"}

[dev-dependencies]
anyhow = "1"

[profile.dev]
opt-level = 0
incremental = true


[profile.release]
opt-level = 3

[workspace]
members = [
    "xylex_wss",
    "xylex_card_generator",
    "xylex_spread_tracker",
    "xylex_scheduler",
    "xylex_hash_router",
    "xylex_discord"
]
[package]
name = "xylex_card_generator"
version = "0.1.0"
edition = "2021"

[dependencies]
image = "0.25.0"
imageproc = "0.24.0"
reqwest = "0.12.2"
rusttype = "0.9.3"
serde = "1.0.197"
serde_json = "1.0.114"
serde_yaml = "0.9.34"
[package]
name = "xylex_discord"
version = "0.1.0"
edition = "2021"

[dependencies]
serenity = { version = "0.12", default-features = false, features = ["client", "gateway", "rustls_backend", "model", "collector"] }
tokio = { version = "1.35.1", features = ["full", "macros", "rt-multi-thread"] }
chrono = { version = "0.4", features = ["serde"] }
serde  = { version = "1.0.196", features = ["derive"] }
thiserror = "1"
serde_json = "1.0.111"
dotenv = "0.15.0"
serde_yaml = "0.9.31"
reqwest = "0.12.2"

[dev-dependencies]
anyhow = "1"

[profile.dev]
opt-level = 0
incremental = true

[profile.release]
opt-level = 3
[package]
name = "xylex_hash_router"
version = "0.1.0"
edition = "2021"

[dependencies]
dotenv = "0.15.0"
serde_json = "1.0.115"
supabase_rs = "0.2.2"
[package]
name = "xylex_spread_tracker"
version = "0.1.0"
edition = "2021"

[dependencies]
regex = "1.10.4"
reqwest = "0.12.2"
serde = "1.0.197"
serde_derive = "1.0.197"
serde_json = "1.0.115"
serde_yaml = "0.9.34"
supabase_rs = "0.2.2"
tokio = "1.36.0"
[package]
name = "xylex_wss"
version = "0.1.0"
edition = "2021"

[dependencies]
amqprs = "1.5.3"
serde = "1.0.197"
serde_json = "1.0.114"
serde_yaml = "0.9.34"
tokio =  "1.36.0"

2 years ago

you can attach files lmao


oh


well this works too


2 years ago

add rustls-tls as a feature of reqwest


2 years ago

also, half these depenendices are obsolete and can be replaced by something better, not needed at all, or stdlib


i mean bro


im sure


but i'm not sure if refactoring 155k lines of rust is something i wanna do tonight


2 years ago

i mean, you don't need strum-macros if you have strum, lazy-static is just stupid these days, dotenv should use dotenvy and thats just to name a few


2 years ago

anyways


2 years ago

do this please


yes its deploying


2 years ago

okay cool


2 years ago

damn Milo knows his stuff


yeah but what's the difference between dotenv and dotenvy


2 years ago

dotenvy is maintained and isn't.. bad



i see ill make a linear ticket ig


2 years ago

same api, literally just changing crate name


2 years ago

anyways!


same issue lmfao


2 years ago

screenshot of error



"certificate verify failed", file


2 years ago

well then i dont know why


2 years ago

show me the line of code


2 years ago

that is causing this erorr


every single one that does an api request


2 years ago

maybe?


2 years ago

show me please


pub async fn discord_logs(
    log_message: &str
) -> Result<(), Box> {
    let client: Client = Client::new();
    let webhook_url: String = env::var("DISCORD_WEBHOOK_URL").unwrap();
    let current_time: String = Utc::now().to_string();

    client.post(&webhook_url)
        .body(json!({
            "content": format!(
                "**xylex-daemon @ {} | **{}",
                current_time, log_message
            )

        }).to_string())
        .header(
            "Content-Type",
             "application/json"
        )
        .send()
        .await?;

    Ok(())
}

all this shouldn't matter as it works fine on any other deploy it's just railway can't build it unless it's 4-8gb ram


2 years ago

well thats norrmal about high ram amounts


2 years ago

yeah no clue, you aren't directly telling it to use openssl and the features make it use rustls so im lost


2 years ago

sorry


ig so but only 10-20k lines r being deployed on railway and idle usage used to be 200mb or so


2 years ago

sorry


2 years ago

idk


thanks for the help though guys, i appreciate your time have a good night


Loading...