ThreadPoolBuildError
sereenaa
PROOP

a year ago

{ kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }

I have a dagster instance deployed on Railway and using multithreading in Python. I also have dbt assets materializing and I am running out of resources, leading to the deployed instance crashing. I'm wondering if this is due to a resource limit on our Pro Railway plan? I've currently got max 8 workers for the multithreading job and 10 concurrent asset materializations allowed.

24 Replies

sereenaa
PROOP

a year ago

5ee74426-d012-4973-be29-4829cc6c9954


a year ago

can you try lowering the workers and jobs?


antonyip
PRO

a year ago

We fixed this by restarting the container every 4 hours. Our software works by starting threads and processes in the container and we were hitting NPROC_LIMIT on the container. Raising the limit was not possible.

RLIMIT_NPROC=(soft:99999 hard:99999)

We suspect that this was unlimited before and now set to 100k


antonyip
PRO

a year ago

Currently issue is this:

1357868831262904300


a year ago

Where are you trying to migrate it to?


antonyip
PRO

a year ago

from useast to useast metal


a year ago

May I ask why you are trying to revert the Postgres version?


antonyip
PRO

a year ago

the thing was on :latest


antonyip
PRO

a year ago

i suspected the latest was 17


antonyip
PRO

a year ago

so wanted to go back to the correct version


antonyip
PRO

a year ago

think its saved now


a year ago

latest is 16 for our images


antonyip
PRO

a year ago

well, its working now, was in abit of panic when the db was down


antonyip
PRO

a year ago

now i can migrate it again?


a year ago

its possible the target region does not have the available space for that migration


antonyip
PRO

a year ago

hm fair enough


antonyip
PRO

a year ago

anyway to confirm this? I should just wait then?


a year ago

Yes you would want to wait until Metal Volumes are GA for Pro users, currently it is only GA for Hobby users


a year ago

hello, it is possible we just fixed this, could you try again?


antonyip
PRO

a year ago

Yup sure


antonyip
PRO

a year ago

nope, still same issue


a year ago

thank you for trying


sereenaa
PROOP

a year ago

hello there, we are continuing to get this error

alembic.util.exc.CommandError: Version table 'alembic_version' has more than one head present; please use get_current_heads()


a year ago

I'm really sorry but I'm not too sure how much we can help you with that as it's unlikely to be related to the Railway platform


Loading...