Hi,

We’re seeing two issues on our RunPod Serverless endpoints and would like help investigating.

1. Text endpoint (vLLM) – worker crashes

• Template: vLLM Serverless, H100 80GB

• Model: kosbu/llama-3.3-70b-instruct-awq (AWQ)

• Config: 1 active worker, RunPod Serverless Worker v1.8.1

Workers sometimes crash mid-request. Logs show:

ERROR [core_client.py:605] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client.

WARNING [multiproc_executor.py:786] WorkerProc was terminated

Followed by:

Exception ignored in: <function AsyncLLM.__del__ at 0x...>

TypeError: 'NoneType' object is not callable

  File ".../vllm/v1/engine/async_llm.py", line 273, in shutdown

Clients then receive 500 - {"error":"Error processing the request"}. This happens intermittently during normal load, not only under spikes.

2. Image endpoint (ComfyUI) – 502 on job-take

• Template: ComfyUI Serverless, RTX 5090

• Config: 1 active worker

Workers sometimes fail when polling for jobs:

Failed to get job. | Error Type: ClientResponseError | Error Message: 502, message='Bad Gateway', 

Questions

1. For the vLLM endpoint: are these crashes a known issue (e.g. OOM, CUDA, or vLLM async shutdown)? Any recommended config changes or versions?

1. For the ComfyUI endpoint: is the 502 on job-take a known RunPod API issue, and is there a recommended retry/backoff strategy?

We’ve added retries on our side for 500/502/503, but we’d like to understand root cause and any server-side mitigations.

Thanks.