Is your code utilizing multiple cores?

I forgot to mention, the application receives a JSON object with multiple image URLs. Each image is processed by two YOLO models. Finally, it returns a JSON with the detection response. Both YOLO models are loaded as soon as FastAPI is deployed to avoid consuming memory during the request. The two models process the same image simultaneously.

Sounds like your bottleneck is going to be your code and or the language itself.

I'm using asyncio.to_thread and asyncio.Semaphore

semaphore = asyncio.Semaphore(20)

gc.collect() #to avoid leak memory

and using batch_size:

async def detect_objects(request: PolesRequest, modelos: List[str], solicitacao_id: int):

response = {solicitacao_id: []}

batch_size = 20

for pole in request.Poles:

images = [photo.URL for photo in pole.Photos]

photo_ids = [photo.PhotoId for photo in pole.Photos]

for i in range(0, len(images), batch_size):

batch_images = images[i:i+batch_size]

batch_photo_ids = photo_ids[i:i+batch_size]

results = await process_images(batch_images, modelos, batch_photo_ids)

pole_results = {"PoleId": pole.PoleId, "Photos": [result.model_dump() for result in results]}

response[solicitacao_id].append(pole_results)

The models simultaneously take an average of 2 seconds per image, but it takes 5 to 7 seconds to access the next image.

This is unlikely an issue with the Railway platform. As brody mentioned, you're probably not doing async correctly and it's doing some synchronous operations in the background instead, or you're not fully utilizing the resources available to you.

Unfortunately we're unable to provide help for application-specific issues -- I'd recommend checking out other communities such as https://stackoverflow.com/ for general programming-related questions.