Tuning Websockets (Fastify) with Twilio and OpenAI Realtime API
growingtwice
FREEOP

a year ago

Hi everyone. I'm trying to run a fairly typical voice conversation system using Twilio and OpenAI realtime API via websockets (using Node fastify) and Redis. This ran well on my local PC for testing with Ngrok, but I'm hoping deploying on Railway will give super smooth websockets and responses.

I'm wondering if my application is right for Railway environment.

The first test, using the provided domain/url partially worked. OpenAI could hear the media stream from the caller, but the caller could not hear what OpenAI sent back.

Things improved by using my own domain/url and the conversation within the call worked well. However in more than half the test calls, the caller doesn't hear the greeting, and towards the end of the call, the caller no longer hears the OpenAI responses.

Calls are about 4-7 mintues long and each turn of conversation is between 3 seconds and, perhaps 60 seconds of speaking.

I'm assuming these are related to websockets response times. Are there ways to improve the websocket response times in railway?

Project id is 0e6a359f-9099-49db-ab98-32669e6eb67a

Thanks!

5 Replies

a year ago

.


a year ago

project id please


growingtwice
FREEOP

a year ago

Hi. Project id is 0e6a359f-9099-49db-ab98-32669e6eb67a


growingtwice
FREEOP

a year ago

Hi Brody, Thanks for the response. Hi. Project id is 0e6a359f-9099-49db-ab98-32669e6eb67a

I've been looking for possible restrictions or timeouts on having open websockets during the conversation (OpenAI Realtime, Twilio media stream) and how Railway proxies (?) may be an issue. Or timeouts? In case of timeouts, I tried some pings, but users rarely talk longer than 20 seconds, and the AI streams rarely last more than 40 seconds. The script hangs up if there is 15 seconds of silence, so I doubt I'm running into timeouts. But I'm guessing here 🙂

Hope to hear from you.


a year ago

Hello,

Given this is unlikely to be a platform issue and we have no other reports of users having issues with WebSocket's, we would need a minimally reproducible example from you, something without any AI or anything complex that still showcases the issue in a reliable way.

Without that we would not be able to look into this issue.


Loading...