Hey Jelly! Have you had success deploying this vLLM?

yeah, I've successfully ran "gpt-2"

1. Set MODEL_NAME to any supported model in your Railway environment variables:
   MODEL_NAME="gpt2"

2. The api-gateway listens for requests, enforces authentication with API_KEY, and forwards them to your vLLM backend.

3. OpenAI-compatible APIs let clients and apps connect with zero code changes.

You can find more info here : https://railway.com/deploy/gpt-vllm

Hey! Thank you for the response.

I'm not a developer so pardon my ignorance with this next question....

Where or how do I obtain the API and URL?

API_KEY is a secret value, something like a secret password you use to talk with vLLM server.

"vLLM server" is where our model currently loads, runs, and does all the hard work.
"api-gateway" is api layer who is a bouncer who just verifies request with API_KEY.
provide it correct value and your request is processed by vLLM server and answered

for VLLM_URL use "http://vllm-server:8000"

I've updated the template so now you don't have to provide that value.
after deploying go to api-gateway, and check there is a domain generated.
if not go to "Settings" and scroll down to find "Public Networking" and clicj "Generate Domain".

this will create a domain, take that value and go to "/docs" route of it.
you will get swagger ui.

If you're confused you can delete and update or redeploy template. I've updated it so you only have to provide this two values
MODEL_NAME & API_KEY . for model name, check https://docs.vllm.ai/en/stable/models/supported_models.html.
where you find the values that you need to use. for ex, "gpt2"

API_KEY is anything that you can use that is a secret which will be used for authentication.

api-gateway is basic template that you can check and edit as per your requirement.
code is here at https://github.com/jellydeck/gpt-oss

hope this helps :)