Issue connecting with ChromaDB over private network
andreboekhorst
PROOP

a year ago

I have a project with a backend and a ChromaDB (ghcr.io/chroma-core/chroma:0.6.3) instance.

Within my application I connect to the chromaDB with the following code:

remote_client = chromadb.HttpClient(
    host=os.environ["CHROMA_HOST"],
    port=int(os.environ["CHROMA_PORT"]),
    ssl=(os.environ["CHROMA_SSL"].lower() == "true"),
    headers={
        "Authorization": f"{os.environ['CHROMA_TOKEN']}"
    },
)

When using these .env (public) it connects:

CHROMA_HOST=chroma-production-3e37.up.railway.app
CHROMA_PORT=443
CHROMA_TOKEN="Bearer ........"
CHROMA_SSL=True

But wen I want to use internal routing, I cannot get it to work?

CHROMA_HOST="chroma-2ndv.railway.internal"
CHROMA_PORT="80"
CHROMA_TOKEN="Bearer ........."
CHROMA_SSL="False"

If've tried swapping ports from 80 to 443, and put ssl to True and False, but with no solution.

I've also tried to spin up a new DB in the same project - but again, working with public networking, not private.

But I always get the error:
```httpcore.ConnectError: [Errno 111] Connection refused```

these are the ChromaDB ENV config:

ANONYMIZED_TELEMETRY="False"
CHROMA_AUTH_TOKEN_TRANSPORT_HEADER="Authorization"
CHROMA_HOST_ADDR="0.0.0.0"
CHROMA_HOST_PORT="${{PORT}}"
CHROMA_PRIVATE_URL="http://${{RAILWAY_PRIVATE_DOMAIN}}"
CHROMA_PUBLIC_URL="https://${{RAILWAY_PUBLIC_DOMAIN}}"
CHROMA_SERVER_AUTHN_CREDENTIALS="............"
CHROMA_SERVER_AUTHN_PROVIDER="chromadb.auth.token_authn.TokenAuthenticationServerProvider"
CHROMA_TIMEOUT_KEEP_ALIVE="30"
CHROMA_WORKERS="1"
IS_PERSISTENT="True"
PORT="80"

I've also tried (with help of Claude) adding this to the code:

private_domain = "chroma-2ndv.railway.internal"

try:
    response = requests.get(f"http://{private_domain}/api/v1/heartbeat", 
                           headers={"Authorization": os.environ["CHROMA_TOKEN"]},
                           timeout=5)
    print(f"Response: {response.status_code}")
    print(f"Content: {response.text}")
except Exception as e:
    print(f"Error: {e}")

getting:
Error: HTTPConnectionPool(host='chroma-2ndv.railway.internal', port=80): Max retries exceeded with url: /api/v1/heartbeat (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fed85c7f230>: Failed to establish a new connection: [Errno 111] Connection refused'))

Could anyone share what I am missing?

Thanks! Haven't been this lost after I can get help from all the large language models.. :(

ps. I'm assuming using private networking would improve the performance and that there are also costs involved into using public networking for this?

Solved

2 Replies

dev
MODERATOR

a year ago

Hey, were you using a template by chance? Based on the service variables it looks to me like you're using this template https://railway.com/template/kbvIRV

In the template instructions under "Connecting to ChromaDB" it's mentioned that the CHROMA_HOST_ADDR variable needs to be set to :: if you want to use the private network and from the env you provided it doesn't look like you did that.

The reason this step is necessary is because public networking uses IPv4, meaning the host should be an IPv4 address (namely, 0.0.0.0) but since the private network uses IPv6, it has to use an IPv6 host, (which would be ::)

pray1
andreboekhorst
PROOP

a year ago

Thanks for pointing me in the right direction. It is something I did check but forgot to mention (and totally forgot to rtfm).

I did get an issue when using :: as CHROMA_HOST_ADDR

But added this snippet in the code:

private_domain = os.environ.get("CHROMA_HOST", "chroma.railway.internal")

try:
    response = requests.get(f"http://{private_domain}/api/v1/heartbeat", 
                           headers={"Authorization": os.environ["CHROMA_TOKEN"]},
                           timeout=5)
    print(f"Response: {response.status_code}")
    print(f"Content: {response.text}")
except Exception as e:
    print(f"Error: {e}")

which resulted in
Content: {"nanosecond heartbeat":1743680420595329914}

So the connectivity was okay. I then hardcoded all env variables in the code:


remote_client = chromadb.HttpClient(
    host="chroma.railway.internal",
    port="80",  
    ssl=False,  
    headers={
        "Authorization": "Bearer ......."
    },
)

And have it working. Still need to figure out what exactly went wrong with Parsing the values, but at least got something working.

Thanks for the quick support!

saluting face1

Status changed to Solved dev • 11 months ago

Loading...