How to wait deployment with health check in GitHub Actions before proceeding

benwkHOBBY

a year ago

Hi everyone,

I’m reaching out for some advice on managing deployments with the Railway CLI in GitHub Actions. My goal is to run automated API tests and report on the pr once the deployment is done. However, the railway up command completes too quickly in the workflow, not actually waiting for health checks.

Because of this, my API tests tend to start before the environment is fully ready. I've tried using project-level webhooks to trigger the tests post-deployment, but managing this for multiple repo services within the same project gets tricky.

Does anyone have tips or methods for this? Maybe there's a workaround or a feature I'm missing?

Thanks for the help!

Ben

25 Replies

a year ago

I think you are looking for the --ci flag?

The command will then wait for the deployment to either fail or succeed, and exit accordingly.


benwkHOBBY

a year ago

Are you referring to doing it like this: railway up --service ${{ service }} --ci?

It looks like what I want. Let me give it a try.


a year ago

Yep!


benwkHOBBY

a year ago

no lucky for me, a particular CLI version for --ci?

error: unexpected argument '--ci' found

  tip: to pass '--ci' as a value, use '-- --ci'

Usage: railway up [OPTIONS] [PATH]

For more information, try '--help'.

a year ago

What version is the action using? the latest is 3.7.0


benwkHOBBY

a year ago

Ah, just realized that the version on npm is still 3.5.2. I've updated it to bash <(curl -fsSL [cli.new](cli.new)) in the action.

It no longer throws errors, but unfortunately, it did not wait for the deployment results.


a year ago

Are you able to share your action and it's logs?

Do you have a source set on your Railway service?


benwkHOBBY

a year ago

The workflow is very straightforward:

name: Railway Deployment

on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
      env-url:
        required: false
        type: string
      service:
        required: true
        type: string
      pr-id:
        required: false
        type: string

jobs:
  deploy:
    timeout-minutes: 10
    name: Deploy to ${{ inputs.environment }} environment
    runs-on: ubuntu-latest
    environment:
      name: ${{ inputs.environment }}
      url: ${{ inputs.env-url }}
    permissions:
      contents: read
      deployments: write
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Install Railway
        run: bash <(curl -fsSL cli.new)
      - name: Deploy
        run: railway up -c -s ${{ inputs.service }} -e ${{ inputs.environment }}
        env:
          RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}

Logs screenshot attached, and it only took 2 seconds.

For the service, repo has been set as the source, but no branch triggers are in place.

I hope this helps. I'll now switch to another project for testing. Many thanks.

Attachments


a year ago

What's with the -c flag on railway up? did you mean to type --ci?


benwkHOBBY

a year ago

-c equal to --ci I think, just followed:

Options:
  -d, --detach                     Don't attach to the log stream
  -c, --ci                         Only stream build logs and exit after it's done
  -s, --service           Service to deploy to (defaults to linked service)
  -e, --environment   Environment to deploy to (defaults to linked environment)
      --no-gitignore               Don't ignore paths from .gitignore
      --json                       Output in JSON format
  -h, --help                       Print help
  -V, --version                    Print version

a year ago

Looks like it, okay I'll have to ask someone who has used --ci
in an action before and I'll get back to you.

Does the --ci flag work as intended locally?


benwkHOBBY

a year ago

Locally, with or without --ci, it will output logs until the health check.


benwkHOBBY

a year ago

Anyway thanks, bro. It's getting late on my end, will continue to dig tomorrow.


a year ago

Locally without --ci it should print build logs and deploy logs and continue streaming the deploy logs for around 10 minutes.

I tested locally with --ci and it exits with Deploy complete after Healthcheck succeeded! so it is working as intended there.


benwkHOBBY

a year ago

Yes, I've noticed that too.

I attempted to set up a clean environment locally and found that the issue lies in the fact that logs cannot be output when running through RAILWAY_TOKEN. This should be a matter of insufficient permissions.

railway up --ci -s ai -e test
  Indexed
  Compressed [====================] 100%
  Uploaded                                                                                                                                                                                                             Build Logs: https://railway.app/project/be84df68-6729-47b0-8f5c-9017c5be6065/service/587c3ee1-c526-4e77-84c7-b4971fe92b58?id=6af9de90-cab2-416d-b198-d637ec7d5fb2&
Failed to stream build logs: Unauthorized. Please login with `railway login`

a year ago

What kind of token are you supplying?


benwkHOBBY

a year ago

Project Token, followed by https://docs.railway.app/guides/cli#project-tokens

I also tried using the API Token at the account level, but it didn't work with the CLI.


a year ago

Are you sure the token you have provided is for the test environment?


benwkHOBBY

a year ago

Yeah, for sure. I created a new one temporarily, and the same permission error still exists. However, the deployment was successful, just unable to access the log stream.


a year ago

Okay, so that's the actual issue here, it's not that --ci doesn't work, it is that the command exits.

When you supplied the account token, did you set it as a RAILWAY_API_TOKEN environment variable?

RAILWAY_TOKEN -> Project Token

RAILWAY_API_TOKEN -> Account Token

Edit, this does not work, you can't specify an API token because then you have no way to specify the project, as the project is gathered from the project token or by interactively running link, sorry for the confusion.

It seems the permissions aren't correct for project tokens on Railway's backend, the CLI has no problem using the project token to upload the code, but then doesn't have the authorization to stream the deployments' build logs, I've asked the team about this.


benwkHOBBY

a year ago

It seems the permissions aren't correct for project tokens on Railway's backend, the CLI has no problem using the project token to upload the code, but then doesn't have the authorization to stream the deployments' build logs, I've asked the team about this.

This is exactly what I had in mind. Thank you so much for your help.


a year ago

Small update, team is now aware and it is in triage, will let you know when I have more information.


benwkHOBBY

a year ago

👍 Many thanks. I have temporarily switched our automated testing to manual triggering until this issue is resolved.


finischerHOBBY

10 months ago

Hey, ist there an update on this problem?

I have exactly this use case to get the deployement exit code before going on to the next pipeline step.


10 months ago

It was implemented but the feature is currently broken and doesn't work.


How to wait deployment with health check in GitHub Actions before proceeding - Railway Help Station