3 months ago
I followed these docs to get a post-deployment action running. We use the github railway app to run deployments and we use preview environments. In the post-deploy action I'd like to run my e2e tests. The railway format for preview environments involves using the github PR number. When printing the github.event blob I can neither find the PR number (as this event is not a pull request its a deployment status) nor can I find the url associated with the preview environement (this one shocks me). Is the postdeploy step the wrong place to be putting e2e tests?
5 Replies
3 months ago
Hey, normally I put e2e tests before deployment, but adding to post-deployment is also reasonable. What you could do is to add before deployment in the github workflow a code snippet that extracts these information so you will see what railway is using.
- name: Debug - Print full event payload
run: |
echo "Full event context:"
echo '${{ toJson(github.event) }}' | jq .
Normally ths workflow file is in the .github/workflows
dardameiz
Hey, normally I put e2e tests before deployment, but adding to post-deployment is also reasonable. What you could do is to add before deployment in the github workflow a code snippet that extracts these information so you will see what railway is using.- name: Debug - Print full event payloadrun: |echo "Full event context:"echo '${{ toJson(github.event) }}' | jq .Normally ths workflow file is in the .github/workflows
3 months ago
The reason I'm trying to add them post deployment is because the e2e tests are playwright tests that are fired against the preview environment. I know how to print the github event information. But given the pre and post deployment actions are different I'm not sure how this solves my problem. I need to get the preview environment url to set as an environment variable that the playwright config reads to set the baseURL so the tests run against that endpoint.
3 months ago
I found an unbelievable hack of a solution but I'm still looking for something better. In github.event.deployment there is a "environment" property. This environment created by the railway app just so happens to include the pr number. It's in the format "APPNAME / reponame-pr-5". So I can write a shell script with sed that rips that pr-5 off and append a new variable to GITHUB_ENV. Of course this is extremely hacky so I would really love any suggestions on a better solution. I'm shocked the actual url isn't in github.event.deployment
3 months ago
The Railway post-deploy step runs on a deployment status event, not a pull request event, so it doesn’t reliably include the PR number or the preview URL. That’s why you can’t find them in github.event. For preview environments, it’s better to run your E2E tests from a GitHub Actions workflow triggered by pull_request, where the PR number is always available and you can build or look up the preview URL. The post-deploy hook is better for production checks, not PR-based testing.
tric2025
The Railway post-deploy step runs on a deployment status event, not a pull request event, so it doesn’t reliably include the PR number or the preview URL. That’s why you can’t find them in github.event. For preview environments, it’s better to run your E2E tests from a GitHub Actions workflow triggered by pull_request, where the PR number is always available and you can build or look up the preview URL. The post-deploy hook is better for production checks, not PR-based testing.
3 months ago
But I can't guarantee the preview environment will be deployed by the time e2e tests run right? In fact by default Railway wants the deployment of the preview environment to only run after the github actions have passed.