My API is unresponsive and i can't access my database through metabase
hazame123
HOBBYOP

3 months ago

I can't access my database through metabase. I also am getting not response from my api thats conmnected to it:

apiAttempt() error - GET (502) /api/v1/productions/eb128da4-fed4-4b4a-8087-8eea001c1c82 {

  type: 'REQUEST_FAILED',

  statusCode: 502,

  message: '{"status":"error","code":502,"message":"Application failed to respond","request_id":"AsbiWAAdRRG1RTxLN8N_Fg"}'

}

I've just uploaded a new migration script for the database and it seems flyway isn't running it, i've double checked its correct. I think maybe a network issue on Railway side?

$10 Bounty

4 Replies

Railway
BOT

3 months ago

Hey there! We've found the following might help you get unblocked faster:

If you find the answer from one of these, please let us know by solving the thread!


3 months ago

Your API is experiencing a 502 because it's crashing on startup, likely blocked by the Flyway migration that isn't executing. When an application fails to start successfully, Railway's proxy cannot reach it and returns a 502 error.

Immediate Steps to Diagnose

  1. Check API deployment logs: In your Railway dashboard, click on your API service -> Deployments tab -> view the most recent deployment logs. Look for:

    • Flyway execution output

    • Database connection errors

    • Timeout errors during migration

  1. Verify database connectivity: Ensure your API can actually reach the database using the correct connection string (should be DATABASE_URL environment variable, not DATABASE_PUBLIC_URL). From your API logs, you should see whether the connection succeeds or fails.

  1. Check Flyway configuration: Confirm:

    • Migration files are in the correct location your app expects

    • File naming follows Flyway conventions (V<version>__<description>.sql)

  1. Check for stuck migrations: If Flyway has already created its history table but marked migrations as failed, it won't retry them on next startup. You may need to:

    • Manually inspect the flyway_schema_history table in your database

    • Reset failed migration states if necessary (this requires manual database access)

    • Review your Postgresql service logs and make sure they seem healthy

Metabase can't connect because your database service is likely unresponsive

Fix the database migrations and Metabase should come back online too.

Next action: Share a sensitive-info-redacted copy of your API and PostgreSQL deployment logs (from the Deployments tab) so we can see exactly where things are going wrong.


ramfat
HOBBY

3 months ago

Hi. I'm having the exact same issue. Internal networking seems to not work at all in my new project. Using public Host & Port / URL works everytime. So far i've tried with a Mysql DB, Redis and MinIO service..
Can we get an official Railway response? It's been causing me massive headaches


mykal

Your API is experiencing a 502 because it's crashing on startup, likely blocked by the Flyway migration that isn't executing. When an application fails to start successfully, Railway's proxy cannot reach it and returns a 502 error.Immediate Steps to DiagnoseCheck API deployment logs: In your Railway dashboard, click on your API service -> Deployments tab -> view the most recent deployment logs. Look for:Flyway execution outputDatabase connection errorsTimeout errors during migrationVerify database connectivity: Ensure your API can actually reach the database using the correct connection string (should be DATABASE_URL environment variable, not DATABASE_PUBLIC_URL). From your API logs, you should see whether the connection succeeds or fails.Check Flyway configuration: Confirm:Migration files are in the correct location your app expectsFile naming follows Flyway conventions (V<version>__<description>.sql)Check for stuck migrations: If Flyway has already created its history table but marked migrations as failed, it won't retry them on next startup. You may need to:Manually inspect the flyway_schema_history table in your databaseReset failed migration states if necessary (this requires manual database access)Review your Postgresql service logs and make sure they seem healthyMetabase can't connect because your database service is likely unresponsiveFix the database migrations and Metabase should come back online too.Next action: Share a sensitive-info-redacted copy of your API and PostgreSQL deployment logs (from the Deployments tab) so we can see exactly where things are going wrong.

hazame123
HOBBYOP

3 months ago

Based on the logs, I can see that my app cannot connect to the PostgreSQL database:

java.net.SocketTimeoutException: Connect timed out org.postgresql.util.PSQLException: The connection attempt failed

This is causing a cascade of failures:

  1. HikariPool can't initialize the connection pool

  2. Flyway can't run database migrations

  3. EntityManagerFactory can't be created

  4. JPA Repositories (like productionCompanyRepository) can't be initialized

  5. Spring beans dependent on the database (like TenantResolver, TenantResolutionFilter) fail to load

  6. Tomcat fails to start

I have made sure the database url is correct postgresql://postgres:xxx@postgres.railway.internal:5432/railway), my database is running (since posting the original message i can now access the database via metabase so the PostgreSQL service is definitely running). The service now crashes every time i redeploy. I've checked all my env variables multiple times to make sure that i have all the correct variables for my application-railway.yml to work:

spring:
  datasource:
    url: jdbc:postgresql://${PGHOST}:${PORT}/${PGDATABASE}
    username: ${PGUSER}
    password: ${PGPASSWORD}
    driver-class-name: org.postgresql.Driver

  jpa:
    hibernate:
      ddl-auto: validate
    show-sql: false
    properties:
      hibernate:
        dialect: org.hibernate.dialect.PostgreSQLDialect
        default_schema: crewdaily

  flyway:
    enabled: true
    baseline-on-migrate: true
    schemas: crewdaily
    default-schema: crewdaily
    create-schemas: true
    locations: classpath:db/migration
    validate-on-migrate: true

logging:
  level:
    org.flywaydb: INFO
    org.hibernate.SQL: DEBUG
    org.hibernate.type.descriptor.sql.BasicBinder: TRACE

This yml file works, as this hasn't changed in a long time and has been working previously.

Any thoughts?


Loading...