Railway ignore and HLS .ts files

akatora28PRO

7 months ago

For my Adonisjs Typescript app I'm working with video files that are transcoded into HLS, as part of the transcoding process from .mp4, .mov, etc into HLS, the output is .m3u8 playlist files, which are text file manifests containing links to the other playlist and video files and .ts files which are segments of the transcoded video.

For testing purposes I have an already transcoded video saved in my Repo so that I can bypass the transcoding process when testing other parts of my app, but this is causing build errors in Railway, I'm assuming it's because the build process is attempting to parse the .ts files with a type of MPEG transport stream data and treating those as Typescript files. I have those files in a tests/assets/ directory in my repo.

I've seen mention of a .railwayignore file and found the "Configure Watch Paths" option, but that looks like it only controls what files to watch for changes when running builds. I was hoping to find a way to have Railway ignore the tests/ directory altogether in the build process but wasn't sure if that's possible.

0 Replies

7 months ago

could you use .dockerignore?


akatora28PRO

7 months ago

I tried that, looks like it still tries to read those files as UTF-8


akatora28PRO

7 months ago

722dccb2-d05a-4099-b21f-f525bef38d3c


7 months ago

have you tried a .railwayignore?


akatora28PRO

7 months ago

Just tried creating a .railwayignore file too with the same contents tests/assets/video_versions/** tests/assets/video_versions/f9732fd3-605c-4bf3-ab6a-d523e59ab4bf/hls/f9732fd3-605c-4bf3-ab6a-d523e59ab4bf_1080p_3Mbps_00001.ts and same issue there, looks like the files are still being viewed as part of the source code in build


akatora28PRO

7 months ago

Since those aren't needed in production I can just make some work around fix in testing only to grab them, was just hoping there was a simple fix


7 months ago

then you can run railway up from a GitHub action then it will respect .railwayignore


7 months ago

then it will respect .railwayignore


akatora28PRO

7 months ago

Sounds good, I'll dig into that a little more


akatora28PRO

7 months ago

I did have a question about connecting to the Redis instance in my Adonisjs app, it seems the app can't connect to the redis.railway.internal host, are you able to take a look at that or should I start a new thread?


7 months ago

ioredis?


akatora28PRO

7 months ago

AdonisJS so using their Redis package https://docs.adonisjs.com/guides/database/redis


7 months ago

does it use ioredis?


akatora28PRO

7 months ago

Yes, it looks like ioredis is listed as a dependency https://github.com/adonisjs/redis/blob/develop/package.json


7 months ago


akatora28PRO

7 months ago

Interesting, looks like switching to the public URL worked so I'll troubleshoot more getting the internal networking to work


7 months ago

have you checked out the link I sent?


akatora28PRO

7 months ago

I did, I'm not sure exactly how the adonisjs/redis package is using the ioredis connection under the hood though so I need to dig into that to figure out how to get the connection to work


7 months ago

no need to overthink it, just set family to 0 either in the url or the config options.


akatora28PRO

7 months ago

fingers crossed, but looks like adding it after the PORT env variable worked 6379?family=0


akatora28PRO

7 months ago

wait, I lied actually, that doesn't work


7 months ago

please send a link to the service that is showing these errors


akatora28PRO

7 months ago

okay, finally figured it out it's not documented in the adonisjs/redis docs but there is a config variable for family.


akatora28PRO

7 months ago

import env from '#start/env'
import { defineConfig } from '@adonisjs/redis'
import { InferConnections } from '@adonisjs/redis/types'

const redisConfig = defineConfig({
  connection: 'main',

  connections: {
    /*
    |--------------------------------------------------------------------------
    | The default connection
    |--------------------------------------------------------------------------
    |
    | The main connection you want to use to execute redis commands. The same
    | connection will be used by the session provider, if you rely on the
    | redis driver.
    |
    */
    main: {
      host: env.get('REDIS_HOST'),
      port: env.get('REDIS_PORT'),
      password: env.get('REDIS_PASSWORD', ''),
      db: 0,
      family: 0,
      keyPrefix: '',
      retryStrategy(times) {
        return times > 10 ? null : times * 50
      },
    },
  },
})

export default redisConfig

declare module '@adonisjs/redis/types' {
  export interface RedisConnections extends InferConnections {}
}

akatora28PRO

7 months ago

thanks for the help pointing to the ioredis family connection thing


7 months ago

no problem!


whosnormanHOBBY

3 months ago

@El Queso Bandito were you able to resolve the HLS .ts files trying to be read as UTF-8? i'm running into that now and have tried many different ignore file types


Railway ignore and HLS .ts files - Railway Help Station