Railway ignore and HLS .ts files
akatora28
PROOP

2 years ago

For my Adonisjs Typescript app I'm working with video files that are transcoded into HLS, as part of the transcoding process from .mp4, .mov, etc into HLS, the output is .m3u8 playlist files, which are text file manifests containing links to the other playlist and video files and .ts files which are segments of the transcoded video.

For testing purposes I have an already transcoded video saved in my Repo so that I can bypass the transcoding process when testing other parts of my app, but this is causing build errors in Railway, I'm assuming it's because the build process is attempting to parse the .ts files with a type of MPEG transport stream data and treating those as Typescript files. I have those files in a tests/assets/ directory in my repo.

I've seen mention of a .railwayignore file and found the "Configure Watch Paths" option, but that looks like it only controls what files to watch for changes when running builds. I was hoping to find a way to have Railway ignore the tests/ directory altogether in the build process but wasn't sure if that's possible.

27 Replies

2 years ago

could you use .dockerignore?


akatora28
PROOP

2 years ago

I tried that, looks like it still tries to read those files as UTF-8


akatora28
PROOP

2 years ago

722dccb2-d05a-4099-b21f-f525bef38d3c


2 years ago

have you tried a .railwayignore?


akatora28
PROOP

2 years ago

Just tried creating a .railwayignore file too with the same contents ```tests/assets/video_versions/**

tests/assets/video_versions/f9732fd3-605c-4bf3-ab6a-d523e59ab4bf/hls/f9732fd3-605c-4bf3-ab6a-d523e59ab4bf_1080p_3Mbps_00001.ts



akatora28
PROOP

2 years ago

Since those aren't needed in production I can just make some work around fix in testing only to grab them, was just hoping there was a simple fix


2 years ago

then you can run railway up from a GitHub action then it will respect .railwayignore


2 years ago

then it will respect .railwayignore


akatora28
PROOP

2 years ago

Sounds good, I'll dig into that a little more


akatora28
PROOP

2 years ago

I did have a question about connecting to the Redis instance in my Adonisjs app, it seems the app can't connect to the redis.railway.internal host, are you able to take a look at that or should I start a new thread?


2 years ago

ioredis?


akatora28
PROOP

2 years ago

AdonisJS so using their Redis package https://docs.adonisjs.com/guides/database/redis


2 years ago

does it use ioredis?


akatora28
PROOP

2 years ago

Yes, it looks like ioredis is listed as a dependency https://github.com/adonisjs/redis/blob/develop/package.json


2 years ago


akatora28
PROOP

2 years ago

Interesting, looks like switching to the public URL worked so I'll troubleshoot more getting the internal networking to work


2 years ago

have you checked out the link I sent?


akatora28
PROOP

2 years ago

I did, I'm not sure exactly how the adonisjs/redis package is using the ioredis connection under the hood though so I need to dig into that to figure out how to get the connection to work


2 years ago

no need to overthink it, just set family to 0 either in the url or the config options.


akatora28
PROOP

2 years ago

fingers crossed, but looks like adding it after the PORT env variable worked 6379?family=0


akatora28
PROOP

2 years ago

wait, I lied actually, that doesn't work


2 years ago

please send a link to the service that is showing these errors


akatora28
PROOP

2 years ago

okay, finally figured it out it's not documented in the adonisjs/redis docs but there is a config variable for family.


akatora28
PROOP

2 years ago

import env from '#start/env'
import { defineConfig } from '@adonisjs/redis'
import { InferConnections } from '@adonisjs/redis/types'

const redisConfig = defineConfig({
  connection: 'main',

  connections: {
    /*
    |--------------------------------------------------------------------------
    | The default connection
    |--------------------------------------------------------------------------
    |
    | The main connection you want to use to execute redis commands. The same
    | connection will be used by the session provider, if you rely on the
    | redis driver.
    |
    */
    main: {
      host: env.get('REDIS_HOST'),
      port: env.get('REDIS_PORT'),
      password: env.get('REDIS_PASSWORD', ''),
      db: 0,
      family: 0,
      keyPrefix: '',
      retryStrategy(times) {
        return times > 10 ? null : times * 50
      },
    },
  },
})

export default redisConfig

declare module '@adonisjs/redis/types' {
  export interface RedisConnections extends InferConnections {}
}

akatora28
PROOP

2 years ago

thanks for the help pointing to the ioredis family connection thing


2 years ago

no problem!


whosnorman
PRO

a year ago

@El Queso Bandito were you able to resolve the HLS .ts files trying to be read as UTF-8? i'm running into that now and have tried many different ignore file types


Welcome!

Sign in to your Railway account to join the conversation.

Loading...