Sqlite 3 on volume linked to laravel project service fails

jeffreysozaTRIAL

6 months ago

I am using sqlite 3 on this project and since i cannot deploy the sqlite file i have locally on my pc for obvious reasons, i need to create one everytime i commit changes to the repo on github, losing the data persistence functionality. So i decided to investigate a bit on the railway help community regarding this matter and found that we can or should use volumes and store/create (1st time) the sqlite 3 file there, thing is that i seem to be having problems with the path i provide to nixpacks build cmd when trying to create that file and run migrations into it.

When i linked the volume to the service, 3 variables were immediately linked to the service, one of them goes as follows: RAILWAYVOLUMEMOUNTPATH ${{RAILWAYVOLUMEMOUNTPATH}}, so i use this one's value to create the file like this: NIXPACKSBUILDCMD="npm run productionbuild:css && touch ${{RAILWAYVOLUMEMOUNTPATH}}/database.sqlite && php artisan migrate --seed --force". Can i get some help with this?

View Deploy details

ⓘ Deployment information is only viewable by project members and Railway employees.

Solved

9 Replies

6 months ago

Looks like your most recent deployment has succeeded?


jeffreysozaTRIAL

6 months ago

Looks like your most recent deployment has succeeded?

It has, though it seems like the migrations were not successfully applied.

Variables important for this db process:

DB_CONNECTION="sqlite"

DB_DATABASE="${{RAILWAY_VOLUME_MOUNT_PATH}}/database.sqlite"

DB_FOREIGN_KEYS="false"

NIXPACKS_BUILD_CMD="touch ${{RAILWAY_VOLUME_MOUNT_PATH}}/database.sqlite && php artisan migrate --seed --force && ls ${{RAILWAY_VOLUME_MOUNT_PATH}} # invalidate cache"

I will show you the logs on the build process:

The linked volume is mounted on this path: /app/database.

On the production server app i get this: Migrations were run but seems like it is applying to another sqlite file, can you further help me on this?

Attachments


6 months ago

The volume would not be available during build, so you are not creating a sqlite file within the database.

You should not need to touch the file at all to begin with.

It's been in my experience that sqlite clients will create the file for you if it doesn't exist.

Please remove the touch command, and move the migration command into your start command.

Make sure to follow the format of <migration command> && <current start command> for the start command.


jeffreysozaTRIAL

6 months ago

The volume would not be available during build, so you are not creating a sqlite file within the database.

You should not need to touch the file at all to begin with.

It's been in my experience that sqlite clients will create the file for you if it doesn't exist.

Please remove the touch command, and move the migration command into your start command.

Make sure to follow the format of <migration command> && <current start command> for the start command.

I was reading the docs about volumes before your response and, indeed, i noticed that volumes are not available at build process, so i followed your advice of moving the migrate command to the start command: Nonetheless, i encountered this error on the deploy logs now: As you can see here, migrations are not fully completed and it fails on looking for the DatabaseSeeder.php file, which is on this directory /app/database, i am not sure why it is looking for it on another directory (vendor/composer), i am sorry if i struggle with, presumably, such basic stuff, it's my first time deploying a sqlite-based project on railway, i've deployed here using MySql and everything cool.

Attachments


6 months ago

Well I guess your code doesn't account for a missing sqlite file, you will want to change that in code so that the file is created if it doesn't exist.


jeffreysozaTRIAL

6 months ago

Well I guess your code doesn't account for a missing sqlite file, you will want to change that in code so that the file is created if it doesn't exist.

I was reading the documentation of railway and found a post dedicated to explain how to deploy a laravel app alongside its migrations, cron jobs and workers, so i decided to follow that; The most important file is the run-app.sh, which content is this:

#!/bin/bash
# Make sure this file has executable permissions, run `chmod +x run-app.sh`

# Create SQLite file if it doesn't exist
php artisan sqlite-db-create

# Run migrations, process the Nginx configuration template and start Nginx
php artisan migrate --force &&  php artisan db:seed --force && node /assets/scripts/prestart.mjs /assets/nginx.template.conf  /nginx.conf && (php-fpm -y /assets/php-fpm.conf & nginx -c /nginx.conf)

As you can see, i use a custom artisan command that i created, which creates a sqlite file if not present.

$dbPath is presumably correct since config('database.connections.sqlite.database') uses the env DB_DATABASE value and i have that one set correctly to point to the volume.

Code is like this:

public function handle()
    {
        if (config('database.default') === 'sqlite') {
            $dbPath = config('database.connections.sqlite.database');
            if (!file_exists($dbPath)) {
                File::put($dbPath, ''); // Create SQLite file

                // Run migrations
                Artisan::call('migrate', ['--force' => true]);
            }
        }
    }

On deploy logs everything seems fine, it apparently finds the sqlite file on the volume, though it fails to find the seeder, i am not sure what to do now, i'm thinking of changing from sqlite to MySql, i would have to minor modifications to some code. Thing is i wanna learn a way around for this problem.

Screenshot of deploy logs:

Attachments


6 months ago

It would definitely be worth switching to MySQL.


jeffreysozaTRIAL

6 months ago

It would definitely be worth switching to MySQL.

Hi again! I managed to make a way around for this problem: (It would be cool if you marked it as a solution to this post)

As we discussed the other day, the app would never find the seeder file when trying to run the migrations, and that's because we had the mount path of our volume as /app/database, which on a normal laravel app contains the seeders, so using that path on the volume kinda overwrites the content of that directory, hence, making the sqlite db file the only one available and present there, no seeders there.

So i thought if i could use another directory deeper in the hierarchy (i'm not sure if i describe it correctly this way lol) to store that sqlite db file and not to interfere with seeders and any other files on the /app/database directory and it worked! So i'll try to explain the process step by step.

Steps to replicate my approach:

1) Set the mount path for your app, remember that this will be the directory in which we will save the sqlite db file we'll use for production, VERY IMPORTANT not to set this path to the same path where seeders, factories, migrations and stuff like that are at as this will delete them and get the error i had before (see previous responses on this post).

2) Create a artisan command using php artisan make:command <name> which creates a SQLite db file when necessary (if there is no file) on two directories that we have on our .env file; They are the following: DB_DATABASE (on railway it usually would be on /app/database/database.sqlite) and another one variable for our volume SQLite db file, whose directory SHOULD NOT be the same as the previous one, in my case, i decided to assign this mount path to my volume: /app/database/sqlite and my .env variable is: DB_DATABASE_VOLUME="${{RAILWAY_VOLUME_MOUNT_PATH}}/database.sqlite"

example code for the command:

environment variables on railway (1st deploy)

3) Add that command to the run-app.sh that we call/invoke.run on the custom deploy command on the setting of our railway app service

note: php artisan check-database-path is another command to, as it says on its name, check where the db files where both the normal db file and the one supposedly stored on the volume were at. Take into account that you can give these commands any convetion you like, you can modify that on the artisan command file when you created the command.

Note: add this command in case you get database readonly error: chmod -R 0777 /app/database

Note 2: using 0777 may not be the best solution as it can cause security issues, if needed any further investigation on a better command should be made.

4) If deploy is successful, you can get rid of the custom build and custom deploy commands and leave them as it is (by default) and set the DB_DATABASE to point to the volume path like this: DB_DATABASE="${{RAILWAY_VOLUME_MOUNT_PATH}}/database.sqlite"

NOTE: THIS IS THE WAY I SOLVED MY PROBLEM, it is definitely NOT the only solution, someone else may have, or may think of another way to solve this.

NOTE 2: Important to follow the tutorial to deploy a laravel that railway published on their docs. Link: Deploy a Laravel App | Railway Docs

Attachments


6 months ago

Awsome, glad you were able to solve this!


Status changed to Solved brody 6 months ago


Sqlite 3 on volume linked to laravel project service fails - Railway Help Station