Hey there Dave,

A well thought out message deserves a well thought out response.

We have now we have a line item within our quarterly retro that talks about product cohesion. But one of the pieces that we have struggled with as developers on this product is making sure that all the pieces of our features work well together.

You could probably tell, three different engineers worked on three different parts of variable experience.

Let's break it down.

1. The Railway team prefers that all services are all listed the same project. The best way to think about it is your Project canvas is representation of the fabric of the network and your would want everything related to your project scoped that way.
2. This is where the first scope comes in, service variables are for secrets that are used for within the service OR that need to be referenced by another service, e.g. connect to a Database.
2.5. What you found that that if you had a second DB in the project, our autocomplete wouldn't have a great time getting your intent. This I will chalk it up to a UX nit on our part, and will track this via Linear ticket on our end. I think stuff gets fuzzy when you have two of the same DB type with similar keys. You can tell the order when the features got shipped. (Variable references -> Multi DBs in the same project.)
3. Shared variables actually came before variable references, this feature was intended to be put in place for API keys that are the same across services, e.g. CLOUDINARY_API_KEYS, STRIPE_API_KEY, you get the gist. We do have some users make a shared variable for a DB only to realize they can just reference it, seems like it caught you- we'll update the docs as soon as we can.
4. The flow doesn't matter but we added arrows to the canvas to help model it out but it's not perfect... yet. I will take this back to our designer who working on the new canvas experience.

Anyway, hope this wall of text helps, and if not, we'll be on the docs asap to unblock you and many others.

(Thanks for writing in and hope you see you around these parts.)

Hello,

Thanks very much for the reply! I thought I posted on the community forum, but really appreciate someone from Railways jumping in.

So what I get from that is that variables were developed incrementally -- well, that's the nature of development I work as a product manager in my day job (not that you'd know from how terribly I have understood this :D) and I understand the issue about cohesion you describe. A big part of my role is cohesion, a unified view.

The Railway team prefers that all services are all listed the same project. The best way to think about it is your Project canvas is representation of the fabric of the network

That by itself is an awesome explanation. In the docs I see things like projects and environments explained, but what you just wrote connects the concepts or explains the expectations. (I may have just missed this in the docs, btw.) Thankyou; that really helped.

So: I have misunderstood the Railway canvas. I re-read the Getting Started and Basics guides just now, and what I've done is that I clicked Create while I had a project open, but created (I think) a second project by accident. So I have my database in one project and db-consumer service in another. What I wanted to do was add the database service to the same project as the db-consumer.

From a UI flow POV, I'm not sure how this works? With a project open and my service visible in the canvas, I clicked Create in the top right, pointed it at my GitHub repo, and got a service -- but it seems to be running in a different project. That doesn't match my expectations, which were that clicking Create would add something to the same "space" (project.) I do have only one object in the canvas in two projects, but that's not what I intuited would happen and I had continued as though they were two services that should be able to see each other.

May I make a UI suggestion? You have a breadcrumb bar at the top:

[Railway icon] > [project] > [environment]

Because a project contains services, and services have environments, I find this a little confusing. In fact, clicking the Railway icon takes me to the workspace which has projects, but I actually have two workspaces and they are not in the breadcrumb bar, so I can't get fully upwards in navigation using the breadcrumbs.

Something like:

[Railway icon - ie dashboard] > [workspace] > [project]

would make more sense to me.

Then within a project, you have the various services on the canvas. I am still new and testing out environments so haven't checked how these are toggled yet, but: if the environment is per-service, choosing that would be within the box on the canvas representing the service, but if the environment is toggled per-project (this would make sense to me because I'm unlikely to want to mix, eg, production db config with test some-other-service config) then the UI option for that should be on the canvas itself applying to the whole canvas (project.) Either way I am not convinced the environment belongs in the same spatial / tree navigation as the rest.

I personally would add a large back arrow button permanently on the top left of the canvas (stationary as you move the canvas around) which navigates back/up to the workspace, and project config settings like environment permanently on the top right of the canvas, which apply to everything you see in the canvas, or change the view.

For explanation: I'm approaching this from the POV of other dev environments where you'll have projects but change active configurations. In that similar design, this treats the environment you're working in as the active configuration. Thus, it's an option to toggle, and the other environments for the same project are still present, just not the ones you're working on currently.

I'd also add a menu separator and then a Create menu item below, so if you wanted another project you'd click the project breadcrumb, see the menu listing your projects, and below that a Create New Project and probably some other manage (where you could delete, etc) menu items. Thus navigation for a context and creation in that context can be found in the same related space onscreen.

We do have some users make a shared variable for a DB only to realize they can just reference it, seems like it caught you- we'll update the docs as soon as we can.

Am I right that this means that from the db-consumer service, I need to reference (create a reference variable in that db-consumer service) that references the variable from the db service?

Btw, sorry for the super-long text. I'll try to be shorter next time :)

Look at this! (In the attached screenshot.) By right-clicking in the project I could create a new service in the project (this is what I expected the Create button to do) and now in my db-consumer service, I get a list of the variables in the db service.

This is what I was after. And it all comes down to misunderstanding the workspace/project/service system. Once again, that sentence that I quoted above that I called "awesome" -- that's what made it click. Thankyou.

I was real confused reading the original message, thinking to myself, why aren't the variables working for them, they should be???

But now that I hear you've deployed the services into two different projects makes sense as to why variables weren't working for you.

What you have done is gone back to the dashboard and created a new project from GitHub instead of clicking the create button within a project.

Super easy to mistakenly do instead of using the create button within a project, and in my option, the user shouldn't be able to create deploys from the dashboard as it makes it easier to create separate projects for each deployment.

Either way, thanks for the other suggestions!!

P.S. This may be a public post in the community forums, but you're Pro so you are guaranteed an answer from the team!

What you have done is gone back to the dashboard and created a new project from GitHub instead of clicking the create button within a project.

Gotcha. I certainly was trying to create something from GitHub... just not a new project.

We can close this thread, I think. Thanks for the help :)