Service with no public networking showing high ingress on metrics screen
stephen-allison
PROOP

4 months ago

Hi,

I have a service in my project that is a django-q worker - it pulls queued tasks from Postgres and uses resend to send emails. The emails are not large, and in total over the last week or so only a few hundred have been sent.

If I look at the 'metrics' tab of this service the 'Public Networking' chart shows a consistent ingress of 10-40MB in each 4 hour time block that I'm struggling to explain - the service does not have public networking enabled. How can I investigate further what this traffic is?

Thanks!

Steve

Solved

11 Replies

ray-chen
EMPLOYEE

4 months ago

Any network traffic (such as sending API requests and receiving responses) counts as egress/ingress (respectively).


Status changed to Awaiting User Response Railway 4 months ago


ray-chen

Any network traffic (such as sending API requests and receiving responses) counts as egress/ingress (respectively).

stephen-allison
PROOP

4 months ago

Thanks - is there a way to see what the incoming traffic (presumably responses from Resend?) is? - it seems a lot of data to me.


Status changed to Awaiting Railway Response Railway 4 months ago


stephen-allison

Thanks - is there a way to see what the incoming traffic (presumably responses from Resend?) is? - it seems a lot of data to me.

stephen-allison
PROOP

4 months ago

Also - am I correct in thinking that traffic to and from the associated Postgres instance using internal networking would not be included in the 'public networking' chart?


ray-chen

Any network traffic (such as sending API requests and receiving responses) counts as egress/ingress (respectively).

stephen-allison
PROOP

4 months ago

Is there a way to see what the traffic is? Monitoring my local development setup doesn't reveal anything that looks like it would cause MBs worth of ingress.

(sorry for duplicate - do I need to directly respond to replies in here?)


david
EMPLOYEE

4 months ago

am I correct in thinking that traffic to and from the associated Postgres instance using internal networking would not be included in the 'public networking' chart

Correct. That would be private networking

Is there a way to see what the traffic is?

Maybe try looking at the HTTP logs per https://docs.railway.com/guides/logs#http-logs


Status changed to Awaiting User Response Railway 4 months ago


david

am I correct in thinking that traffic to and from the associated Postgres instance using internal networking would not be included in the 'public networking' chartCorrect. That would be private networkingIs there a way to see what the traffic is?Maybe try looking at the HTTP logs per https://docs.railway.com/guides/logs#http-logs

stephen-allison
PROOP

4 months ago

This service doesn't show an HTTP Logs section on the dashboard.

On my local setup I ran a net flow (nfcapd/nfdump) analysis and the amount of traffic with the resend API is measured in kb, what there was a lot of was traffic to/from Postgres as the worker polls the database for tasks. I reduced the polling frequency and the amount of ingress traffic has dropped significantly since which to me suggests that database traffic is being included on the public network ingress even though I'm pretty sure it should all count as 'private' traffic and as far as I can see only private networking is active on my Postgres instance.


Status changed to Awaiting Railway Response Railway 4 months ago


stephen-allison

This service doesn't show an HTTP Logs section on the dashboard.On my local setup I ran a net flow (nfcapd/nfdump) analysis and the amount of traffic with the resend API is measured in kb, what there was a lot of was traffic to/from Postgres as the worker polls the database for tasks. I reduced the polling frequency and the amount of ingress traffic has dropped significantly since which to me suggests that database traffic is being included on the public network ingress even though I'm pretty sure it should all count as 'private' traffic and as far as I can see only private networking is active on my Postgres instance.

I think this is the likely culprit. To set expectations, although this is an annoying bug, I don't have an ETA nor eyes on when we can pick up a remediation on this issue (unless it affects billing, but I don't think this is the case).


Status changed to Awaiting User Response Railway 4 months ago


Railway
BOT

4 months ago

Hello!

We're acknowledging your issue and attaching a ticket to this thread.

We don't have an ETA for it, but, our engineering team will take a look and you will be updated as we update the ticket.

Please reply to this thread if you have any questions!


Added a ticket so we can look into the issue and try to re-pro the bug.


angelo-railway

I think this is the likely culprit. To set expectations, although this is an annoying bug, I don't have an ETA nor eyes on when we can pick up a remediation on this issue (unless it affects billing, but I don't think this is the case).

stephen-allison
PROOP

4 months ago

Thanks for looking - it's not affecting billing as far as I can tell - I just wasn't sure what I was seeing.


Status changed to Awaiting Railway Response Railway 4 months ago


Noted, you should see the Linear bot populate as we get motion on this issue. (Appreciate you raising it up!)


Status changed to Awaiting User Response Railway 4 months ago


Railway
BOT

4 months ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 4 months ago


Loading...