Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Excessive log noise from go-smtp dependency due to AWS Load Balancer healthchecks #3335

Closed
mastercactapus opened this issue Oct 4, 2023 · 0 comments · Fixed by #3339
Closed
Labels
bug Something isn't working

Comments

@mastercactapus
Copy link
Member

Describe the Bug:
When GoAlert's SMTP server handled AWS Load Balancer health checks, a lot of unnecessary log entires are being generated. These health checks are simply establishing a TCP connection, without sending any data or receiving SMTP protocol greeting, resulting in unnecessary log entries. In addition, these entries are not using the structured logging library.

Steps to Reproduce:

  1. Start the SMTP server to listen on localhost:9025
  2. Issue nc -vz localhost 9025 in a terminal
  3. Observe the log entries

Specifically, this can be seen in an environment similar to Kubernetes hosted on AWS. Error messages like handler error: read tcp 127.0.0.1:9025->127.0.0.1:55372: read: connection reset by peer can be seen.

Expected Behavior:
There should be no log entries for health checks (TCP connection with no data to the SMTP port) and when errors do occur they should go through the normal GoAlert logger. The logging should be minimal and only pertinent entries should be there.

Observed Behavior:
Non-essential log entries are being created due to AWS Load Balancer health checks which aren't utilizing structured logging. Additionally, the logs get spammed with these non-essential entries at regular intervals.

Application Version:
This bug is present in GoAlert version 0.31.1 and latest (master).

Additional Context:
emersion/go-smtp#236 for this already exists in the go-smtp project repository as the root cause seems to be the handling of the TCP health checks at that level. But for GoAlert's perspective, it's causing unnecessary log noise and it does not adhere to the structured logging mechanism, which needs to be addressed.

@mastercactapus mastercactapus added the bug Something isn't working label Oct 4, 2023
@nimjor nimjor mentioned this issue Oct 4, 2023
7 tasks
mastercactapus pushed a commit that referenced this issue Oct 5, 2023
* allow passing Logger instance in config

* pass main app logger to smtp config

* implement smtp.Logger in smtpsrv to make logs go through app.cfg.Logger #3335

* filter out TCP ECONNRESET error messages

* early return instead of negation; add TODO

---------

Co-authored-by: Jordan Nimlos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant