Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tyr-worker] feat: trying to avoid 'logs forwarding' to fail the entire process #3884

Merged
merged 1 commit into from
Dec 20, 2022

Conversation

woshilapin
Copy link
Contributor

Let's try to protect us against any failure triggered by the log forwarding. Note that this PR is a best effort, but probably doesn't fix https://navitia.atlassian.net/browse/NAV-1653.

About the original ticket, after more searching, we think we now have a good explanation of what is going on (but it's hard to verify for sure).

The best hypothesis is:

  • "log forwarding" apply the .decode operation per chunks (calling read_async() and therefore .read())
  • the chunk contains multi-bytes characters like é and one of the chunk ends with only half of one of this multi-byte characters
    • note the error message which is [...]donn\xc3', 4095, 4096, 'unexpected end of data')
    • 4096 = 2¹² which looks pretty much like an ideal chunk size
    • the \xc3 is missing something to represent données (in the logs, we have a full version of it a line above donn\xc3\xa9es)

If the hypothesis is correct, then #3880 should have already fixed the problem. But this PR should also prevent a bit more.

@woshilapin woshilapin requested review from pbougue, pbench, azime, kadhikari and a team and removed request for kadhikari December 20, 2022 11:38
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

@woshilapin woshilapin merged commit a24c5a9 into dev Dec 20, 2022
@woshilapin woshilapin deleted the fix-logs-forwarding branch December 20, 2022 13:03
@woshilapin woshilapin changed the title feat: trying to avoid 'logs forwarding' to fail the entire process [tyr-worker] feat: trying to avoid 'logs forwarding' to fail the entire process Dec 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants