Pyroscope Integration #5243
Merged
Pyroscope Integration #5243
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit introduces a profiling service using Pyroscope in the `application-generic` package. Several environment variables for configuring the profiling service are added, and the relevant dependencies are included in the package.json file. The new service enables and initializes Pyroscope if it's enabled, and it is also exported in the index file of the package.
This update integrates the ProfilingModule from the '@novu/application-generic' package into the 'webhook', 'ws', 'api', and 'worker' applications. This new incorporation allows for enhanced performance tracking across various modules. The commit also adjusts the 'packageJson' import statement to a more efficient syntax using 'import * as'.
This commit introduces a new profiling controller in the application-generic package. The controller contains methods to profile CPU and Heap data using the NestJS framework. Error handling also added in case there is an issue with collecting profiling data.
The profiling service and module have been enhanced for extended profiling tasks. A new profiling controller with methods for CPU and Heap data profiling has been added. Furthermore, improvements have been made for logging, error handling, and the existing initialization process. Configuration variables have also been renamed for clarity.
This commit adds a new docker-compose.monitoring.yml file for the development environment. It includes configurations for LocalStack, MongoDB, Redis, and Pyroscope services, each with their respective health checks. This will allow us to manage these services more easily and improve monitoring during local development.
The commit removes the unused Logger import and a console.log statement from profiling.service.ts in the application-generic package. This helps to clean up redundant code and make the module more concise.
Instead of using an imported instance of PinoLogger, the NestJS Logger has been directly used for logging in the profiling.service.ts. This change streamlines the code by making use of the built-in NestJS utilities, removing an unnecessary dependency.
✅ Deploy Preview for dev-web-novu ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
# Conflicts: # pnpm-lock.yaml
Package versions in pnpm-lock.yaml have been updated to '^0.24.0' from the old version '^0.23.1'. All instances of the old version have been replaced across numerous dependencies and development dependencies. This ensures that all packages use the latest stable version, maintaining consistency and reducing possible conflicts or issues due to version mismatch.
…he provided code diffs, here's the correct commit message: Add new words to cspell.json New words "pyroscope", "HEAY", "Pyroscope", "PYROSCOPE" have been added to the dictionary in .cspell.json. This expands our dictionary to reduce false positives during spell checking and helps ensure accurate and consistent spelling across the project.
LetItRock
approved these changes
Mar 8, 2024
Comment on lines
+6
to
+7
| @Controller('/debug/pprof') | ||
| @ApiExcludeController() |
There was a problem hiding this comment.
I guess these won't be accessible from the outside right?
There was a problem hiding this comment.
Currently, I have not hooked them to the api yet as with AWS ALB it is not able to block these request but istio can.
I can also have this set up where it is conditionally enabled.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What change does this PR introduce?
This PR introduces the integration of Pyroscope, an open-source continuous profiling platform, into the Novu codebase. By incorporating Pyroscope, we enable real-time performance monitoring and profiling of our Node.js application. This integration involves adding the Pyroscope Node.js agent to our dependencies and initializing it within our application's entry point to begin automatic profiling.
Why was this change needed?
As Novu continues to grow and scale, ensuring optimal performance and resource utilization becomes increasingly critical. The integration of Pyroscope provides us with a powerful tool for continuous profiling, allowing us to identify performance bottlenecks, optimize code, and enhance the overall efficiency and reliability of Novu. This proactive approach to performance monitoring ensures that we can maintain a high-quality service for our users, even as the complexity and scale of our operations increase.
By implementing continuous profiling with Pyroscope, we gain deeper insights into the runtime behavior of our application, enabling us to make data-driven decisions for performance improvements. This change is a step forward in our commitment to delivering a robust and high-performing notification system.