Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make MMAP_SIZE_THRESHOLD in OffHeapSingleTreeBuilder builder is configurable #15093

Open
chrajeshbabu opened this issue Feb 19, 2025 · 0 comments

Comments

@chrajeshbabu
Copy link
Contributor

chrajeshbabu commented Feb 19, 2025

Currently when the buffer size required for sort and aggregate the records require for star tree building above 500mb using MMAP based files used which is making star tree building slower compared to direct memory usage when the number of docs and dimensions are higher. Would be better to make it configurable so based on the available memory and number of segments to build the startree.


  // If the temporary buffer needed is larger than 500M, use MMAP, otherwise use DIRECT
  private static final long MMAP_SIZE_THRESHOLD = 500_000_000;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants