Hey @kozistr, could you please explain why setting those options explicitly is required? AFAIK, most of them are already used by default.

Hey @kozistr, could you please explain why setting those options explicitly is required? AFAIK, most of them are already used by default.

you're right. most of them are already used by default except codegen-units. I just initially thought applying those options explicitly would be good, but never mind. I'll remove them!

changed fce1e2b

Still, could you please provide some explanation why we would want codegen-unit and what are the possible cons

Still, could you please provide some explanation why we would want codegen-unit and what are the possible cons

sure! AFAIK, setting codegen-units to 1 could bring the performance gain or reduce the binary size although taking a longer compilation time (followed by here). I guess it might help to boost the Qdrant performance.

Maybe, if you've already considered this option and thought it wasn't suitable for the Qdrant project, please let me know. (I didn't run lots of tests, or know how codegen-units works deeply, so i might be wrong)

It would be nice to have some benchmarks for this..

More specifically, run some searches using https://github.com/qdrant/bfb

/bounty $50

💎 $50 bounty • Qdrant

💎 $50 bounty • Qdrant

Steps to solve:

1. Start working: Comment /attempt #3982 with your implementation plan
2. Submit work: Create a pull request including /claim #3982 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to qdrant/qdrant!

Add a bounty • Share on socials

Reducing code-gen units to 1 does indeed open the door for more performance optimizations. The problem is that it'll make building extremely slow. Qdrant builds already take a significant amount of time, and I don't think we want to see that as side effect.

The problem is that it'll make building extremely slow.

How extremely? If we need to build it once per release, it is not that bad

The problem is that it'll make building extremely slow.

How extremely? If we need to build it once per release, it is not that bad

I just ran a docker building on my local machine (i7-7700K CPU, on WSL2, based on main branch),

• command line: docker build -f Dockerfile . --no-cache

% time could be differed on the Github action host

On my machine I'm seeing compilation times of 414 seconds versus 519 with a single codegen unit. It's less of a difference than I had anticipated.

That's a good point about compile times varying based on system resources. It's possible GitHub Actions might have a larger difference due to their build environment.

Using codegen-units=1 can improve performance, and small optimizations can add up. Here's something else to consider for future optimizations: exploring potential improvements to multi-threading.

/claim #3982

I am mostly interested in search perf benchmarks

/claim #3982

I am mostly interested in search perf benchmarks

I ran some benchmarks with the bfb, hash 8ecc126

• Environment

  • OS: Windows 10 (WSL 2)
  • CPU: i7-7700K
  • Disk: Samsung SSD 970 EVO Plus 1TB
  • RAM: DDR4 48GB

• treatment: Qdrant 1.8.4

• experimental: Qdrant 1.8.4 with codegen-units = 1

building index

• command line: docker run -it --rm --network host bfb ./bfb --dim 1024
  • dimension : 1024

Search

• Distance: Dot

• on-disk: (payload, index, vectors)

• command line : docker run -it --rm --network host bfb ./bfb --dim 1024 --search --distance Dot --on-disk-payload --on-disk-index true --on-disk-vectors true

  • Dimension: 1024
  • Distance: Dot
  • on-disk: (payload, index, vectors)
  • The reason that I set it up like this is because it's similar to my production environment (on-disk & 1024 dims).

Search timings

Server timings

RPS

Although there are many configs to test and the result depends on the environment, it seems that codegen-units = 1 gives a slightly better result in my experiments.

I'll do more tests if I'm available.

/tip $50 @kozistr

👉 @generall: Navigate to your dashboard to proceed

@kozistr: You just got a $50 tip! 👉 Complete your Algora onboarding to collect your payment.

🎉🎈 @kozistr has been awarded $50! 🎈🎊

• Share on socials
• Give feedback