[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix indexed-only behavior for proxy shard #3998

Merged
merged 3 commits into from
Apr 11, 2024
Merged

Conversation

generall
Copy link
Member
@generall generall commented Apr 9, 2024

This is an attempt to rethink a suggestion proposed in #3186

Instead of introducing a new parameter, we simply fix the problem of excluding proxy shard with too many changes from the search by allowing each individual raw segment to decide if it can do search or not.

P.S.
There is a follow-up PR with post-refactoring of simple search method.

@generall generall requested a review from timvisee April 9, 2024 21:51
@keremnc
Copy link
Contributor
keremnc commented Apr 9, 2024

@generall thanks for reviving this discussion.
I think there may still be a shortcoming with this approach - a search request with an extremely restrictive filter could skip a large plain segment, even though the cost of performing the plain search would be very little.
Maybe it makes sense to additionally consider the cardinality estimate of the query here?

@generall
Copy link
Member Author

Indeed, moving it to the level of index would be a better option. Also resolves the bug with sparse vectors we had.

WDYT about the current version @keremnc ?

@keremnc
Copy link
Contributor
keremnc commented Apr 10, 2024

Thanks @generall, this looks good to me

@generall generall merged commit ebc9ba0 into dev Apr 11, 2024
17 checks passed
@generall generall deleted the proxy-shard-indexed-only branch April 11, 2024 11:16
timvisee pushed a commit that referenced this pull request Apr 22, 2024
* move check for unindexed segment size inside the segment to allow proxy shard decide where to search better

* fmt

* move indexed_only check inside the plain index
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants