Bug Fixes

• Set MaxSegmentFileSize for force merge when SingleSegmentMergePlanOptions goes into use (#2050)
• Protect code against any bolt failures (#2043)
• Fix size estimation for various field types (#2052)
• Address an out-of-bounds panic that could occur with zapx/v16 in the event of a single non-vector field (#2058)
  • blevesearch/zapx#252

Improvements

• Support new search params for vector search queries - [ivf_probe_pct, ivf_max_codes_pct] (#2049)

Milestone

• v2.4.2

Bug Fixes

• Addressed a bug where ClientContextID was missing from SearchRequest when library used with vectors (#2014)
• Fix to a memory leak in vector query path (#2023)
• Fix to issue #2027, error in parsing a certain datetime syntax (#2030)
• Fix that mitigates a race between persister's stale segment removal and index copy operations (#2032)
• Nested field mapping determination was broken (#2031)

Improvements

• Vector search requires the faiss dynamic library to be built from blevesearch/faiss@d9db66a
• Support for new data type - vector_base64 which allows for interpreting vector float32s encoded as base64 following littleEndian byte ordering (#2012)
• Vector dimensionality support up to 4096 from 2048 (#2015)
• New vector index optimization - memory_efficient alongside pre-existing recall and latency (#2040)
• Better resource usage and efficiency in vector indexing (blevesearch/zapx#241, blevesearch/zapx#234)
• Delayed releasing of deserialized vector indexes to support better query throughput (#2016)

Milestone

• v2.4.1

Bug Fixes

• Fixes to spanish accents normalization (#1957)
• Critical fix around facets merging which could've caused inconsistent results (#1946)

Improvements

• Support for vectors indexing and search
  • Please read: vectors.md for more details
  • With this we will now default to a new zap format: v16, which co-locates text and vector indexes within scorch segments. v16 will be able to interpret v15 zap segments.
  • Requires the faiss dynamic library to be built from blevesearch/faiss@7b119f4
• An empty query attribute within a SearchRequest will now be interpreted as a match_none (#2000)

Milestone

• v2.4.0

Bug Fixes

• Fix format of date times displayed in search result, to not drop precision (#1868)

Improvements

• Upgraded to golang.org/x/text@v0.8.0 to overcome a vulnerability (#1845)
• Support for polish language analyzer - pl (#1825)
• Additional attribute in search response to determine partial/complete hits for match queries (#1848)
• Performance optimization to speed up documents processing per mapping definition (#1865)
• Support fuzziness factor for matchphrase, multiphrase & phrase queries (#1847)
• Geo shape polygon decode optimization w/ buffer re-use (#1864)
• Override-able Marshal/Unmarshal methods (#1880)
• Enhanced data time format support:
  • sanitizedgo that validates layout against supported golang constants (#1877)
  • Ability to apply custom datetime parsers to query criteria and facets (#1860, #1878)
  • Ability to parse date times in these formats - unix_micro, unix_sec, unix_milli, unix_nano (#1870)
  • Additional layouts support with percentstyle and isostyle (#1866)

Milestone

• v2.3.10

Bug Fixes

• Fixes along recording IO bytes read for different query types (#1829)
  • Also changes the attribute name in the SearchResult construct from bytesRead to cost
• Fix incorrect facet counts when facetted queries are coupled with sorting over the same field (#1836)

Improvements

• Upgrade to RoaringBitmap/roaring@v1.2.3 (#1822)
• Upgrade to etcd-io/bblot@v1.3.7 (#1824)
• Consolidated some redundant work in a hot index path with #1832 which showed index time speed up by about 30% in certain situations especially while working with large documents.
• Upgrade to vellum@v1.0.10 which contains a minor optimization (#1838)

Milestone

• v2.3.9

Bug Fixes

• Addresses panics that could occur within zap (with zap@v15.3.10)
  • blevesearch/zapx#105
  • blevesearch/zapx#150
  • blevesearch/zapx#151
• Fixes analyzer inheritance for default mapping with #1807
• Addresses #1816 by fixing CLI's completion generation with #1817

Enhancements

• Introduces a new not-so-aggressive plural stemmer (stemmer_en_plural) for english with #1808

Milestone

• v2.3.8

Bug Fixes

• Replaced deprecated io/ioutil functions with #1781
• Fixed situation where bytesRead will not appear in search response if query gets everything it needs from mmaped content with #1788
• Fixed a panic due to misalignment within chunkedContentCoder on 32-bit platforms with blevesearch/zapx#148 and #1791 (only applicable to zap@v15)
• A bot upgraded the golang.org/x/text version we use from 0.3.7 to 0.3.8 with #1792 🥇
• Include bytesWritten into bbolt (metadata for scorch) with #1774

Enhancements

• Support for improved rollback behavior by allowing sampling of snapshots based on a time interval with #1776 . Default behavior remains the same
• Changed the signature of scorch's AsyncErrorCallback with #1800 to provide more context. This is a breaking change.

Milestone

• v2.3.7

Bug Fixes

• More efficient alignment for chunkedContentCoder (blevesearch/zapx#147)
• Addressed issue in indexing unigrams before punctuations in Japanese (cjk) (#1724)
• Addressed facets string output limitation for date ranges and numeric ranges reported by #1763 with #1764
• Fixed inaccurate IO stats accounting while loading segment metadata (#1769)

Enhancements

• New bytesRead attribute in search response (#1752, blevesearch/zapx#142)
  • Applicable to zapx v15 only
• New tests showcasing performance benchmarks for popular queries (#1755)

Milestone

• v2.3.6

Bug Fixes

• Fixed throughput regression reported with #1731
  • #1738
  • blevesearch/bleve_index_api#30
  • blevesearch/scorch_segment_api#20
• Fix intersection logic for geo JSON searches, more unit tests
  • blevesearch/geo#10
  • #1720
• Fix to support IP Range query
  • #1735

Other

• Breaking change in defining custom analyzers
  • #1729 (this is needed to allow custom analyzer definition without needing to always define CharFilters, Tokenizer, TokenFilters)
• Update spf13/cobra version v1.4.0 (to drop a number of indirect dependencies)
  • aba7d6c

Milestone

• v2.3.5

Enhancements

• Support store option for the geoshape field (#1714)
• Publishing read/write stats based on usage (#1702 , #1710)