[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Range queries on JSON fields #1709

Open
xrl opened this issue Dec 5, 2022 · 9 comments
Open

Range queries on JSON fields #1709

xrl opened this issue Dec 5, 2022 · 9 comments
Labels
good first issue help wanted

Comments

@xrl
Copy link
xrl commented Dec 5, 2022

Is your feature request related to a problem? Please describe.

I cannot perform range queries on JSON fields. For example, the examples/json_field.rs has a search like so:

    {
        let query = query_parser.parse_query("cart.product_id:103")?;
        let count_docs = searcher.search(&*query, &Count)?;
        assert_eq!(count_docs, 1);
    }

But I cannot rewrite the query like this:

    {
        let query = query_parser.parse_query("cart.product_id < 110")?;
        let count_docs = searcher.search(&*query, &Count)?;
        assert_eq!(count_docs, 1);
    }

the count comes back 0.

Describe the solution you'd like

We should be able to search nested JSON documents with the usual <, >, etc.
@adamreichold
Copy link
Collaborator

Isn't the syntax for range queries supported by the built-in QueryParser different, meaning it would be something like

cart.product_id:[0 TO 110}

(assuming that 0 is the smallest possible ID).

@PSeitz
Copy link
Contributor
PSeitz commented Dec 6, 2022 •
edited
Loading

The parser doesn't handle this currently, but this should work cart.product_id:<110 or cart.product_id:{* TO 110}

Related quickwit issue: quickwit-oss/quickwit#2431

@Nickersoft
Copy link
Nickersoft commented Apr 16, 2023 •
edited
Loading

@PSeitz Attempting the :< syntax of :{{* TO 110}} (Rust is complaining about the unescaped {}, thus the double-brackets) returns Unsupported query: Range query are not supported on json field for me.

@PSeitz
Copy link
Contributor
PSeitz commented Apr 17, 2023

Indeed, it's disabled.
I don't think there's a inherent reason, except some code missing to handle that. @fulmicoton?

@fulmicoton
Copy link
Collaborator

If people want to contribute?

@yollotltamayo
Copy link
yollotltamayo commented Apr 19, 2023 •
edited
Loading

Hi @fulmicoton I would like to contribute, if you can give any entry point to start working on this, thanks.

@PSeitz
Copy link
Contributor
PSeitz commented Jun 21, 2023

@yollotltamayo

https://github.com/quickwit-oss/tantivy/tree/main/src/query/range_query

The code is sometimes bound to a fixed field in the schema. These would need to be replaced with something that can handle JSON, e.g. "myjson.fielda"
https://github.com/quickwit-oss/tantivy/blob/main/src/query/range_query/range_query.rs#L335

A range query can run on the columnar storage and on the inverted index. I would implement it first for the inverted index, as it should be simpler.

let me know if your are still interested

@ppodolsky
Copy link
Contributor

I've implemented similar ExistsQuery for jsons, feel free to port it back: https://github.com/izihawa/summa/blob/master/summa-core/src/components/queries/exists_query.rs#L90

@PSeitz PSeitz mentioned this issue Nov 6, 2023
Open
@michel-kraemer
Copy link
michel-kraemer commented May 30, 2024 •
edited
Loading

Here's my custom implementation of a range query for JSON fields for anyone interested:
https://github.com/georocket/georocket/blob/bac0325889d43f93389a54327b90338527ef03c2/rust/core/src/index/tantivy/json_range_query.rs

It's basically the same code as that of Tantivy's range query. I just changed the type of field from String to Field.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue help wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@michel-kraemer @xrl @fulmicoton @PSeitz @Nickersoft @ppodolsky @adamreichold @yollotltamayo