431 Commits

Author SHA1 Message Date
Kishore Nallan
341515f614 Consider synonym locale during synonym matching.
Some checks failed
tests / test (push) Has been cancelled
2024-11-01 18:51:12 +05:30
Kishore Nallan
2bcbfafbcf Fix edge cases in facet by parent parameter.
Some checks are pending
tests / test (push) Waiting to run
2024-10-31 16:25:56 +05:30
Harpreet Sangar
89c2c87dc1
Load async_reference property. (#2023)
Some checks failed
tests / test (push) Has been cancelled
* Load `async_reference` property.

* Use `set` instead of `vector` to avoid duplicates.

* Fix test.

* Refactor `reference_info_t::operator <` and `reference_pair_t::operator <`.
2024-10-21 09:10:04 +05:30
Harpreet Sangar
a5aa1a4966
Add Collection::get_referenced_geo_distance. (#1990)
Some checks are pending
tests / test (push) Waiting to run
* Add `Collection::get_referenced_distance`.

* Add test.

* Refactor geo distance logic.

* Add `get_collection_name_with_lock`.

* Refactor to use ternary operator instead.
2024-10-14 12:06:50 +05:30
Kishore Nallan
9135869813 Refactor topster init + consider pagination.
Some checks are pending
tests / test (push) Waiting to run
2024-10-13 20:53:30 +05:30
Kishore Nallan
8e9bf97f78 Improve performance of nested fields compaction + collection meta saving. 2024-10-08 21:38:16 +05:30
Krunal Gandhi
ca84d55beb
Hybrid search aux score (#1968)
Some checks are pending
tests / test (push) Waiting to run
* aux score for hybrid search

* move redundant code to loop

* compute aux text_match using posting lists

* release expanded lists, update tests

* fix merge issues

* make compute_aux_score as member function

* update flag name to rerank_hybrid_matches
2024-10-03 10:52:08 +05:30
Harpreet Sangar
a891ff1a15
Add max_filter_by_candidates option. (#1969)
* Add `max_filter_by_candidates` option.

* Add tests.

* Fix tests.
2024-09-30 15:13:17 +05:30
Krunal Gandhi
b256736986
allow updating remote model api_key (#1944)
Some checks failed
tests / test (push) Has been cancelled
* allow updating remote model api_key

* add nullptr checks and unique_lock

* refactor field updation flow

* Revert "refactor field updation flow"

This reverts commit a6a9847be80ce3d8cee7ea944f5e9453091a949f.

* refactor api_key updation flow

* add locks in remote embedder, update api_key after validation

* fix parsing of embed field

* fix wrong params and parse

* update minor changes, update entry in text_embedders

* add comment

* update embeded_fields with new api_key

* add lock for updating embedded fields

---------

Co-authored-by: Kishore Nallan <kishorenc@gmail.com>
2024-09-25 08:58:32 +05:30
Harpreet Sangar
fa6bdb685c
Filtering shouldn't timeout during Export, Delete by query, Update by query, Cascade deletion of references, and Updating async references. (#1942) 2024-09-10 16:17:02 +05:30
Harpreet Sangar
c8cd7e0472
Async reference field. (#1835)
* Add `async_reference` property to reference field.

* Accept `"async_reference": true` only when `reference` is provided.

* Avoid reference lookup during updation if ref helper field is present.

* Clear `referenced_in_backlog` on dispose.

* Add references asynchronously to singular reference field.

Store only the reference field name in the indexes instead of reference helper field name.

* Only update async references when a document is indexed successfully.

* Remove unused params.

* Refactor reference index.

* Move `update_async_references` into `Index`.

* Refactor `Collection::update_async_references_with_lock`.

* Handle async update of reference array field.

* Add test case.

* Remove `update_async_references` method from `Collection`.

* Add `Join` class.
2024-08-30 11:20:05 +05:30
Harpreet Sangar
bca83ef529
Object reference field. (#1911)
* Including referenced documents of object's reference field shouldn't depend on whether the query excludes the reference field.

* Refactor `Collection::include_references`.

* Only copy `doc` when references are involved.
2024-08-26 10:34:10 +05:30
Harpreet Sangar
65151cd51b
Fix crash when reference expression is passed in _eval. (#1863) 2024-08-01 16:20:26 +05:30
Harpreet Sangar
d9b83a9fc7
Fix cascade deletion in case of a reference array field. (#1844)
* Add `Collection::cascade_remove_docs`.

* Fix recursive mutex locking.
2024-07-18 12:02:46 +05:30
Kishore Nallan
99e11064a6 Fix highlighting for nested array of string. 2024-07-14 17:25:04 +05:30
Harpreet Sangar
87e33e1970
Allow joining export documents. (#1822)
* Allow joining export documents.

* Add tests.
2024-07-10 10:48:57 +05:30
Kishore Nallan
37d7ecabe7 Do value coercion before removing value from index. 2024-06-11 16:09:03 +05:30
Krunal Gandhi
629def6f0f
support updation of collection meta (#1779)
* support updation of collection meta

* error on altering anything except fields and meta

* fix test

* add alter schema validation test

* extend alter schema fields check
2024-06-07 20:44:53 +05:30
Krunal Gandhi
3ab8517a8c
facet parent for array fields (#1756)
* return object itself as parent for array fields

* make variables const reference
2024-05-28 08:32:53 +05:30
Ozan Armağan
ed87e63961
Fix stemming for non-English locales (#1745)
* Fix stemming for non-English locales

* Fix synoyms

* Use `query_tokens_t.q_unstemmed_tokens` for passing unstemmed tokens-

* Refactoring

* Move for loop in parse_search_query to process_tokens completely

* Use q_phrase_dummy and q_exclude_tokens_dummy for unstemmed process_tokens

* Add stopwords_set to process_tokens

* remove unused variables
2024-05-23 01:20:20 +05:30
Harpreet Sangar
8e808d4e29
Fix cascade deletion bug. (#1742)
* Fix cascade deletion bug.

* Add test case.
2024-05-21 19:52:21 +05:30
Krunal Gandhi
39cb7ba9ab
use filter_curated_hits to filter pinned_ids (#1711) 2024-05-07 20:34:23 -04:00
Kishore Nallan
0082c0f0da Rename the facet index type parameter to facet strategy. 2024-05-05 21:26:52 +05:30
Kishore Nallan
506ead9066 Expose facet_index_type parameter. 2024-05-02 04:08:55 +05:30
Kishore Nallan
14338ce491 Return metadata in collection summary response. 2024-04-30 22:45:38 +05:30
Lukas Laurinaitis
2dbbb43e94
Added config option max_per_page (#1577)
* Added limit page size parameter (#4)

* Added max_per_page in cofig

* added default topster size config

* removing const default_topster_size

* typo fix

* Update tests.yml

* few test fixes

* bug fix

* test fix

* added couple missing lines

* last test fix

* Added limit page size parameter (#5)

* Added max_per_page in cofig

* added default topster size config

* removing const default_topster_size

* typo fix

* Update tests.yml

* few test fixes

* bug fix

* test fix

* added couple missing lines

* last test fix

* few fixes after updates from origin

* restore default_topster_size parameter to const

* removing unrelated change and changing int to unsigned int

---------

Co-authored-by: Kishore Nallan <kishorenc@gmail.com>
2024-04-24 08:35:07 +05:30
Krunal Gandhi
15114a6c87
add boolean for enabling typos for alphanumeric tokens (#1651)
Co-authored-by: Kishore Nallan <kishorenc@gmail.com>
2024-04-05 14:25:28 +05:30
Kishore Nallan
3d4ad20bba Merge branch 'main' into v27
# Conflicts:
#	include/collection.h
#	include/index.h
2024-04-04 16:07:54 +05:30
Kishore Nallan
4328ee359c Add cli flag for enabling lazy filter. Disable by default. 2024-03-30 22:31:34 +05:30
Kishore Nallan
e7f9bd30bd Merge branch 'v0.26-facets' into v27
# Conflicts:
#	include/collection.h
#	include/index.h
#	src/collection.cpp
#	src/collection_manager.cpp
#	src/index.cpp
#	test/collection_synonyms_test.cpp
2024-03-30 16:44:57 +05:30
Kishore Nallan
c1b49ef0b9 Add flag for lazy filtering. 2024-03-30 15:30:10 +05:30
Kishore Nallan
8ef6bcb939 Merge branch 'v0.26-facets' into v27
# Conflicts:
#	include/collection.h
#	include/collection_manager.h
#	include/synonym_index.h
#	include/tsconfig.h
#	src/collection.cpp
#	src/collection_manager.cpp
#	src/synonym_index.cpp
#	src/typesense_server_utils.cpp
#	test/collection_manager_test.cpp
#	test/collection_synonyms_test.cpp
2024-03-20 15:34:28 +05:30
Krunal Gandhi
c3560dee27
Synonyms typos (#1620)
* correct query token typos before synonym reduction

* use art_tree for typo searching synonyms

* add prefix support, add typos, prefix config params
2024-03-18 12:21:03 +05:30
Krunal Gandhi
d7f638059e
add boolean to enable synonyms (#1603) 2024-03-07 19:58:31 +05:30
Krunal Gandhi
cc815297ef
pagination for synonyms (#1591)
* pagination for synonyms

* return synonym_op directly
2024-03-01 21:23:02 +05:30
Krunal Gandhi
3f5386eb77
Override pagination (#1588)
* add limit and offset support for collection

* use req->params, dont restrict limit param when records are less

* add more checks and not sort with pagination

* minor updations

* pagination support for overrides
2024-03-01 10:45:03 +05:30
Harpreet Sangar
1f6fbed372
Refactor referenced_in initialization of a collection. (#1585)
* Refactor `referenced_in` initialization of a collection.

* Review changes.

* Review changes.

---------

Co-authored-by: Kishore Nallan <kishorenc@gmail.com>
2024-02-28 20:48:31 +05:30
Ozan Armağan
2c21e1306b
Fix stemming bug when searching with synonyms (#1569)
* Fix stemming bug when searching with synonyms

* Remove unnecessary parameter in `parse_search`query`
2024-02-22 16:26:03 +05:30
Krunal Gandhi
01bedbb342
fix the wrong boolean val (#1559) 2024-02-19 20:31:06 +05:30
Ozan Armağan
59cc66248d
Add stemming for queries (#1548) 2024-02-13 15:05:56 +05:30
Krunal Gandhi
36fcdccddc
add typo_prefix score and num_tokens_dropped in text_match_info (#1529)
* add num_drop_tokens info in text_match_info

* add typo_prefix_score in text_match_info

* add more tests

* add test with drop_token_threshold=1
2024-02-05 20:44:39 +05:30
Harpreet Sangar
ec4311635e
Fix document update failing if reference field was not included. (#1515) 2024-01-27 12:26:36 +05:30
Kishore Nallan
94c560429d Fix hash-based faceting for coerced field values.
When an integer field is coerced to string via schema update, the underlying data on disk will still be integer. We have to account for this during hash based faceting.
2024-01-25 11:19:05 +05:30
Krunal Gandhi
e51055712c
facilitate to add metadata while creating collection (#1491)
* facilitate to add metadata while creating collection

* allow empty metadata keys

* refactor style
2024-01-17 18:57:32 +05:30
Krunal Gandhi
85dd93bc10
Disable typo tolerance for digits (#1478)
* set max_cost to 0 when matching digits in art_fuzzy_search

* fix condition of trailing digits

* refactor the approach

* updating test

* change param name
2024-01-16 15:01:17 +05:30
Ozan Armağan
3b0c518643
Merge branch 'v0.26-facets' into v0.26-facets 2024-01-10 13:41:17 +03:00
ozanarmagan
cffd1a0fcf Fix naming conventions 2024-01-10 13:34:55 +03:00
ozanarmagan
3c8b79781d Add support for audio query 2024-01-09 22:41:45 +03:00
Kishore Nallan
99c991b2fc
Merge pull request #1427 from happy-san/nested_join
Nested Join.
2024-01-02 21:10:40 +05:30
Kishore Nallan
bc4d91616b
Merge pull request #1457 from ozanarmagan/v0.26-facets
Add support for pq in vector queries
2024-01-02 16:37:37 +05:30