4056 Commits

Author SHA1 Message Date
Harpreet Sangar
ec19052db1
Fix unindexed fields of joined collection not being included with wildcard include. (#2107)
Some checks are pending
tests / test (push) Waiting to run
2024-12-16 11:56:55 +05:30
Kishore Nallan
a282aafe65 Remove ARM lg page size for jemalloc.
Some checks failed
tests / test (push) Has been cancelled
2024-12-11 21:07:23 +05:30
Kishore Nallan
57e1548acb Fix error: add flag for jemalloc for cache oblivious. 2024-12-11 20:49:07 +05:30
Kishore Nallan
61ebe244aa Add flag for jemalloc for cache oblivious. 2024-12-11 20:31:42 +05:30
Krunal Gandhi
9923a5455e
hybrid search flat_search_cutoff (#2099)
Some checks failed
tests / test (push) Has been cancelled
* hybrid search flat_search_cutoff

* refactor repeatative code

* make both approach exclusive
2024-12-10 19:03:20 +05:30
Kishore Nallan
225ed057cd Revert removing jemalloc unused memory as part of calculations.
Some checks failed
tests / test (push) Has been cancelled
2024-12-09 19:38:23 +05:30
Ozan Armağan
f6bdba15eb
Fix custom dimension initialization (#2092)
Some checks are pending
tests / test (push) Waiting to run
* Fix custom dimension initialization

* Add test
2024-12-06 17:53:46 +05:30
Kishore Nallan
04cb05aec3 Remove stray log. 2024-12-06 11:42:51 +05:30
Kishore Nallan
67d6f21991 Use explicit num_dim passed to auto embedding remote models. 2024-12-06 11:25:58 +05:30
Krunal Gandhi
adbfa7360e
Normalize plurals (#2062)
* initial commit

* allow multiple plurals set

* acess dictionary word from stemmerManager directly

* use find over count

* make dictionary post request async

* add endpoints to get dictionary

* update function names

* update core_api function names

* fix the test

* update end-point names

* reload dictionaries on restart

---------

Co-authored-by: Kishore Nallan <kishorenc@gmail.com>
2024-12-05 14:06:25 +05:30
Ozan Armağan
d126f91aa1
Refactor get_history_collection by overloading (#2090)
Some checks failed
tests / test (push) Has been cancelled
* Refactor `get_history_collection` by overloading

* Return error when model doesn't have history collection

* Refactoring

* Remove unnecessary function

* Refactor add_conversation

* Make conversation model id necessary for deleting conversation
2024-12-03 19:06:52 +05:30
Ozan Armağan
83f1cad4b9
Fix conversation not found error (#2087)
Some checks are pending
tests / test (push) Waiting to run
* Fix `conversation not found` error

* Add test and refactor

* Move getting last n messages logic to a functing
2024-12-03 15:36:25 +05:30
Ozan Armağan
8ec85d7dba
Move public models to ts_ prefix directories (#2070)
Some checks are pending
tests / test (push) Waiting to run
* Move public models to ts_ prefix directories

* Use `butil::Move` to rename

* Refactor saving model config

* Delete the dest folder before move for model prefix migration

---------

Co-authored-by: Kishore Nallan <kishorenc@gmail.com>
2024-12-02 18:53:03 +05:30
Harisaran G
d06de6f14d
add: GET /events endpoint (#2084)
Some checks failed
tests / test (push) Has been cancelled
* add: get_events endpoint

* fix: query_compaction_interval for test build

* fix: tests

* fix: analytics_manager.h

* fix: tests
2024-11-29 15:15:33 +05:30
Kishore Nallan
84a25e2aa5 Add env flag for disabling embedding model memory check.
Some checks are pending
tests / test (push) Waiting to run
Useful for local dev where swap usage is ok.
2024-11-29 09:55:21 +05:30
Kishore Nallan
f829d4f922 Access guard for tokenizer. 2024-11-29 09:54:43 +05:30
Kishore Nallan
c6ed3c9be1 Consider all fields for aux text match score.
Some checks are pending
tests / test (push) Waiting to run
2024-11-28 11:21:06 +05:30
Kishore Nallan
5ac477a13b Refactor on-demand snapshot validation logic.
Some checks failed
tests / test (push) Has been cancelled
2024-11-26 13:08:35 +05:30
Kishore Nallan
bf5172fe40 Add test case for validation of group by field name.
Some checks are pending
tests / test (push) Waiting to run
2024-11-25 21:17:53 +05:30
Kishore Nallan
4752c09227 Implement validate field names for group by. 2024-11-25 20:46:51 +05:30
Harpreet Sangar
0379072083
When validate_field_names is false, ignore the filter whose field name is not found in the schema. (#2077)
* When `validate_field_names` is false, ignore the filter whose field name is not found in the schema.

* Add test.

* Ignored filter should match 0 documents.

* Respect `validate_field_names` in export, update, delete, overrides, _eval, and joins.

* Refactor `validate_field_names` initialization.

* Update comment.
2024-11-25 18:30:01 +05:30
Harisaran G
67ae222c96
fix: analytics_manager validation and tsconfig (#2079)
* fix: analytics_manager validation and tsconfig

* fix: review comments
2024-11-25 15:43:01 +05:30
Kishore Nallan
b26a856827 Refactor memory stats calculations.
Some checks failed
tests / test (push) Has been cancelled
2024-11-23 11:19:07 +05:30
Kishore Nallan
1f590e443e Fix facet value intersection galloping. 2024-11-23 08:41:32 +05:30
Kishore Nallan
a103d97487 Fix updates to empty vector record.
Some checks are pending
tests / test (push) Waiting to run
2024-11-21 18:51:40 +05:30
Kishore Nallan
30c442ee0d Fix test.
Some checks are pending
tests / test (push) Waiting to run
2024-11-21 11:57:10 +05:30
Kishore Nallan
6fa4296392 Add index for resolving facet string value. 2024-11-21 11:40:18 +05:30
Kishore Nallan
5b900be858 Move get_proc_meminfo to system metrics. 2024-11-16 16:42:07 +05:30
Kishore Nallan
ce276e1606 Add flag for skipping validation of field names.
Some checks failed
tests / test (push) Has been cancelled
2024-11-15 14:13:22 +05:30
Ozan Armağan
05be6d1eef
Fine grain embedding locks (#2067)
Some checks are pending
tests / test (push) Waiting to run
* Fine grain  embedding locks

* Fix unique lock bug
2024-11-14 19:23:31 +05:30
Harisaran G
4373b2b6fd
add: personalization model APIs (#2001)
Some checks are pending
tests / test (push) Waiting to run
* add: basic recommendations endpoint

* add: POST `/recommendations/models` recommendations endpoint

* add: get specific model endpoint

* add: DELETE `/recommendations/models/:id` and GET all models recommendations endpoint

* add: ONNX model upload endpoint

* add: save the file as a .tar.gz instead of ONNX

* add: verify_tar_gz function

* add: support for model upload

* add: PUT `/recommendations/model/:id` to update the model, name, collection

* refactor: recommendations -> personalization

* add: personalization tests

* add: personalization_model_manager_tests

* fix: review comments

* fix: review comments
2024-11-14 13:08:42 +05:30
Ozan Armağan
6a762d637a
Fix race condition for image embedding models (#2065) 2024-11-14 11:31:40 +05:30
Fanis
5705717987
Fix vector query format validation error messages (#2063)
* fix: correct vector query format validation error message

- Fix parser to properly detect missing colon in vector query format
- Remove redundant condition check for colon validation
- Ensure format `fieldname:([values])` is strictly enforced
- Update test cases to match expected error messages

* fix(vector-search): make missing colon error more descriptive

* chore: update comments to reflect proper vector query formatting
2024-11-13 15:11:35 +05:30
Kishore Nallan
a05b046d1d Skip external snapshot copy if path not provided. 2024-11-10 20:50:56 +05:30
Kishore Nallan
26f37f9fdf Tweak bad query check interval. 2024-11-10 19:22:21 +05:30
Kishore Nallan
b8badef310 Use slot len when composing content type header value.
Some checks failed
tests / test (push) Has been cancelled
2024-11-09 22:15:54 +05:30
Kishore Nallan
7cd4f90453 Log long running queries with large memory usage. 2024-11-09 19:17:49 +05:30
Jason Bosco
ec1e53aac5
Improve error message, to prevent confusion with auto-embedding feature (#2056)
Some checks failed
tests / test (push) Has been cancelled
2024-11-08 09:29:49 +05:30
Kishore Nallan
555f421a33 Cleanup code. 2024-11-08 09:05:26 +05:30
Harpreet Sangar
970da52f15
Support enable_lazy_evaluation in case of id: * filter. (#2052)
Some checks are pending
tests / test (push) Waiting to run
2024-11-07 18:32:18 +05:30
Kishore Nallan
fd100f73d0 Fix text match score being overwritten. 2024-11-07 18:22:38 +05:30
Kishore Nallan
32598540c6 Implement per-collection alter throttling.
Some checks failed
tests / test (push) Has been cancelled
2024-11-05 16:22:29 +05:30
Fanis
7b7c3d4e1e
fix(embeddings): add validation to ensure embedding fields are of type float[] (#2043)
Some checks are pending
tests / test (push) Waiting to run
* fix(embeddings): add simple check for field type to be float array

- Add a simple if check for when the embed prop is present, so the field
type must be of type `float[]`

* style: move test to vector search test suite

* test(vector): fix test class name for embed field validation
2024-11-04 11:33:14 +05:30
Kishore Nallan
341515f614 Consider synonym locale during synonym matching.
Some checks failed
tests / test (push) Has been cancelled
2024-11-01 18:51:12 +05:30
Ozan Armağan
ba5963a9ee
Fix returning conversation history partially (#2041)
Some checks are pending
tests / test (push) Waiting to run
* Fix returning conversation history partially

* Update test
2024-10-31 21:56:30 +05:30
Kishore Nallan
82aacacce0 Fix faceting not following curation rules. 2024-10-31 17:23:57 +05:30
Kishore Nallan
2bcbfafbcf Fix edge cases in facet by parent parameter.
Some checks are pending
tests / test (push) Waiting to run
2024-10-31 16:25:56 +05:30
Kishore Nallan
cf12afe4ac Fix typo threshold for curated hits.
Some checks failed
tests / test (push) Has been cancelled
2024-10-29 21:00:08 +05:30
Kishore Nallan
db892457a8 Refactor curated ids usage. 2024-10-29 21:00:08 +05:30
Krunal Gandhi
643f825d1c
Sortby decay funtions (#2036)
* add support for decay function with sort_by

* add validation tests

* add test

* update score as float and add test

* add func key and other minor changes

* add diff function
2024-10-29 20:34:13 +05:30