1843 Commits

Author SHA1 Message Date
Kishore Nallan
ae49dab8fd Vector flat search: handle missing values. 2022-10-21 16:42:23 +05:30
Kishore Nallan
790fac008b Optimize numerical filter ID consolidation. 2022-10-20 15:42:39 +05:30
Kishore Nallan
4808d1610a Flat search cutoff for vector search. 2022-10-19 13:48:07 +05:30
Harpreet Sangar
9651c1abdf
Add Tokenize test. (#756)
* Add Tokenize test.

* Add Tokenize test case.

* Review changes.
2022-10-18 12:07:39 +05:30
Kishore Nallan
ff8fbc0642 Fix an issue with multi-field phrase matching. 2022-10-18 11:49:35 +05:30
Jason Bosco
61986e700a
Tweaks to get docker build to work on Apple Silicon machines (#745)
* Update SHOWCASE.md (#661)

* Update README.md (#670)

* Update README.md

* Update README.md

* Update README.md

* Specify `--platform` explicitly in Docker build commands, so that Docker uses the intended platform on Apple Silicon machines.

* Added image for CLion docker-based remote build env

* Port map 8108

* Compile gdb with python support, so stl container values are shown when debugging in CLion

Co-authored-by: Ray Jasson <48130098+rayjasson98@users.noreply.github.com>
Co-authored-by: rzrymiak <106121613+rzrymiak@users.noreply.github.com>
2022-10-18 08:33:51 +05:30
Kishore Nallan
10ad8162b6 Fix repeating tokens in phrase search. 2022-10-17 16:15:37 +05:30
Kishore Nallan
67f45d8813 Fix regression: allow include/exclude of stored fields. 2022-10-15 18:27:44 +05:30
Kishore Nallan
643ba73ec7 Fix edge case with field names containing dots.
Should not be treated as nested fields.
2022-10-15 17:34:05 +05:30
Kishore Nallan
374cb6948b Reclaim deleted memory on vector insert. 2022-10-11 16:25:13 +05:30
Harpreet Sangar
f75cbca920
Filter query (#737)
* Implement advance filter query parser.

* Remove `exclude_filters`.

* Review changes

* Implement do_filtering2

* Use updated implementation

* Pass failing tests.

* Handle special characters in value.

* Add space around  in embedded filters

* Update tokenize

* Rename to `recursive_filtering`

* Deconstruct filter tree.

* Remove separating space requirement for operators

* Fix rebase errors.

* Undo filter override change

* Format changes

* Pass StaticFilteringMultipleRuleMatch

* Undo formatting changes

* Pass DynamicFilteringMultipleRuleMatch

* Review changes.

* Add filter_query guard.
2022-10-11 11:21:20 +05:30
Kishore Nallan
cf908fb357 Validate query_by id. 2022-10-07 12:51:29 +05:30
Kishore Nallan
af28e9d239 Fix regression in out_of in exclude_fields. 2022-10-07 10:55:05 +05:30
Kishore Nallan
36a0405416 More prefix expansion tweaks. 2022-10-07 10:28:03 +05:30
Kishore Nallan
7b59f10d46 Fix float validation to handle scientific notation. 2022-10-06 11:46:47 +05:30
Kishore Nallan
85d5ce06b2 Refactor prefix search expansion. 2022-10-05 16:14:32 +05:30
Kishore Nallan
77af30ef93 Handle prefix expansion for the same field. 2022-09-29 17:29:30 +05:30
Kishore Nallan
e3a0eb60b0 Remove unused variable. 2022-09-29 12:59:54 +05:30
Ozan Armağan
3280665cc9
Added rate limiting feature (#709)
* Added rate limiting and blocking IP and API key  feature

* Fix for core_api differences

* Coding style improvements

* Coding style improvement II

* Design Update for RateLimitManager

* Updated some of comments

* Design Update II

* Design Update III

* Updated unit test and bug fixes

* Design Update IV

* Design Update V

* Refactoring update for add_rule and edit_rule

* Update to store rate limits and bans in DB

* Bug fix for bans prefix

* Bug fix for rate limit manager unit test

* Refactoring and updates

* Added More Tests & Fixes

* Updated Rate Limit Manager class tests
2022-09-25 10:37:08 +05:30
Yonas Yanfa
34c4c0142f
Fix FreeBSD build by upgrading to latest jemalloc (#733)
* Update SHOWCASE.md (#661)

* Update README.md (#670)

* Update README.md

* Update README.md

* Update README.md

* Fix FreeBSD build by upgrading to latest jemalloc

Latest version of jemalloc as of September 21, 2022: https://github.com/jemalloc/jemalloc/releases/tag/5.3.0

Co-authored-by: Ray Jasson <48130098+rayjasson98@users.noreply.github.com>
Co-authored-by: rzrymiak <106121613+rzrymiak@users.noreply.github.com>
Co-authored-by: Jason Bosco <jason@typesense.org>
2022-09-22 08:52:07 +05:30
Kishore Nallan
eaa2b5027a Remove unused arg. 2022-09-21 16:10:15 +05:30
Kishore Nallan
4679d298c7 Geopoint field cannot be a default sorting field. 2022-09-20 15:12:31 +05:30
Kishore Nallan
940c2b982d Support locale and symbols_to_index in synonyms. 2022-09-20 14:38:23 +05:30
Kishore Nallan
c8afd1e7c6 Use k value from vector query params. 2022-09-17 16:00:56 +05:30
Kishore Nallan
383ed093f3 Fix nested alter drop + add. 2022-09-17 07:44:36 +05:30
Kishore Nallan
8895435591 Ensure that vec distance is always positive for cosine. 2022-09-16 16:48:59 +05:30
Kishore Nallan
d5968b85f7 Fix flat fields being persisted during update. 2022-09-15 18:09:05 +05:30
Kishore Nallan
49268d648e Persist vector field properties and restore. 2022-09-15 17:32:17 +05:30
Kishore Nallan
c7f879bf30 Return vector distance in response. 2022-09-15 11:34:27 +05:30
Kishore Nallan
31559f15b2 Numerical facet field values must be fully removed. 2022-09-14 16:22:15 +05:30
Kishore Nallan
d2daad2c3d Basic alter support for nested fields.
No longer storing .flat inside the document.
2022-09-13 14:47:17 +05:30
Kishore Nallan
dff7f8b746 Don't persist flattened nested field values on disk.
Instead, we will reconstruct them when we fetch them from disk, trading cpu for storage.
2022-09-12 15:04:12 +05:30
Kishore Nallan
aeb4424662 Better error messages for nested field type mismatch. 2022-09-11 18:53:14 +05:30
Kishore Nallan
f9f7afff52 Always do validations during schema alter reindexing. 2022-09-11 16:29:48 +05:30
Kishore Nallan
36f38c31c1 String sorting should handle accented characters. 2022-09-11 08:40:33 +05:30
Kishore Nallan
832b519633 Optimize wildcard query sans filter/sort/facets. 2022-09-10 18:21:13 +05:30
Kishore Nallan
0e39736327 Refactor / simplify schema update logic. 2022-09-09 14:14:21 +05:30
Kishore Nallan
4d86106bc3 Enable nested fields flag should be returned in summary. 2022-09-09 10:49:25 +05:30
Kishore Nallan
56344a20ca Remove spurious logs. 2022-09-08 16:38:56 +05:30
Kishore Nallan
ba2d5d10ce Fix vector search filter: don't assume order of IDs. 2022-09-08 16:19:46 +05:30
Kishore Nallan
d489702fca Handle vector index purge. 2022-09-07 10:57:52 +05:30
Kishore Nallan
c5b55a4ca3 More careful handling of custom request data destruction. 2022-09-06 19:16:50 +05:30
Kishore Nallan
ea835bf17f Don't return empty override rule query and match. 2022-09-06 15:27:00 +05:30
Kishore Nallan
cbd251aa2d Allow override filter_by rule without query/match. 2022-09-05 18:43:28 +05:30
Kishore Nallan
a28ee865bf Fix valgrind warnings. 2022-09-03 19:23:36 +05:30
Kishore Nallan
44edeb8a5e Buffer IDs during filtering. 2022-09-03 18:35:14 +05:30
Kishore Nallan
720855f406 Use cosine similarity as default vector distance. 2022-09-02 17:48:02 +05:30
Kishore Nallan
b20c32046b Support simple filter_by rule in overrides. 2022-09-02 13:27:17 +05:30
Kishore Nallan
0edee3ae12 Generify removal of field with different type due to auto schema detection. 2022-09-01 16:23:46 +05:30
Kishore Nallan
943b2ef06f Handle update of field indexed with object value, but later on as string. 2022-09-01 13:47:36 +05:30