4224 Commits

Author SHA1 Message Date
Fanis Tharropoulos
c2f628a712
feat(parse): add loadConfig function to load config files
- Add `loadConfig` function to load config files using `cosmiconfig`
- Update `parseOptions` function to handle `ZodObject` schema
- Add `TupleOfLength` type helper for tuple types
- Export `parseOptions` and `TupleOfLength` for external use
2025-01-30 18:06:23 +02:00
Fanis Tharropoulos
15f885a277
chore: add cosmicconfig 2025-01-30 12:30:35 +02:00
Fanis Tharropoulos
be232a8631
style: add printwidth to prettierrc 2025-01-30 11:35:00 +02:00
Fanis Tharropoulos
177170d091
chore: bump influx version and remove redudant time prop
- Remove redudant `time: INanoDate` from generic
2025-01-22 20:22:03 +02:00
Fanis Tharropoulos
08b182fd12
style(benchmark): increase width of plots to 150 units 2025-01-22 17:50:58 +02:00
Fanis Tharropoulos
e496f89200
fix(benchmark): fix sort comparison for commit timestamps
- modify timestamp comparison logic to use explicit < > checks
- avoid potential undefined behavior with numeric subtraction
2025-01-22 17:46:34 +02:00
Fanis Tharropoulos
cc41cfe5ab
fix(types): add time object to influx query result 2025-01-22 17:45:43 +02:00
Fanis Tharropoulos
ce72beb96a
feat(benchmark): sort results by commit's first occurrence time
- modify `mapHistoricalResults` to track first timestamp of each commit hash
- sort commit hashes chronologically using `getTime()` values
- maintain result structure while ensuring chronological commit order
2025-01-22 17:08:59 +02:00
Fanis Tharropoulos
82997d379c
fix(benchmark): remove redudant vu entry length check 2025-01-22 15:24:50 +02:00
Fanis Tharropoulos
4bcf2a1552
fix(benchmark): make plots work with any number of vu-scenarios 2025-01-22 15:19:21 +02:00
Krunal Gandhi
98dfb5b686
support geopolygon search and field type (#2150)
Some checks failed
tests / test (push) Has been cancelled
* support geopolygon search and field type

* use compressed ids, format the code

* update test, format code
2025-01-22 17:32:17 +05:30
Fanis Tharropoulos
215a3612bf
ci: ignore workflow conclusion for influxdb historical data 2025-01-22 12:00:13 +02:00
Fanis Tharropoulos
4c59ce6502
ci: change the download action 2025-01-22 10:40:00 +02:00
Fanis Tharropoulos
e919bd0ba6
ci: don't check for successful runs only on influxdb 2025-01-22 10:27:35 +02:00
Fanis Tharropoulos
e48eb2b145
fix(benchmark): return err instead of throwing in format func 2025-01-22 10:27:17 +02:00
Fanis Tharropoulos
cb10441bf6
ci: let benchmarks run on demand as well 2025-01-21 18:29:16 +02:00
Fanis Tharropoulos
dc0b6b6860
feat(benchmark): add historical performance visualization
- integrate historical performance tracking for p95 search times
- add 30-day historical data querying from influxdb
- group and visualize metrics by scenario and virtual user count
- support comparison across multiple commits with dual-line charts
2025-01-21 13:46:55 +02:00
Fanis Tharropoulos
d88bfe24ad
feat(benchmark): add ASCII plot utility for benchmark visualization
- add `plot.ts` utility for creating ASCII charts with customizable configs
- implement support for multiple data series with custom labels and colors
- add automatic scaling and tick generation for x/y axes
- support plot titles, axis labels, and legend rendering
2025-01-21 13:35:59 +02:00
Fanis Tharropoulos
2d27889780
chore: add asciichart deps 2025-01-21 13:33:20 +02:00
Kishore Nallan
3114efec08 Split / joined token should not be prefix searched.
Some checks are pending
tests / test (push) Waiting to run
2025-01-21 12:57:53 +05:30
Jason Bosco
ec8331a9fb
Merge pull request #2149 from tharropoulos/upload-artifact
Some checks failed
tests / test (push) Has been cancelled
ci: bump actions/upload-artifact to v4
2025-01-17 18:25:01 -06:00
Fanis Tharropoulos
d6854dbbbf
ci: bump actions/upload-artifact to v4 2025-01-17 14:43:26 +02:00
Fanis Tharropoulos
ae8a11a3d0
ci: increase benchmark to 1m 2025-01-17 14:38:32 +02:00
Fanis Tharropoulos
8ab4b1756a
fix: optimize k6 and influxdb for high-load scenarios
- increase influxdb memory limits to 4G and add 2G reservation
- tune k6 batch settings for better throughput
  - set batch size to 50k and flush interval to 5s
  - increase concurrent writes to 40
  - adjust push interval to 2s
- optimize influxdb configuration
  - increase cache sizes and write buffer
  - adjust wal fsync delay to 100ms
  - extend write timeout to 60s
  - double write concurrency to 80
- simplify k6 search scenarios
  - remove system tags
  - standardize search phrase selection to 10 VUs
2025-01-17 14:38:07 +02:00
Fanis Tharropoulos
6e9c6c69f5
refactor(benchmark): adjust k6 scenario load testing parameters
- increase gap between test scenarios from 1s to 5s for better isolation
- update first virtual user stage from 10 to 50 for more realistic load
- rename scenario keys to reflect new 50vu configuration
2025-01-17 14:37:43 +02:00
Fanis Tharropoulos
a94a85193c
fix(k6): remove check for status and vus_max after benchmarks 2025-01-17 14:36:56 +02:00
Fanis Tharropoulos
ea84f56d94
ci: map to the internal docker localhost when running in ci 2025-01-17 11:51:58 +02:00
Fanis Tharropoulos
12f9f8df7a
feat(benchmark): add musicbrainz dataset download step
- add dataset download step in `performBenchmarks` method
- integrate with filesystem service to fetch musicbrainz dataset
- update benchmark execution flow to include download step
2025-01-17 11:51:58 +02:00
Fanis Tharropoulos
cbadd348e7
feat(benchmark): add dataset download and decompression support
- add `downloadTypesenseDataset` method to fetch datasets from urls
- implement `decompressStream` to handle gunzip and tar extraction
- add support for jsonl file extraction and saving
- update file system imports to include `writeFile`
2025-01-17 11:51:58 +02:00
Fanis Tharropoulos
5dd95cfe7d
chore: add tar-stream and gunzip-maybe 2025-01-17 11:51:58 +02:00
Fanis Tharropoulos
2e16614b3c
feat(k6-service): add the ability to pass more env vars 2025-01-17 11:51:58 +02:00
Fanis Tharropoulos
d102ea0f8b
chore: rename bin to ts-cli 2025-01-17 11:51:58 +02:00
Fanis Tharropoulos
2e094e7cc2
docker: expose influxdb database directory 2025-01-17 11:51:58 +02:00
Fanis Tharropoulos
8e1bc9dcb9
fix(ci): fix ordering of commit hashes in benchmarking workflow 2025-01-16 20:06:44 +02:00
Fanis Tharropoulos
47a7a39cec
ci(workflow): add benchmark testing workflow
- add new github action workflow for automated benchmark testing
- configure workflow to run every 12 hours via cron
- implement binary comparison between successful test runs
- setup influxdb for storing benchmark metrics
- add artifact management for test binaries and benchmark data
2025-01-16 13:25:34 +02:00
Kishore Nallan
871d576783 Fix root synonym erase.
Some checks failed
tests / test (push) Has been cancelled
2025-01-15 12:38:34 +05:30
Fanis Tharropoulos
0abb754de5
ci: consolidate integration tests into main workflow
- remove separate `integration-tests.yml` workflow
- add integration tests as a dependent job in `tests.yml`
- update artifact download action to use `actions/download-artifact`
- simplify workflow triggers by removing workflow_run dependency
2025-01-14 17:00:24 +02:00
Fanis Tharropoulos
adc80532b4
feat(process): make peering address optional for typesense process
- make `ipAddress` optional in `TypesenseProcessManager`
- remove default ip address fallback
- conditionally add peering address arguments to process startup
- restructure argument building for better readability
2025-01-14 16:50:43 +02:00
Fanis Tharropoulos
196efe8645
feat(client): sync typesense client log level with application
- add dynamic log level setting based on application's logger level
- use `debug` level when application is in debug mode
- fallback to `info` level for all other cases
2025-01-14 16:49:19 +02:00
Fanis Tharropoulos
fc26511bcb
feat(ci): add dynamic ip detection for integration tests
- add `networkInterfaces` detection for CI environments
- make `ip` parameter optional in test command
- add fallback to default ip for non-CI environments
- introduce `isInCi` flag to control ip detection behavior
- handle null ip case in `writeToNodesFile`
2025-01-14 16:45:28 +02:00
Fanis Tharropoulos
0f52828ac3
fix(fs): auto-create missing directory is passed in yes 2025-01-14 16:30:37 +02:00
Fanis Tharropoulos
b9d33234cc
docs: update README 2025-01-13 17:16:24 +02:00
Fanis Tharropoulos
9cff0be6ad
chore(tsconfig): update tsconfig to include importing by file extension 2025-01-13 17:12:44 +02:00
Fanis Tharropoulos
26b7811bb6
feat(cli): add benchmark command for performance testing
- Add benchmark command to compare performance between commits
- Add time-based metrics for search and indexing operations
- Add result formatting with percentage change calculations
- Add configurable failure threshold for performance regression
- Add container and process management for test execution
2025-01-13 17:12:30 +02:00
Fanis Tharropoulos
8793f1b0a1
feat(benchmark): add k6 search benchmark scenarios
- Add multiple search test scenarios (filter, sort, facet, group)
- Add instant search simulation with character-by-character queries
- Add processing time and timeout metrics collection
- Add dynamic VU (Virtual User) scaling support
- Add search phrase generation with stop words filtering
2025-01-13 17:12:12 +02:00
Fanis Tharropoulos
04d4931f39
feat(benchmark): add k6 indexing benchmark script
- Add document import benchmark with k6 metrics tracking
- Add environment validation for benchmark configuration
- Add success checks for import operations
- Add import duration trend tracking
- Add batch size parameter support
2025-01-13 17:11:53 +02:00
Fanis Tharropoulos
c6f9cda8e2
feat(benchmark): add k6 environment validation
- Add `validateK6Environment` function for env var validation
- Add type-safe validation for required k6 environment variables
- Add specific validation rules for numeric, string, and duration values
- Add error reporting for invalid environment configurations
2025-01-13 17:11:35 +02:00
Fanis Tharropoulos
3cec7a2cd8
feat(benchmark): add k6 benchmarking service
- Add `K6Benchmarks` class for managing performance tests
- Implement search and indexing benchmark methods
- Add environment variable configuration for k6 tests
- Add collection schema for benchmark dataset
- Add logging and error handling for benchmark execution
2025-01-13 17:11:12 +02:00
Fanis Tharropoulos
7df0d5b7a4
feat(process): improve typesense node configuration and startup
- Add `NodeConfig` interface for typesense node configuration
- Add `initNode` method to validate node configuration
- Make `snapshotPath` optional with default value
- Add option to disable multi-node mode in `startProcess`
- Simplify process error handling and cleanup
2025-01-13 17:09:31 +02:00
Fanis Tharropoulos
40c971dc6c
refactor(logger): refactor logging logic for process management 2025-01-13 17:08:23 +02:00