1
0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-25 00:30:00 +08:00

139 Commits

Author SHA1 Message Date
Markus Pilman
8af056e7b0 fdbclient now compiling 2022-06-23 18:05:36 -06:00
Lukas Joswiak
b80ed948f1 Check initialization status before accessing field 2022-06-22 14:50:17 -07:00
Josh Slocum
b3597ef3a8 Added plumbing for tenant-aware purge granules 2022-06-16 13:04:34 -05:00
Hao Fu
f00606fd71
Enable MATCHED and UNMATCHED mode for index prefetch ()
* Enable MATCHED and UNMATCHED mode for index prefetch

MATCHED mode returns index entries whose secondary KVs are present,
UNMATCHED mode returns index entries whose secondary KVs are absent.

Note that the conflict read range of this txn is set in 2 steps:
* Set the conflict range for primary query according to request
* Set the conflict ranges for secondary queries according to responses.

As a result, conflicts of different match_index mode are taken care of.

* Fix c binding
2022-05-18 09:16:28 -07:00
Hao Fu
853e6a346b
Optimization: support removing index conditionally () 2022-05-13 10:10:43 -07:00
Vaidas Gasiunas
a3102c0db4 Fix updating database shared state on upgrades 2022-05-12 15:28:53 +02:00
Ray Jenkins
dc9e782ccc
OpenTelemetry Tracing Perf Fixes () 2022-05-02 14:56:51 -05:00
Ray Jenkins
1c5bf135d5
Revert "Migrate to OpenTelemetry tracing. ()" ()
This reverts commit 5df3bac110d9b5b88931b008b852433688bb7eb0.
2022-04-25 09:29:56 -05:00
Vaidas Gasiunas
e48ab69c6b An option to specify a custom directory for temporary client library copies 2022-04-21 19:27:19 +02:00
Ray Jenkins
5df3bac110
Migrate to OpenTelemetry tracing. () 2022-04-20 09:26:37 -05:00
Dan Lambright
1b3b4166c6
Merge branch 'main' into vv 2022-04-08 17:18:13 -04:00
Josh Slocum
6276cebad9
Blob integration ()
* Fixing leaked stream with explicit notify failed before destructor

* better logic to prevent races in change feed fetching

* Found new race that makes assert incorrect

* handle server overloaded in initial read from fdb

* Handling more blob error types in granule retry

* Fixing rollback metadata problem, added better debugging

* Fixing version race when fetching change feed metadata

* Better racing split request handling

* fixing assert

* Handle change feed popped check in the blob worker

* fix: do not use a RYW transaction for a versionstamp because of randomize API version ()

* more merge conflict issues

* Change feed destroy fixes

* Fixing change feed destroy and move race

* Check error condition in BG file req

* Using relative endpoints for blob worker interface

* Fixing bug in previous fix

* More destroy and move race fixes

* Don't update empty version on destroy in case it gets rolled back. moved() and removing will take care of ensuring it is not read

* Bug fix ()

* fix: do not use a RYW transaction for a versionstamp because of randomize API version

* fix: if the initialSnapshotVersion was pruned, granule history was incorrect

* added a way to compress null bytes in printable()

* Fixing durability issue with moving and destroying change feeds

* Adding fix for not fully deleting files for a granule that child granules need to re-snapshot

* More destroy and move races

* Fixing change feed destroy and pop races

* Renaming bg prune to purge, and adding a C api and unit test for it

* more cleanup

* review comments

* Observability for granule purging

* better handling for change feed not registered

* Fixed purging bugs ()

* fix: do not use a RYW transaction for a versionstamp because of randomize API version

* fix: if the initialSnapshotVersion was pruned, granule history was incorrect

* added a way to compress null bytes in printable()

* fixed a few purging bugs

Co-authored-by: Evan Tschannen <evan.tschannen@snowflake.com>
2022-04-08 14:15:25 -07:00
Dan Lambright
5bdc525353
Merge branch 'main' into vv 2022-04-08 13:16:04 -04:00
Jon Fu
b0ae22f16d Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-04-06 12:28:28 -04:00
Jingyu Zhou
64d4658034 Merge branch 'main' into vv
Fix Conflicts:
	flow/error_definitions.h
2022-04-01 21:49:24 -07:00
A.J. Beamon
68f15650a1 Make sure closed and tenantUpdater are read/written in the same critical section. 2022-03-31 11:12:01 -07:00
Jingyu Zhou
4fd414a8ed Merge branch 'main' into vv
Fix Conflicts:
	fdbclient/NativeAPI.actor.cpp
2022-03-31 09:38:36 -07:00
Jingyu Zhou
cfcf0f152c Merge branch 'main-4a085fc84' into vv
Fix Conflicts:
	fdbclient/NativeAPI.actor.cpp
	fdbserver/ClusterRecovery.actor.cpp
	fdbserver/MasterInterface.h
	fdbserver/masterserver.actor.cpp
	flow/error_definitions.h
2022-03-30 22:28:06 -07:00
Jingyu Zhou
b34f4052cd Merge branch 'main-f28dfc12b' into vv
Fix Conflicts:
	fdbclient/MultiVersionTransaction.actor.cpp
	fdbclient/NativeAPI.actor.cpp
	fdbclient/NativeAPI.actor.h
	fdbserver/storageserver.actor.cpp
2022-03-30 21:01:25 -07:00
Jingyu Zhou
00b57d4cce Merge branch 'main-67eba5ec7' into vv
Fix Conflicts:
	fdbclient/CommitProxyInterface.h
	fdbclient/NativeAPI.actor.cpp
	fdbclient/StorageServerInterface.h
	fdbserver/CommitProxyServer.actor.cpp
	fdbserver/storageserver.actor.cpp
2022-03-30 20:05:55 -07:00
A.J. Beamon
c7d53b31ee Use a TenantState object in the MVC implementation to help manage tenant lifetime. 2022-03-30 12:52:27 -07:00
Jon Fu
f266695a76 clear shared state map entries on version upgrade and use currentProtocolVersion instead of piping version through function parameters 2022-03-30 15:28:55 -04:00
Jon Fu
9d0a520fe0 add ProtocolVersion to shared state and state cluster map 2022-03-30 13:28:06 -04:00
Jon Fu
53d252fda0 Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-03-29 17:23:08 -04:00
Jon Fu
bdbe78e48f address code review comments 2022-03-29 17:20:22 -04:00
Josh Slocum
029c30d035 fix formatting 2022-03-24 16:05:51 -05:00
Josh Slocum
b42c49ca0a Adding full file length for future-proofing 2022-03-24 15:56:03 -05:00
Josh Slocum
b92c6ea92c granule parallelism, metrics, and other improvements to bg client bindings 2022-03-24 13:44:09 -05:00
Jon Fu
7c3151142e Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-03-23 16:41:33 -04:00
Jon Fu
4466e00716 Fix multithreaded GRV cache code and enable grv caching in multithreaded ctest suite 2022-03-23 16:09:12 -04:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Jon Fu
9731d96797 Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-03-16 14:32:33 -04:00
Jon Fu
7ab205391b Revert "remove all multi-version client code related to grv cache"
This reverts commit 6d05ecffc21d8497b4f753d11b17beb6102f32ab.
2022-03-16 14:26:36 -04:00
A.J. Beamon
68069c9784 Undo some unused changes. Fix tenant cache eviction. 2022-03-15 09:23:30 -07:00
A.J. Beamon
8bc2b283e1 Create a special keys API to create, delete, and read tenants. Remove the C API to create/delete tenants. 2022-03-15 09:23:30 -07:00
A.J. Beamon
c89b1e3537 Use TenantName type instead of StringRef in various places. Add a function on transactions to get the tenant name being used for the transaction. 2022-03-15 09:21:27 -07:00
A.J. Beamon
c635dcd3ad Add tenant support in the FDB native client 2022-03-15 09:21:27 -07:00
Tao Lin
e2c7c30faf
GetMappedRange support serializable & check RYW & continuation () 2022-03-10 10:05:44 -08:00
Dan Lambright
9544379cdf rebase 2022-01-20 11:12:33 -05:00
Josh Slocum
b7b2ad0a6f Handling timeout and transaction changes for ThreadResults properly 2021-12-10 12:47:53 -06:00
Josh Slocum
9cb6fb5114 fixing unrelated code formatting 2021-12-01 17:20:11 -06:00
Josh Slocum
c11b8a3625 Improved comments 2021-12-01 17:04:55 -06:00
Josh Slocum
a82845af43 Merge branch 'master' into bg_bindings 2021-12-01 16:55:28 -06:00
Josh Slocum
7f4fcc8c2c Added FDBResult and made readBlobGranules use it 2021-12-01 16:22:05 -06:00
A.J. Beamon
264c75b9a6 Add some extra client logging details:
1. Add a trace event when a database is created and move the cluster file / connection string from ClientStart to the new trace event
2. Add a detail for the path to the image being loaded
3. Add a detail for whether a client library is primary or not
4. Set a thread name for each external client thread that includes the release version
2021-11-29 09:57:10 -08:00
Josh Slocum
329091e14f Merge branch 'master' into bg_bindings 2021-11-11 10:13:37 -06:00
Josh Slocum
b8ac4213a1 Switched BG APIs to transaction instead of database 2021-11-11 08:59:06 -06:00
Tao Lin
fdb3b72e35 Introduce GetRangeAndFlatMap to push computations down to FDB
Re-introduce 
2021-11-09 13:52:28 -08:00
Tao Lin
586cc3b102
Revert "Introduce GetRangeAndFlatMap to push computations down to FDB" 2021-11-04 08:46:56 -07:00
Tao Lin
6c98e35893 Rename Hop to FlatMap 2021-11-03 13:32:01 -07:00