apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-02 19:25:52 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,939 Commits 35 Branches 318 Tags
Commit Graph

41 Commits

Author SHA1 Message Date
Alex Miller
283fd3af27 Add a knob which controls writing prefix compressed kvs mem snapshots. 
Which will be set to on by default in 7.0
 2020-05-12 17:01:52 -07:00
Steve Atherton
fd08f888ea Added comments about when snapshot deltas can't be used, and fixed some whitespace oddities. 2020-04-04 03:19:01 -07:00
Steve Atherton
eca107cb18 Bug fixes regarding when snapshot item deltas are okay, and added sanity checks on delta lengths. Also, rewrote the snapshot progress actor to advance its iterator to the next KV pair rather than doing an upperbound search of the previous key to find the next item. 2020-04-04 02:27:36 -07:00
Steve Atherton
5b18e56d11 KeyValueStoreMemory now uses key delta compression in partial snapshots. This is done with a new log opcode, OpSnapshotItemDelta, so old files can still be read but new files would not be readable by the old code. 2020-04-03 23:54:28 -07:00
Evan Tschannen
96258b9809 Merge branch 'release-6.2' 
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
 2020-02-21 19:09:16 -08:00
A.J. Beamon
df2b0452b4 Step 3 of fixing storage server range reads: change return type of readRange from VectorRef<KeyValueRef> to RangeResultRef. 2020-02-06 13:19:24 -08:00
mengranwo
115ff8bf65 change getKey() interface, pass in uint8_t * only 2020-01-15 13:49:45 -08:00
mengranwo
f597aa7e18 WIP : deployable/stable version since Nov 3. Start rebase to master branch 2020-01-15 13:49:45 -08:00
Evan Tschannen
7a932479dd throw away state if we ever read popped data from the disk queue adapter 2019-07-30 10:14:39 -07:00
Alex Miller
23de5b64ad Memory storage engine to use crc32c DiskQueue by default (in 6.2). 2019-07-01 13:38:06 -07:00
Stephen Atherton
ebc96a7e0e Merge branch 'master' of github.com:apple/foundationdb into feature-redwood 
# Conflicts:
#	fdbserver/VersionedBTree.actor.cpp
 2019-05-21 23:49:27 -07:00
mpilman
d01cbf3455 Addressed code review comments 2019-04-05 13:12:20 -07:00
mpilman
1c16f87a4e Remove trace-calls to printable (in non-workloads) 2019-04-05 13:12:19 -07:00
Stephen Atherton
d5c8b6b083 Merge branch 'master' of github.com:apple/foundationdb into feature-redwood 
# Conflicts:
#	fdbserver/VersionedBTree.actor.cpp
#	flow/flow.h
 2019-03-27 13:37:15 -07:00
A.J. Beamon
7f4adcc338
Merge pull request #1314 from etschannen/feature-ssd-memory-spill 
configure memory now selects the ssd engine for transaction log spilling
 2019-03-19 08:22:22 -07:00
Evan Tschannen
44e25e219c do not suppress KeyValueStoreMemory_OutOfSpace in simulation 2019-03-17 00:35:48 -07:00
Alex Miller
bf247eeed0 If TLogVersion >= 3, use crc32c for the DiskQueue hash for TLogs. 
We don't have a forward compatibility story for the memory storage
engine, so its DiskQueue will still be hashlittle2 until one exists.
 2019-03-15 21:01:16 -07:00
Stephen Atherton
6660ccd936 Large rewrite/refactor from the page format outward. Replaced the in-page format, PrefixTree, with more general and templated implementation of the same concept, DeltaTree, which will be incrementally modifiable soon. New implementation for in-page cursors to use a per-page arena and share decoded binary tree nodes between cursors in the same page. New implementation for internal record B+Tree cursor which uses reference counted path hops which makes cloning a cursor very cheap. New implementation of external facing B+Tree cursor which makes efficient use of two internal cursors to find user-visible KV pairs and return them without copying keys and values except in the case of a large sharded value. 2019-02-21 02:46:30 -08:00
mpilman
3a0f9839b9 Fix minor IDE build errors 2019-02-19 15:16:59 -08:00
Robert Escriva
268093a96d Adjust all includes to be relative to the root. 
Remove the use of relative paths.  A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h".  Adjust so that every include references such a header with the
latter form.

Signed-off-by: Robert Escriva <rescriva@dropbox.com>
 2018-10-19 17:35:33 +00:00
Evan Tschannen
db71b60d72
Merge pull request #819 from satherton/feature-redwood 
Redwood storage engine, initial/experimental version
 2018-10-18 18:38:11 -07:00
Evan Tschannen
0217aed74c Merge branch 'release-6.0' 
# Conflicts:
#	bindings/go/README.md
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/MasterProxyServer.actor.cpp
#	versions.target
 2018-10-15 18:38:51 -07:00
Evan Tschannen
a8feecbfad added a comment to explain code ordering 2018-10-12 16:27:13 -07:00
Evan Tschannen
17a1e3ce35 fix: the master proxy would log an OpCommit for empty commits to the txnStateStore 2018-10-12 12:58:17 -07:00
Stephen Atherton
2fc86c5ff3 Merge branch 'master' of github.com:apple/foundationdb into feature-redwood 
# Conflicts:
#	fdbrpc/AsyncFileCached.actor.h
#	fdbserver/IKeyValueStore.h
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/workloads/StatusWorkload.actor.cpp
#	tests/fast/SidebandWithStatus.txt
#	tests/rare/LargeApiCorrectnessStatus.txt
#	tests/slow/DDBalanceAndRemoveStatus.txt
 2018-09-20 03:39:55 -07:00
Evan Tschannen
90301f497f Merge branch 'release-6.0' 
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbrpc/TLSConnection.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/Status.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/StatusWorkload.actor.cpp
#	versions.target
 2018-09-05 16:06:33 -07:00
Evan Tschannen
90bf277206 require key value store memory to recover cleanly when recovering the txnStateStore, since all of the data it is recovering has been fsync’ed 2018-08-31 13:07:48 -07:00
Alex Miller
fb31a6999f Rewrite all files to have #include actorcompiler.h as the last include. 2018-08-14 15:50:26 -07:00
Alex Miller
535b5701e5 Rewrite all Void _ = wait(...) -> wait(...). 
This takes advantage of the new actorcompiler functionality to avoid
having duplicate definitions of `Void _` when trying to feed the
un-actorompiled source through clang.
 2018-08-14 15:50:26 -07:00
Stephen Atherton
2878f30f29 Merge branch 'master' of github.com:apple/foundationdb into feature-redwood 
# Conflicts:
#	fdbserver/IKeyValueStore.h
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/storageserver.actor.cpp
 2018-06-13 15:56:06 -07:00
A.J. Beamon
e5488419cc Attempt to normalize trace events: 
* Detail names now all start with an uppercase character and contain no underscores. Ideally these should be head-first camel case, though that was harder to check.
* Type names have the same rules, except they allow one underscore (to support a usage pattern Context_Type). The first character after the underscore is also uppercase.
* Use seconds instead of milliseconds in details.

Added a check when events are logged in simulation that logs a message to stderr if the first two rules above aren't followed.

This probably doesn't address every instance of the above problems, but all of the events I was able to hit in simulation pass the check.
 2018-06-08 11:11:08 -07:00
Evan Tschannen
12ef63b698 knobify replace contents bytes 2018-05-01 19:43:35 -07:00
Evan Tschannen
10d25927cd Merge branch 'master' into feature-remote-logs 
# Conflicts:
#	fdbserver/DataDistribution.actor.cpp
 2018-04-30 22:15:39 -07:00
Stephen Atherton
2752a28611 Merge branch 'release-5.2' of github.com:apple/foundationdb into feature-redwood 2018-04-06 16:29:37 -07:00
Evan Tschannen
b36e08f08f first version of non-copying recovery. Upgrades are broken, and it has not been tested using fearless configurations yet 2018-03-29 15:12:38 -07:00
Clement Pang
64deb0e0a1 Address review comments. 2018-03-20 14:38:04 -07:00
Clement Pang
b46ffb4cbc Available space should take into account both memory and disk 2018-03-20 14:38:04 -07:00
Alec Grieser
0bae9880f1 remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py 2018-02-21 10:25:11 -08:00
Stephen Atherton
248dab79b6 Created “redwood” storage engine option and many changes to support that including IKeyValueStore::init() and custom DiskQueue file extensions. 2017-09-21 23:51:55 -07:00
Alec Grieser
660729839c moved Notified.h from flow -> fdbclient ; flow bindings package does better job when excluding testers 2017-07-14 15:49:30 -07:00
FDB Dev Team
a674cb4ef4 Initial repository commit 2017-05-25 13:48:44 -07:00