apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-15 02:18:39 +08:00
Code Issues Packages Projects Releases Wiki Activity
16,245 Commits 35 Branches 318 Tags
Commit Graph

109 Commits

Author SHA1 Message Date
Evan Tschannen
fcb8bd6475
Revert "Make the sim2 run loop match the behavior of the net2 run loop." 2021-06-22 14:50:01 -07:00
Evan Tschannen
2ae4f51d19 added a comment 2021-06-22 09:42:00 -07:00
Evan Tschannen
86ebb0a646 throw non-file not found errors first to better handle injected faults 2021-05-20 16:46:00 -07:00
sfc-gh-tclinkenbeard
5c2d7b6080 Create RangeResult type alias 2021-05-03 13:14:16 -07:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
A.J. Beamon
aaf0a9aa7b Merge branch 'release-6.3' into merge-release-6.3-into-master 
# Conflicts:
#	build/docker-compose.yaml
#	cmake/ConfigureCompiler.cmake
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/IAsyncFile.h
#	fdbrpc/IRateControl.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbservice/ServiceBase.cpp
 2021-02-08 12:58:34 -08:00
A.J. Beamon
67e783acf8 Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3 
# Conflicts:
#	cmake/CompileBoost.cmake
#	cmake/FDBComponents.cmake
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/storageserver.actor.cpp
#	flow/Knobs.h
#	flow/network.h
 2021-02-08 09:20:28 -08:00
Steve Atherton
8547abbbe4 Bug fix: In simulation, KVStoreSQLite can outlive its process during a simulated reboot so its file references must be cleared explicitly during shutdown instead of implicitly during destruction. Re-enabled a useful trace event. Some cleanup, removed unnecessary member var initializer and removed unnecessary method. 2021-02-05 21:26:39 -08:00
Chaoguang Lin
f6342376ab Update the knob name 2021-02-02 13:31:00 -08:00
Chaoguang Lin
17671604c0 Add option to define I/O size 2021-02-02 12:39:05 -08:00
Markus Pilman
e4b953ab38
Merge pull request #4229 from sfc-gh-satherton/afcache-write-limit 
AsyncFileCached RateControl support and bug fixes
 2021-01-31 15:13:03 -07:00
Steve Atherton
d607ba3ac1 Removed merge artifact which calls functions not present in release-6.2. 2021-01-31 03:59:36 -08:00
Chaoguang Lin
15aa877388 Use VFSAsyncFile::checkInjectedError() to replace old injected fault detection, see pr#4212 2021-01-30 18:25:43 -08:00
Steve Atherton
bb6608ccd8 Another bug fix in SQLite file instance lifetimes. Writer can be destroyed before doClose() or stopOnError() have a chance to react, so Writer can't be used to get access to file handles to remove the rate limiter and trigger pending requests to complete. New logic is for Writer initialization to place opened file handles in KeyValueStoreSQLite instance and move disableRateControl() there, to be called from doClose(). 2021-01-30 18:23:50 -08:00
Steve Atherton
4982787a80 Another attempt to fix SQLite lifetime issues. KeyValueStoreSQLite::doClose() can now access the Writer instance and call disableRateControl() on its SQLiteDB instance prior to stopping the reader and writer thread pools. This should prevent rate limiting from interfering with SQLite shutdown. 2021-01-30 18:18:36 -08:00
Steve Atherton
b0fd6bfbcf More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent. 2021-01-30 18:18:16 -08:00
Steve Atherton
52c2695f59 Bug fixes involving use of AFCached rate control in simulation. 2021-01-30 18:13:44 -08:00
Steve Atherton
e56fe02a25 Merge branch 'add-throttling-on-AsyncfileCached' of github.com:sfc-gh-clin/foundationdb into afcache-write-limit 2021-01-30 18:11:54 -08:00
Steve Atherton
14af5857ed Another refactor of SQLite injected error handling to address failures of the previous attempt. Some code cleanup. Added documentation of injected error handling works and why. 2021-01-30 18:06:05 -08:00
Steve Atherton
e72d486215 Refactored how injected fault state is managed to make debugging injected fault handling easier. Moved injected error tracking for open() into an INetwork global since it should be unique per simulated process. Fixed bug where injected error in WAL file was not recognized as injected because it occurred before SQLiteDB's vfsWAL pointer was initialized. Simplified logic in SQLiteDB. Added comments. 2021-01-30 18:06:04 -08:00
Steve Atherton
1a5c40fcef Bug fixes: Check for null pager and wal pointers, and initialize vfsWAL after SQLite has opened the WAL. 2021-01-30 18:06:04 -08:00
Steve Atherton
e1f8023577 Rewrote how injected faults are handled in SQLite to be more reliable and work with an upcoming write throttling feature in AsyncFileCached. 2021-01-30 18:06:04 -08:00
Chaoguang Lin
4c4d1ec6b0 Use VFSAsyncFile::checkInjectedError() to replace old injected fault detection, see pr#4212 2021-01-29 15:28:08 -08:00
Steve Atherton
14e39a8747 Another bug fix in SQLite file instance lifetimes. Writer can be destroyed before doClose() or stopOnError() have a chance to react, so Writer can't be used to get access to file handles to remove the rate limiter and trigger pending requests to complete. New logic is for Writer initialization to place opened file handles in KeyValueStoreSQLite instance and move disableRateControl() there, to be called from doClose(). 2021-01-28 00:59:05 -08:00
Steve Atherton
e3a6531477 Another attempt to fix SQLite lifetime issues. KeyValueStoreSQLite::doClose() can now access the Writer instance and call disableRateControl() on its SQLiteDB instance prior to stopping the reader and writer thread pools. This should prevent rate limiting from interfering with SQLite shutdown. 2021-01-27 05:31:02 -08:00
Steve Atherton
989fee908a More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent. 2021-01-27 00:01:33 -08:00
Andrew Noyes
564c70acb4
Merge branch 'master' into anoyes/modernize-use-override 2021-01-25 18:26:18 -08:00
Andrew Noyes
4ee97c0784 Use clang-tidy to automatically fix missing overrides 
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.

Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.

- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
 2021-01-26 02:04:12 +00:00
Andrew Noyes
adde95bea5 Merge branch 'release-6.3' into anoyes/merge-6.3-to-master 
Commit merge conflicts for now
 2021-01-25 18:55:24 +00:00
Steve Atherton
0d32bd0c44 Bug fixes involving use of AFCached rate control in simulation. 2021-01-24 01:50:09 -08:00
Steve Atherton
523a8c08db Merge branch 'add-throttling-on-AsyncfileCached' of github.com:sfc-gh-clin/foundationdb into afcache-write-limit 
# Conflicts:
#	flow/Knobs.h
 2021-01-17 05:30:55 -08:00
Steve Atherton
11927f6fff Another refactor of SQLite injected error handling to address failures of the previous attempt. Some code cleanup. Added documentation of injected error handling works and why. 2021-01-17 04:17:13 -08:00
Steve Atherton
f1adafaf8c Refactored how injected fault state is managed to make debugging injected fault handling easier. Moved injected error tracking for open() into an INetwork global since it should be unique per simulated process. Fixed bug where injected error in WAL file was not recognized as injected because it occurred before SQLiteDB's vfsWAL pointer was initialized. Simplified logic in SQLiteDB. Added comments. 2021-01-16 05:04:30 -08:00
Steve Atherton
8df6874417 Bug fixes: Check for null pager and wal pointers, and initialize vfsWAL after SQLite has opened the WAL. 2021-01-15 21:13:31 -08:00
Steve Atherton
6da78b1b19 Rewrote how injected faults are handled in SQLite to be more reliable and work with an upcoming write throttling feature in AsyncFileCached. 2021-01-15 19:29:14 -08:00
Chaoguang Lin
0613a149a6 Add rate control on both db file and log file 2020-12-11 00:22:26 -08:00
Chaoguang Lin
9156d38216 Add knob SQLITE_READER_THREADS to control the number of read threads 2020-12-09 12:09:07 -08:00
Chaoguang Lin
5b55216252 Add rate control to AsyncFileCached 2020-12-09 12:03:32 -08:00
Andrew Noyes
7fbc4d7391 Resolve conflicts 2020-12-04 23:58:42 +00:00
Andrew Noyes
877997632d Merge branch 'release-6.3' into anoyes/merge-release-6.3-master 
Include conflict markers for review purposes
 2020-12-04 01:38:07 +00:00
sfc-gh-tclinkenbeard
aa07df6a91 Backport to 6.3 the ability to read xxhash3 checksum for sqlite pages 2020-11-24 00:28:49 -08:00
Trevor Clinkenbeard
4029217c2f
Update fdbserver/KeyValueStoreSQLite.actor.cpp 
Co-authored-by: Andrew Noyes <63815641+sfc-gh-anoyes@users.noreply.github.com>
 2020-11-23 10:07:08 -08:00
sfc-gh-tclinkenbeard
156a617ed8 Replace xxhash64 with xxhash3 
The goal here is to improve performance
 2020-11-21 18:25:35 -08:00
sfc-gh-tclinkenbeard
bcda617b80 Benchmark hashing algorithms in flowbench 2020-11-17 23:14:55 -08:00
sfc-gh-tclinkenbeard
e9bceda8ca Add code coverage macros to PageChecksumCodec::checksum 2020-11-16 19:13:18 -08:00
sfc-gh-tclinkenbeard
287ab51502 Fix some bugs in PageChecksumCodec::checksum 2020-11-15 20:35:05 -08:00
sfc-gh-tclinkenbeard
145cddbb99 Clear the first 8 bits of xxHash64 checksum of SQLite pages 
When upgrading a cluster to 7.0 which has stale data from 6.2, we don't
want to calculate an xxHash64 checksum for every page that's read. By
clearing the first 8 bits of all checksums written using xxHash64, the
server can skip this checksum in 255/256 cases when reading pages
written from version <=6.2.
 2020-11-15 17:48:44 -08:00
sfc-gh-tclinkenbeard
c5689c4a72 Use xxhash64 for checksumming SQLite pages 2020-11-15 17:48:44 -08:00
sfc-gh-tclinkenbeard
4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
Xin Dong
410d418c3e Revert "Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining" 
This reverts commit e258dffc65a2cf3e7768f2c9f833f9f553a52d74, reversing
changes made to 0e96233015c25608df620a8c1b81f9945bbd6a33.

SS commit seems to be causing write performance degredation.
 2020-10-19 09:28:38 -07:00