apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-22 06:40:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
13,678 Commits 35 Branches 318 Tags
Commit Graph

71 Commits

Author SHA1 Message Date
Andrew Noyes
7fbc4d7391 Resolve conflicts 2020-12-04 23:58:42 +00:00
Andrew Noyes
877997632d Merge branch 'release-6.3' into anoyes/merge-release-6.3-master 
Include conflict markers for review purposes
 2020-12-04 01:38:07 +00:00
sfc-gh-tclinkenbeard
aa07df6a91 Backport to 6.3 the ability to read xxhash3 checksum for sqlite pages 2020-11-24 00:28:49 -08:00
Trevor Clinkenbeard
4029217c2f
Update fdbserver/KeyValueStoreSQLite.actor.cpp 
Co-authored-by: Andrew Noyes <63815641+sfc-gh-anoyes@users.noreply.github.com>
 2020-11-23 10:07:08 -08:00
sfc-gh-tclinkenbeard
156a617ed8 Replace xxhash64 with xxhash3 
The goal here is to improve performance
 2020-11-21 18:25:35 -08:00
sfc-gh-tclinkenbeard
bcda617b80 Benchmark hashing algorithms in flowbench 2020-11-17 23:14:55 -08:00
sfc-gh-tclinkenbeard
e9bceda8ca Add code coverage macros to PageChecksumCodec::checksum 2020-11-16 19:13:18 -08:00
sfc-gh-tclinkenbeard
287ab51502 Fix some bugs in PageChecksumCodec::checksum 2020-11-15 20:35:05 -08:00
sfc-gh-tclinkenbeard
145cddbb99 Clear the first 8 bits of xxHash64 checksum of SQLite pages 
When upgrading a cluster to 7.0 which has stale data from 6.2, we don't
want to calculate an xxHash64 checksum for every page that's read. By
clearing the first 8 bits of all checksums written using xxHash64, the
server can skip this checksum in 255/256 cases when reading pages
written from version <=6.2.
 2020-11-15 17:48:44 -08:00
sfc-gh-tclinkenbeard
c5689c4a72 Use xxhash64 for checksumming SQLite pages 2020-11-15 17:48:44 -08:00
sfc-gh-tclinkenbeard
4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
Xin Dong
410d418c3e Revert "Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining" 
This reverts commit e258dffc65a2cf3e7768f2c9f833f9f553a52d74, reversing
changes made to 0e96233015c25608df620a8c1b81f9945bbd6a33.

SS commit seems to be causing write performance degredation.
 2020-10-19 09:28:38 -07:00
sfc-gh-tclinkenbeard
dc4612712d Merge remote-tracking branch 'origin/master' into use-override-more 2020-10-09 07:12:41 -07:00
Steve Atherton
e258dffc65
Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining 
Pipeline the commit process in SS.
 2020-10-07 20:34:57 -07:00
sfc-gh-tclinkenbeard
8571dcfe28 Use override where applicable in fdbserver 2020-10-07 18:41:19 -07:00
Xin Dong
34d1d04904 Commit pipelining attempt #2 2020-09-23 20:03:04 -07:00
sfc-gh-tclinkenbeard
0814841827 Replace NULL with nullptr in fdbserver 2020-09-20 11:31:49 -07:00
sfc-gh-tclinkenbeard
77e3b314f6 Mark IKeyValueStore::getStorageBytes const 2020-06-17 18:20:43 -07:00
sfc-gh-tclinkenbeard
2a393633b6 Improved const-correctness 
Using the recently added IndexedSet::const_iterator
(https://github.com/apple/foundationdb/pull/3185), we can improve the
const-correctness of many functions. In this PR const is added where
applicable. Also, wherever I came across the following while adding
const, I made the following changes:

- virtual function overrides are marked as override
- NULL is replaced with nullptr
- git clang-format is applied
 2020-06-17 18:20:43 -07:00
A.J. Beamon
1c6aef76b5 When one of the sqlite reader or writer thread pools fail, fail the other with the same error. 2020-02-24 12:39:04 -08:00
Evan Tschannen
96258b9809 Merge branch 'release-6.2' 
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
 2020-02-21 19:09:16 -08:00
A.J. Beamon
df2b0452b4 Step 3 of fixing storage server range reads: change return type of readRange from VectorRef<KeyValueRef> to RangeResultRef. 2020-02-06 13:19:24 -08:00
mpilman
d09e07f1f5 Merge remote-tracking branch 'upstream/master' into features/icc 2020-02-04 10:26:18 -08:00
Alex Miller
da73164eda Move crc32c from fdbrpc to flow 
So that we can use it from a piece of flow code without breaking module
boundaries.

Also rename generated-constants to crc32c-generated-constants so that
it's more apparent that they're related files.
 2020-01-13 18:19:30 -08:00
Andrew Noyes
41583aa576 Guard unlikely indexCode calculation per Steve 2019-12-03 21:48:30 -08:00
Andrew Noyes
36e9f40fc2 Fix negative shift exponent 
fdbserver/KeyValueStoreSQLite.actor.cpp:438:11: runtime error: shift exponent -1 is negative
 2019-12-02 12:51:33 -08:00
Alex Miller
7a500cd37f A giant translation of TaskFooPriority -> TaskPriority::Foo 
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
 2019-06-25 02:47:35 -07:00
mpilman
844dd60202 FDB compiling with intel compiler 2019-06-20 09:29:01 -07:00
Steve Atherton
ba52623637
Merge pull request #1582 from tclinken/features/sqlite-crc32c 
Use crc32 for sqlite page checksums
 2019-06-17 14:20:41 -07:00
A.J. Beamon
603721e125 Merge branch 'master' into thread-safe-random-number-generation 
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/genericactors.actor.cpp
#	fdbrpc/sim2.actor.cpp
#	fdbserver/DiskQueue.actor.cpp
#	fdbserver/workloads/BulkSetup.actor.h
#	flow/ActorCollection.actor.cpp
#	flow/Net2.actor.cpp
#	flow/Trace.cpp
#	flow/flow.cpp
 2019-05-23 08:35:47 -07:00
Evan Tschannen
f4fbaac6b0 Merge branch 'release-6.1' 
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	versions.target
 2019-05-19 10:27:59 -07:00
Trevor Clinkenbeard
12ff747e6a Avoid tracing in PageChecksumCodec::checksum if silent flag is set 2019-05-17 10:49:53 -07:00
Trevor Clinkenbeard
3fac380b90 Avoid tracing in PageChecksumCodec::checksum if silent flag is set 2019-05-17 10:43:28 -07:00
Trevor Clinkenbeard
20e93c67ea Allow sqlite pages to be checked for CRC32 checksum 
Future versions of FDB will write sqlite pages with CRC32 checksums. In
order to roll back to this version from a version that writes CRC32
checksums, this version must be able to verify those checksums.
 2019-05-17 01:05:06 -07:00
Trevor Clinkenbeard
d7bcbe1210 Refactored PageChecksumCodec::checksum 2019-05-16 16:07:35 -07:00
Trevor Clinkenbeard
90d886df95 Trace both hashlittle2 and crc32 checksums for SQLitePageChecksumFailure 2019-05-16 15:51:21 -07:00
Trevor Clinkenbeard
04a72bdad6 Eliminate duplicate code in PageChecksumCodec::checksum 2019-05-16 11:09:37 -07:00
Trevor Clinkenbeard
aca90cd4e2 Don't use memcpy in PageChecksumCodec::checksum 2019-05-16 07:25:58 -07:00
Trevor Clinkenbeard
601c38ad82 Use crc32 for sqlite page checksums 2019-05-14 13:43:55 -07:00
A.J. Beamon
5f55f3f613 Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used. 2019-05-10 14:01:52 -07:00
Evan Tschannen
2d5043c665 Merge branch 'release-6.1' 
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	versions.target
 2019-04-30 18:27:04 -07:00
Evan Tschannen
9ff8aca1da Increased the SQLITE_CHUNK_SIZE to 100MB (left at 4MB for simulation) 2019-04-26 13:53:56 -07:00
A.J. Beamon
253d2400ef Merge branch 'release-6.1' into speed-up-and-parameterize-spring-cleaning 
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
 2019-04-23 14:38:52 -07:00
A.J. Beamon
ea7abff9df Clean up from review 2019-04-23 14:16:52 -07:00
A.J. Beamon
4ad0496b39 Increase the frequency that lazy deletes are run. Add more parameters for better control over the spring cleaning process. 2019-04-23 14:01:51 -07:00
Stephen Atherton
83db547306 Implemented the chunk size and db size hint fileControl options in our SQLite VFS implementation. KeyValueStoreSQLite now sets file chunk size based on a new knob, SQLITE_CHUNK_SIZE_PAGES. 2019-04-23 04:50:58 -07:00
Evan Tschannen
6220a5ce0f
Merge pull request #1370 from jzhou77/fix-unreferenced 
Remove unused functions
 2019-04-09 11:49:45 -07:00
mpilman
1c16f87a4e Remove trace-calls to printable (in non-workloads) 2019-04-05 13:12:19 -07:00
mpilman
bb82f8560a process all volatile ints correctly in traces 2019-04-05 13:12:19 -07:00
mpilman
c008e16c81 Defer formatting in traces to make them cheaper 
This is the first part of making `TraceEvent` cheaper. The main idea is
to defer calls to any code that formats string. These are the main
changes:

- TraceEvent::detail now takes a c-string instead of std::string for
  literals. This prevents unnecessary allocations if the trace is not
  going to be printed in the first place (for example for SevDebug).
  Before that `detail` expected a `std::string` as key, which mean that
  any string literal would be copied on each call.
- Templates Traceable and SpecialTraceMetricType. These templates can be
  specialized for any type that needs to be printed. The actual
  formatting will be deferred to after the `enabled` check. This
  provides two benefits: (1) if a TraceEvent is disabled, we don't pay
  for the formatting and (2) TraceEvent can trace types that it doesn't
  know about.
- TraceEvent::enabled will be set in the constructor if the Severity is
  passed. This will make sure that `TraceEvent::init` is not called.
- `TraceEvent::detail` will be inlined. So for disabled TraceEvent
  calls, a call to detail will only introduce a if-branch which is much
  cheaper than a function call.
 2019-04-05 13:12:19 -07:00