apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-05-15 10:22:20 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,059 Commits 35 Branches 318 Tags
Commit Graph

51 Commits

Author SHA1 Message Date
A.J. Beamon
5f55f3f613 Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used. 2019-05-10 14:01:52 -07:00
Andrew Noyes
6207d724f8 Fix all -Wunused-variable warnings 2019-04-15 18:13:00 -07:00
Evan Tschannen
cd5c9d91fa
Merge pull request #1443 from etschannen/master 
Merge 6.1 into master
 2019-04-10 17:43:07 -07:00
A.J. Beamon
a7288e1325 Throw process_behind instead of future_version when all storage nodes on a team are behind. process_behind gets the same backoff behavior as not_committed. Add proxy_memory_limit_exceeded to the retryable predicate. 2019-04-08 14:21:24 -07:00
A.J. Beamon
45eb6b0973 Update fdbclient/BackupAgentBase.actor.cpp 
Co-Authored-By: mpilman <markus@pilman.ch>
 2019-04-05 13:12:19 -07:00
A.J. Beamon
389ec1c2ac Update fdbclient/BackupAgentBase.actor.cpp 
Co-Authored-By: mpilman <markus@pilman.ch>
 2019-04-05 13:12:19 -07:00
mpilman
1c16f87a4e Remove trace-calls to printable (in non-workloads) 2019-04-05 13:12:19 -07:00
Evan Tschannen
b6008558d3 renamed BinaryWriter.toStringRef() to .toValue(), because the function now returns a Standalone<StringRef>() 
eliminated an unnecessary copy from the proxy commit path
eliminated an unnecessary copy from buffered peek cursor
 2019-03-28 11:52:50 -07:00
A.J. Beamon
1429ffe8ab Initialize the tm.tm_isdst field because it isn't set by strptime. 2019-03-26 09:00:45 -07:00
A.J. Beamon
76e6a3f56d
Update fdbclient/BackupAgentBase.actor.cpp 
Co-Authored-By: satherton <stevea@apple.com>
 2019-03-22 11:02:38 -07:00
Stephen Atherton
cabe7ca844 Stopped using %z to parse timezone offset with strptime() because it only seems to work as expected on MacOS. Updated time input/output unit tests so that they don't assume what the local timezone is. 2019-03-21 19:38:07 -07:00
Stephen Atherton
ddf25e50ec _get_timezone() was being called incorrectly. Refactored getting / using current timezone offset slightly. 2019-03-20 12:36:08 -07:00
Stephen Atherton
22ac25c868 Updated comments for clarity, changed _timezone to _get_timezone() on Windows. Added TODO to eventually use std::get_time() based implementation on all platforms. 2019-03-20 11:53:24 -07:00
Stephen Atherton
81caf66b1c On Windows, use -_timezone instead of tm.tm_gmtoff. 2019-03-20 02:51:40 -07:00
Stephen Atherton
c193bc8cb5 Revert parseTime() back to using strptime() for Linux and MacOS. 2019-03-20 02:39:49 -07:00
Stephen Atherton
827107469f Fixed typo in attempt to get tm.tm_gmtoff to exist on Windows. 2019-03-20 02:19:36 -07:00
Stephen Atherton
bb58114ab1 Attempt to get tm.tm_gmtoff to be defined on Windows. 2019-03-20 01:53:50 -07:00
Stephen Atherton
3eadcf835b Added missing header. 2019-03-20 01:36:25 -07:00
Stephen Atherton
d5e50e6963 Rewrote BackupAgentBase::parseTime() to use std::get_time() so it compiles on all supported platforms. Added unit test for parseTime() and formatTime(). 2019-03-20 01:18:37 -07:00
Evan Tschannen
3da85f3acd implemented the \xff/metadataVersion key, which can be used by layers to help them cheaply cache metadata and know when their cache is invalid 2019-02-28 17:45:00 -08:00
mpilman
78dd80ea8a Proper fwd decl in BackupAgent 
Also BackupAgent.h -> BackupAgent.actor.h
 2019-02-19 15:16:59 -08:00
mpilman
479a4d7c22 Minor fixes in fdbclient for intellisense 2019-02-19 15:16:59 -08:00
Robert Escriva
268093a96d Adjust all includes to be relative to the root. 
Remove the use of relative paths.  A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h".  Adjust so that every include references such a header with the
latter form.

Signed-off-by: Robert Escriva <rescriva@dropbox.com>
 2018-10-19 17:35:33 +00:00
Evan Tschannen
3922e477a5 Merge branch 'release-6.0' 
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/ManagementAPI.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/LogSystemDiskQueueAdapter.actor.cpp
#	fdbserver/SimulatedCluster.actor.cpp
#	fdbserver/TLogServer.actor.cpp
 2018-10-03 16:57:18 -07:00
A.J. Beamon
b727f0475c Fix warnings about ACTORs not having waits. Fix shadowing of future variable in KVFileIntegrityCheck. 2018-09-10 10:51:41 -07:00
Alex Miller
535b5701e5 Rewrite all Void _ = wait(...) -> wait(...). 
This takes advantage of the new actorcompiler functionality to avoid
having duplicate definitions of `Void _` when trying to feed the
un-actorompiled source through clang.
 2018-08-14 15:50:26 -07:00
A.J. Beamon
99c9958db7 Some more trace event normalization 2018-06-08 13:57:00 -07:00
A.J. Beamon
c12b235080 Fix case in a few commented out trace events 2018-06-08 11:20:06 -07:00
A.J. Beamon
e5488419cc Attempt to normalize trace events: 
* Detail names now all start with an uppercase character and contain no underscores. Ideally these should be head-first camel case, though that was harder to check.
* Type names have the same rules, except they allow one underscore (to support a usage pattern Context_Type). The first character after the underscore is also uppercase.
* Use seconds instead of milliseconds in details.

Added a check when events are logged in simulation that logs a message to stderr if the first two rules above aren't followed.

This probably doesn't address every instance of the above problems, but all of the events I was able to hit in simulation pass the check.
 2018-06-08 11:11:08 -07:00
A.J. Beamon
216404de45 Merge branch 'release-5.2' of github.com:apple/foundationdb 
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	versions.target
 2018-06-06 15:25:37 -07:00
Evan Tschannen
4120062bb9 fix: backup initialized its begin version at 1 instead of the read version of the starting transaction 
fix: erasing log ranges did not properly divide up work between transactions to prevent making transactions which were too large
 2018-06-06 13:05:53 -07:00
Alec Grieser
69e831d522
Merge remote-tracking branch 'upstream/release-5.2' into merge-release-5.2 2018-04-28 17:44:52 -07:00
Yichi Chiang
6bddf8aefa Upgrade DR from 5.1 to 5.2 2018-04-26 17:24:40 -07:00
yichic
ede5cab192
Merge pull request #89 from yichic/share-log-mutations-5.2 
Share log mutations 5.2
 2018-03-19 12:01:26 -07:00
Yichi Chiang
ec02e54f64 Refactor EraseLogData() 2018-03-19 11:56:01 -07:00
Yichi Chiang
1f2602d2b3 Fix all review comments 2018-03-19 11:33:33 -07:00
Yichi Chiang
d6559b144f Share log mutations between backups and DRs which have the same backup range 2018-03-19 11:32:50 -07:00
Yichi Chiang
582b875a05 Refactor EraseLogData() 2018-03-16 18:09:23 -07:00
Yichi Chiang
f12c1d811c Fix all review comments 2018-03-16 18:09:23 -07:00
Yichi Chiang
26b93ff920 Share log mutations between backups and DRs which have the same backup range 2018-03-16 18:09:23 -07:00
Evan Tschannen
cf9d02cdbd
Merge pull request #48 from apple/release-5.2 
Merge release-5.2 into master
 2018-03-08 13:21:26 -08:00
Stephen Atherton
dcf5b2e35d All readCommitted() functions now use Transaction instead of ReadYourWritesTransaction to reduce memory consumption in Backup and DR. Also removed one readCommitted() variant as it is just a special case of another definition. 2018-03-07 13:56:34 -08:00
Evan Tschannen
37a6a81634 Merge commit '7f6fc3e039c911cd84b8540f7f799fc38a1c1822' into feature-remote-logs 
# Conflicts:
#	fdbserver/workloads/RestartRecovery.actor.cpp
 2018-02-23 12:33:28 -08:00
Alec Grieser
0bae9880f1 remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py 2018-02-21 10:25:11 -08:00
Evan Tschannen
fbadcc6eea changing a storage server’s tag must be the first mutations applied in a version, because privatized mutations applied earlier in the same version will use the old tag 2018-02-09 18:21:29 -08:00
Stephen Atherton
e8f9568bbe Simulation improvement, readCommitted() calls that run for a long time would sometimes go too slow depending on the buggified limit, so now the limit is updated for each fetch loop. 2017-12-21 17:21:05 -08:00
Alex Miller
f70e3b9fe8 Add or change a bunch of comments to provide descriptions of function contracts. 
This cleans up a bit of the VersionStamp DR work I did, and leaves hints and
advice for anyone who will be touching mutation applying code in the future.
 2017-12-20 16:57:14 -08:00
Stephen Atherton
3dfaf13b67 IBackupContainer has been rewritten to be a logical interface for storing, reading, deleting, expiring, and querying backup data. The details of how the data is organized or stored is now hidden from users of the interface. Both the local and blobstore containers have been rewritten, the key changes being a multi level directory structure and no more use of temporary files or pseudo-symlinks in the blob store implementation. This refactor has a large impact radius as the previous backup container was just a thin wrapper that presented a single level list of files and offered no methods for managing or interpreting the file structure so all of that logic was spread around other places in the code base. This made moving to the new blob store schema very messy, and without this refactor further changes in the future would only be worse. 
Several backup tasks have been cleaned up / simplified because they no longer need to manage the ‘raw’ structure of the backup.  The addition of IBackupFile and its finish() method simplified the log and range writer tasks.  Updated BlobStoreEndpoint to support now-required bucket creation and bucket listing prefix/delimiter options for finding common prefixes.  Added KeyBackedSet<T> type.  Moved JSONDoc to its own header.  Added platform::findFilesRecursively().

Still to do:  update command line tool to use new IBackupContainer interface, fix bugs in Restore startup.
 2017-11-14 23:33:17 -08:00
Alex Miller
f9b7ce9a2f Add write conflict ranges to metadata modifications on backup data dumps. 2017-09-29 15:58:36 -07:00
Evan Tschannen
ef41b07bb3 renamed past_version to transaction_too_old 
implemented read_lock_aware option
 2017-09-28 16:35:08 -07:00