apple/foundationdb
1
0
Fork 0
mirror of https://github.com/apple/foundationdb.git synced 2025-06-01 10:45:56 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,900 Commits 35 Branches 318 Tags
Commit Graph

4900 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Evan Tschannen
966668905e update installer WIX GUID following release 2019-05-12 20:08:50 -07:00
Evan Tschannen
23e1e355d5 update versions target to 6.1.7 2019-05-12 20:08:50 -07:00
Evan Tschannen
26fbbbf798
Merge pull request #1574 from etschannen/prepare-release-6.1.6 
update installer WIX GUID following release
6.1.6 		2019-05-12 19:06:57 -07:00
Evan Tschannen
84f8381905 update installer WIX GUID following release 2019-05-12 19:06:04 -07:00
Evan Tschannen
c07c36ca2a
Merge pull request #1573 from etschannen/release-6.1 
updated documentation for 6.1.6
 2019-05-12 19:04:53 -07:00
Evan Tschannen
6bcefcbf86
Merge pull request #1571 from ajbeamon/fix-invalid-trace-type 
TLS was creating trace events with invalid types that contained spaces.
 2019-05-12 19:03:43 -07:00
Evan Tschannen
59a9030ed6 updated documentation for 6.1.6 2019-05-12 19:02:09 -07:00
Evan Tschannen
0a9685c811
Merge pull request #1556 from alexmiller-apple/tstlog-6.1 
Fix replaceFile on Windows and Linux
 2019-05-12 18:56:09 -07:00
Alex Miller
4a7e0319c7 Refactor away pushlock. 
Pushing was already a serialized, sequential operation.

Instead make it explicit that there are two waits as part of a push:
1. The setup work to reserve a spot on in the file
2. The work of writing and sync'ing the data

And we return a Future<Future<Void>> to force these to be done sequentially.
 2019-05-10 20:30:52 -10:00
Alex Miller
ea12a54946 Rename DISK_QUEUE_MAX_TRUNCATE_EXTENTS -> ..._BYTES 
So as to not make filesystem assumptions.  This knob did technically
appear in (only the) 6.1.5 release, but this feature was broken 6.1.5,
so thus impossible to use anyway.
 2019-05-10 18:26:22 -10:00
Alex Miller
c95d09f9fd Convert truncate(0) to truncate(4KB) on Windows. 
Blindly, in case Windows doesn't like 0 length truncates too.
 2019-05-10 14:55:11 -10:00
Alex Miller
c502ed3d15 Fix a variety of problems stemming from a wait() being added to push(). 
And that this code was previously insufficiently tested.
 2019-05-10 14:55:11 -10:00
A.J. Beamon
c328b15d36 TLS was creating trace events with invalid types (containing spaces). 2019-05-10 14:51:20 -07:00
Alex Miller
fbc4e7b351 Add a release note 2019-05-08 23:10:18 -10:00
Alex Miller
510b0b2fcd Fix DiskQueue not replaceFile'ing frequently enough for the final time. 2019-05-08 23:08:25 -10:00
Alex Miller
c6c33a4daa Make replaceFile more likely to be tested. 2019-05-08 21:23:42 -10:00
Alex Miller
0d0f54d1e6 Fix IAsyncFileSystem::open() flags to stop a crash. 
OPEN_ATOMIC_WRITE_AND_CREATE was missing a required OPEN_CREATE.

I'm honestly baffled how this was missed in testing.
 2019-05-08 21:22:40 -10:00
Alex Miller
b50926c792 replaceFile is truncate(0) on windows 2019-05-08 21:22:14 -10:00
Evan Tschannen
16307e0d79
Merge pull request #1554 from etschannen/post-release-cleanup-6.1.5 
Post release cleanup 6.1.5
 2019-05-08 18:09:27 -07:00
Evan Tschannen
fa173ce187 update installer WIX GUID following release 2019-05-08 18:08:50 -07:00
Evan Tschannen
1a44eff761 update versions target to 6.1.6 2019-05-08 18:08:50 -07:00
Evan Tschannen
91547e71e9
Merge pull request #1551 from etschannen/prepare-release-6.1.5 
update installer WIX GUID following release
6.1.5 		2019-05-08 15:55:05 -07:00
Evan Tschannen
bd2077252c update installer WIX GUID following release 2019-05-08 15:54:32 -07:00
Evan Tschannen
16681a990e
Merge pull request #1550 from etschannen/release-6.1 
updated documentation for 6.1.5
 2019-05-08 15:53:05 -07:00
Evan Tschannen
977e64a4fe updated documentation for 6.1.5 2019-05-08 15:51:55 -07:00
Evan Tschannen
e8ef401aaa
Merge pull request #1545 from alexmiller-apple/tstlog-6.1 
Prefer re-creating a DiskQueue file rather than performing a large truncate
 2019-05-08 15:39:27 -07:00
Alex Miller
e4ba2f5788 Add an ending TraceEvent. 2019-05-08 12:35:12 -10:00
Alex Miller
c093017c2f Add a TraceEvent and release note. 2019-05-08 12:34:25 -10:00
Alex Miller
0685e6c1c7 Avoid large truncates in the DiskQueue. 
And instead create a new file while incrementally truncating the old one
down.  This avoids queueing up a massive number of filesystem metadata
operations in one call, thus flooding the disk with requests and
stalling out all other filesystem operations.

This sets the knobs so that a truncate of >10GB causes us to create a
new file rather than trying to truncate the old one.
 2019-05-08 12:33:31 -10:00
Alex Miller
36dfbf4fb3 Only truncate DiskQueues down to TLOG_HARD_LIMIT*2. 
DiskQueue shrinking was implemented for spill-by-reference, as now
a DiskQueue could grow "unboundedly" large.

Without a minimum file size, write burst workloads would cause the
DiskQueue to shrink down to 100MB, and then grow back to its usual ~4GB
size in a cycle.  File growth means filesystem metadata mutations, which
we'd prefer to avoid if possible since they're more unpredicatble in
terms of latency.

In a healthy cluster, the TLog never spills, so the disk of a single
DiskQueue file should stay less than 2*TLOG_SPILL_THRESHOLD.  In the
worst case of spill-by-value, the DiskQueue could grow to
2*TLOG_HARD_LIMIT.  Therefore, having this limit will cause DiskQueue
shrinking to never behave sub-optimally for spill-by-value, and will
cause the DiskQueue files to return to the optimal size with
spill-by-reference.
 2019-05-08 12:33:31 -10:00
Alex Miller
a269a784cc Convert push() into an actor. 2019-05-08 12:33:31 -10:00
Evan Tschannen
68c773987c
Merge pull request #1544 from etschannen/release-6.1 
The team tracker does not provide data movement priority information for non-failure related data movement
 2019-05-08 11:39:17 -07:00
Alvin Moore
f7b74be520
Merge pull request #1523 from AlvinMooreSr/cmake-vendor 
Fixed CMake Vendor Config
 2019-05-08 08:21:42 -07:00
Evan Tschannen
0d4fcd526b
Merge pull request #1543 from bnamasivayam/release-6.1 
Add a workload to trigger repeated recoveries.
 2019-05-07 18:39:29 -07:00
Balachandar Namasivayam
d45e7bf0b1 Addressed review comments 2019-05-07 17:19:59 -07:00
Evan Tschannen
30ebdc3b47 updated release notes 2019-05-07 17:15:45 -07:00
Evan Tschannen
d9a4553270 fix: The team tracker does not provide data movement priority information for non-failure related data movement 2019-05-07 17:06:54 -07:00
Balachandar Namasivayam
5d824f5fbc Address review comments 2019-05-07 17:06:52 -07:00
A.J. Beamon
adc0800b93
Merge pull request #1541 from etschannen/feature-trace-safety 
Fixed a crash when memory tracking trace events are called recursively
 2019-05-07 17:06:28 -07:00
Evan Tschannen
c1a58bd27a updated release notes 2019-05-07 17:04:46 -07:00
Balachandar Namasivayam
a0cc3d98a1 Add a workload to trigger repeated recoveries. 2019-05-06 18:16:44 -07:00
Evan Tschannen
aefd68e1e7 The memory tracking trace events can crash if the memory that is being allocated is coming from a trace event itself. Specifically TDMetrics within trace events uses fast allocated memory. Trace events are supposed to be short lived, so this commits adds a global count of outstanding trace events, and disables memory tracking when a trace event exists. 2019-05-06 17:41:32 -07:00
Evan Tschannen
61018dae71 Merge branch 'release-6.1' of github.com:apple/foundationdb into release-6.1 2019-05-06 17:37:35 -07:00
Evan Tschannen
93eb2a9395
Merge pull request #1527 from alexmiller-apple/tstlog-6.1 
Spill-by-reference knob + TLog6.0 Spilled Peek deprioritization
 2019-05-03 17:19:45 -07:00
Alex Miller
c918b21137 Deprioritize spilled peeks in spill-by-value, and improve its logic. 
This deprioritizes before calling peekMessagesFromMemory, which should
improve the memory usage of the TLog, and makes sure to keep txsTag
peeks at a high priority to help recoveries stay fast.
 2019-05-03 15:27:11 -07:00
Alex Miller
4052f3826a Add a knob to limit the number of commits indexed per key. 
Theoretically, we could spill 20MB of 22B mutations for one key, which
would generate a very long value being stored in SQLite, and very
inefficiently read back.  This stops that from being a problem, at the
cost of some extra write calls.
 2019-05-03 15:27:10 -07:00
Evan Tschannen
12088119d2
Merge pull request #1517 from alexmiller-apple/tstlog-6.1 
Add a knob to limit amount of data read from sqlite for one PeekRequest.
 2019-05-03 11:01:11 -07:00
Alvin Moore
085a9242e4 Changed specification of the path to the License.txt file since previous value resulted in search at top level (/) directory. 2019-05-03 00:12:07 -07:00
Alvin Moore
f308549be7 Commented out Readme file for mac os until working within packaging 2019-05-02 23:45:16 -07:00
Alvin Moore
345e8496ab Fixed the cmake package vendor and email information in order to prevent cpack errors 2019-05-02 22:14:18 -07:00