Zhe Wang 83f42e13d9
Make BulkDump work with S3 (#11822)
* init

* Add bulkdump to blobstore:// (s3)

* cmake/CompileBoost.cmake
 Add boost url. Needed parsing blobstore:// urls.

* documentation/sphinx/source/bulkdump.rst
 Minor edit to allow addition of blobstore target.

* fdbcli/BulkDumpCommand.actor.cpp
* fdbclient/BulkDumping.cpp
 s/blobstore/s3/ -- more generic and aligns with
 how backup/restore refers to "s3" thingies.

* fdbclient/include/fdbclient/S3Client.actor.h
* fdbclient/S3Client.actor.cpp
 Add batch upload handler.

* fdbclient/tests/seaweedfs_fixture.sh
 Add  run seaweed method. Also look for
 weed and if installed use it else download.

* fdbserver/BulkDumpUtil.actor.cpp
 appendToPath does the right thing when passed an URL
 Add bulkDumpTransportBlobstore_impl.
 Add upload to blobstore.

* tests/loopback_cluster/run_custom_cluster.sh
 Complain if unrecognized arguments.

* Add ctest for bulkload with simple bulkdump test for now.

* Add new test to ctest list

* fix bugs

* nit

* nits

* nits

---------

Co-authored-by: stack <stack@duboce.com>
2024-12-18 13:29:36 -08:00
..
2021-07-19 16:15:23 -05:00
2023-10-25 14:04:43 +02:00
2024-02-07 01:16:00 +02:00
2022-12-15 12:29:36 -08:00
2024-02-07 01:16:00 +02:00
2019-07-30 18:36:10 -07:00
2021-09-21 09:35:32 -07:00
2024-02-07 01:16:00 +02:00
2024-02-07 01:16:00 +02:00
2024-02-07 01:16:00 +02:00
2024-02-07 01:16:00 +02:00
2019-06-12 17:08:45 +01:00
2020-01-30 15:01:03 +01:00
2020-09-25 14:55:44 -07:00
2021-09-21 09:35:32 -07:00
2024-12-03 22:01:07 -08:00
2024-02-07 01:16:00 +02:00
2021-06-28 16:19:55 -07:00