Writing a shell script correctly can be hard even for a skilled
programmer. shellcheck is a static analysis tool that helps catch
common errors in shell scripts. We now have 36 executable scripts in
our repository, for which shellcheck reports 126 errors (calculated
like find . -type f -executable -exec bash -c '[ "$(file --brief
--mime-type "$1")" == "text/x-shellscript" ]' sh {} \; -exec shellcheck
-f gcc --exclude=SC2086 {} \; | cut -d: -f1 | sort | uniq | wc -l).
This commit fixes these warnings and adds a GitHub actions workflow
that runs shellcheck on all the executable shell scripts in the
repository. The warning SC2086: Double quote to prevent globbing and
word splitting is disabled globally, because it has little practical
consequences, sometimes leads to false positives, and is general is too
widespread because people forget to quote.
If tools like `pg_regress` or `pg_isolation_regress` are not found, an
error is generated telling you that you need to disable the tests
explicitly using `REGRESS_CHECKS`. This is an inconvenience for the
user since in most cases you just want to build the system with what
you have.
This commit changes that by automatically disabling regression tests,
isolation tests, TAP tests, and formatting targets when tools cannot be
found, which means that users do not normally have to disable tests
explicitly. A notice is still printed that the targets are not added.
In addition, the commit adds an option `REQUIRE_ALL_TESTS` for the rare
case when you want to make sure that all tests are indeed executed (for
example, in CI runs), which by default is off.
The sanitizer test is running on PG11.1 where the cluster test is
expected to fail. This patch changes the sanitizer test to ignore
the test result of the cluster test.
This commit adds the ability to open remote transactions
that share transactional semantics with the ongoing local
transaction. This is done by adding txn hooks to the local
transaction that performs txn commands on remote nodes.
This commit adds the 1pc variant of the commit protocol.
A 2-pc variant is going to be added in an upcoming PR.
The 1pc variant exists mostly for performance testing (i.e. how much is the 2pc penalty).
But, there are also cases where 1pc is good enough (non-critical
data and/or data backed by Kafka).
Note that the remote_txn test is not run during sanitizer tests due to
an apparent bug in PostgreSQL. (The test fails on an assertion of a
global variable `AbortStartTime`, which is non-atomic although read
across the ServerLoop and signal handlers.)
Docker images are build with latest versions of PostgreSQL, thus
updating regression tests to run on the latest PG versions. Fixing
authentication for PG docker images, since it is required after a
recent change in docker-library/postgres@42ce743.
Running isolation tests on new versions of PG produces additional
output:
- Notifications are not missed to print postgres/postgres@ebd4992
- Notifications are prefixed with session name
postgres/postgres@a28e10e
- Timeout cancellation is printed in isolation tests
postgres/postgres@b578404
The expected outputs are modified to succeed on latest PG versions,
while the affected isolation tests are disabled for earlier versions
of PG.
The `test_sanitizer.sh` test failed because source code was being
copied from the host to the container as user `postgres` and this user
did not have read permissions on the mounted directory. This is fixed
by copying the files as `root` and then changing the owner to
`postgres`.
The commit also removes `wait_for_pg` since PostgreSQL server status is
not relevant for the tests since they start their own temporary
instance.
The commit also switches to use here-is documents for the execution for
readability purposes.
We were not building timescaledb with debug enabled in the sanitizer
tests, causing tests which needed debug symbols to fail. This commit
changes the sanitizer tests to include such symbols. We still enable
optimizations, as that results in more realistic build environment for
memory tests.
This commit also ensures some tests which should only be run in debug
builds are in fact only run in debug builds.
With the forthcoming new sanitizer image running tests in parallel
causes machines to run out of resources, and fail tests spuriously
This commit forces test_sanitizers.sh to run tests sequentially to
prevent this.
This commit adds a sanitizer run script, and updates travis to
run it in our nightly builds. Valgrind has been very slow and
unstable, so much so that we only run it during pre-release
tests, and even then it doesn't necissarily provide useful
output. The sanitizer tests run about 10x faster, and seem
stable.
We currently only run ASan and UBSan, as MSan has too many
issues within postgres's own initialization functions. As MSan
cannot be added to a binary that enables ASan, it would require
separate jobs anyway.
