GitHub removed default installation of clang-9 from the ubuntu
images but those are required to build postgres with llvm so we have
to explicitly install the required packages.
https://github.com/actions/virtual-environments/pull/3381
The TAP testing PG framework meets our requirements for doing
multinode/stand-alone-binaries testing for scenarios which need
individual multiple PG instances (multi-node testing is a prime
example).
This commit adds the necessary wrappers to allow the use of the TAP
testing framework in timescale code base. The README in "test/perl"
directory gives a fair idea of how to write tap tests. A simple tap
test has been added to provide a reference point for developers to
write new ones.
One can go to "build/tsl/test" and invoke "make checkprove" to see
the tap test in action.
Also includes changes for enabling github CI to run these taptests by
installing the requisite dependencies.
Includes changes to license checking scripts to handle new *.pl and
*.pm files.
Also added a new scripts/perltidyrc to aid in formatting of these
files.
Change CI tests to run against pg 11.12, 12.7 and 13.3.
The latest postgres versions adjusts the event_trigger test to
output the list of currently defined event triggers making the
test output different when timescaledb is installed since we
define our own event triggers. Due to this change we have to
ignore the result of this test in our CI.
https://github.com/postgres/postgres/commit/f3c45378
Grants and revokes where not propagated to compressed hypertables, if
the hypertable had a compressed hypertable, meaning that `pg_dump` and
`pg_restore` would not be able to dump nor restore the compressed part
of a hypertable unless they were owners and/or was a superuser.
This commit fixes this by propagating grants and revokes on a
hypertable to the associated compressed hypertable, if one exists,
which will then include the compressed hypertable and the associated
chunks in the grant and revoke execution.
It also adds code to fix the permissions of compressed hypertables and
all associated chunks in an update and adds an update test to check
that the permissions match.
Fixes#3209
Tables, indexes, and sequences that are recreated as part of an update
does not propagate permissions to the recreated object. This commit
fixes that by saving away the permissions in `pg_class` temporarily and
then copying them back into the `pg_class` table.
If internal objects are created or re-created, they get the wrong
initial privileges, which result in privileges not being dumped when
using `pg_dump`. Save away the privileges before starting the update
and restore them afterwards to make sure that the privileges are
maintained over the update.
For new objects, we use the initial privileges of the `chunk` metadata
table, which should always have correct initial privileges.
Fixes#3078
The update smoke test script can be used to test that an extension
works correctly. It will run the normal update tests, but do it against
a single server.
It also makes some minor refactorings of the update tests to make it
possible to run the tests against different versions of the server and
also support the smoke tests where some features cannot be tested.
Most places in the source tree use README.md as filename for
additional documentation. This patch changes the filenames of
files using a different capitalization.
Switch CI tasks currently running on Ubuntu 18.04 to 20.04 and
also switch some tasks that dont require specific version because
they run in docker containers to use ubuntu-latest instead.
All tasks switched to ubuntu-latest are expected to be ubuntu-version
agnostic because they run their actions in docker containers.
When building timescaledb on Ubuntu 20.04 mangle_path is present
as exported symbol in binaries, making the export prefix check fail.
This patch changes the export prefix check to ignore mangle_path.
In order to implement repair tests, changes are made to the
`constraint_check` table to simulate a broken dependency, which
requires the constraints on that table to be dropped. This means that
the repair runs without constraints, and a bug in the update test could
potentially not get caught.
This commit fixes this by factoring out the repair tests from the
update tests and run them as a separate pass. This means that the
contraints are not dropped in the update tests and bugs there will be
caught.
In addition, some bash functions are factored out into a separate file
to avoid duplication.
Due to a validation bug involving interval validation and the current
daylight savings time switch in the 1.7 releases the update tests that
includes 1.7 versions is currently failing. This patch changes CI
to run in GMT instead.
In order to support smoke-testing with a single server, the update
tests are refactored to not require a `postgres` user with full
privileges.
To support both running smoke tests and update tests, the following
changes where made:
- Database creation code was factored out of tests and is only executed
for update tests.
- Since the default for `docker_pgscript` was to use the `postgres`
database and the database creation code also switched database to
`single` as part of the exection, the default of `docker_pgscript` is
now changed to `single`.
- Parts of tests that changes roles during execution was removed since
it is more suitable for a regression test. Some `GRANT` statements
were kept for the update tests since they test that non-superuser
privileges are propagated correctly.
- Operations that require escalated privileges are factored out into
support functions that execute the original code for update tests and
are no-ops for smoke tests.
- A dedicated `test_update_smoke` script was added that can run a smoke
test against a single server.
This commit is part of a longer sequence to refactor the update tests
for use with smoke testing.
The multinode parts of the tests are broken out into a new version to
make the logic simpler.
- All multinode tests are now added to a new version, `v7` which allow
us to create multinode objects only for versions that support
multinode.
- Test script `test_updates_pg11` and `test_updates_pg12` are changed
to use `v6` for updates from versions preceeding 2.0 and `v7` for
versions 2.0 and later.
- Setup file `post.update.sql` is not needed any more since it was used
to "fix" a pre-2.0 updated version to add data nodes so that it
matched the clean setup. This is not necessary any more since v6 does
not add data nodes for some versions and not for others.
In order to support smoke-testing with a single server, the update
tests are refactored. This is part of a longer sequence of changes to
support both running smoke tests and update tests and contain the
following changes:
- All timestamp columns now have `not null` added to avoid spamming the
output file and help finding the real issue.
- Extension of dump files are changed from `sql` to `dump` since they
use the custom format and are not SQL files.
The sanitizer test is running on PG11.1 where the cluster test is
expected to fail. This patch changes the sanitizer test to ignore
the test result of the cluster test.
In contrast to the default for CMake files, `Release` is used as the
default in `CMakeFiles.txt` which causes the `bootstrap` script to do a
release build with development flags, in particular `-Werror`. Since
warnings are triggered in a release build, this cause the build to
fail while a debug build works fine.
This commit fixes this by removing the `-Werror` flag (by setting
`WARNINGS_AS_ERRORS` to `OFF`) on anything that is not a debug build
and also disable the warnings that (currently) trigger the warnings in
the release build.
The commit also changes some of the GitHub workflows to run without
`WARNINGS_AS_ERRORS` since it should always work without this option
regardless of build type (on release build this should be disabled, on
debug builds this should be enabled). But it is set to `ON` for the
full release and debug builds to ensure that we do not generate any
warnings, which will capture new surfacing warnings.
Fixes#2770
When changing the column type of a column that is part of an index
that is being clustered on with either postgres CLUSTER or reorder
the alter type operation would fail with a segfault because it
couldn't lookup the index.
The commit fixes two bugs in the repair scripts that could
prevent an update in rare circumstances.
For the 1.7.1--1.7.2 repair script: if there were several missing
dimension slices in different hypertables with the same column name,
the repair script would be confused on what constraint had what type
and generate an error.
For the 2.0.0-rc1--2.0.0-rc2 repair script: if a partition constraint
was broken, it would generate an error rather than repairing the
dimension slices because BIGINT_MIN would be cast to a double float and
then an attempt would be made to cast it back to bigint, causing an
overflow error.
This commit also creates an update repair test that breaks a few tables
for pre-2.0 versions to ensure that the repair script actually fixes
them. The integrity check for the update tests already contain a check
that dimension slices are valid, so there is no need to add a test for
that.
This commit adds an extra dimension in the workflow to test updates
with repair and run that separately. It also changes the update test
scripts to by default run without repair tests and add the additional
option `-r` for running repair tests in addition to the normal tests.
Fixes#2824
Copy ACL privileges (grants) from the query view (user-facing object)
to the internal objects (e.g., materialized hypertable, direct view,
and partial view) when updating the extension to the new version. A
previous change added such propagation of privileges when executing
`GRANT` statements, but didn't apply it retrospectively to the
internal objects of existing continuous aggregates.
Having the right permissions on internal objects is also necessary for
the watermark function used by real-time aggregation since it queries
the materialized hypertable directly.
The update script copies the ACL information from the user-facing view
of every continuous aggregate to its internal objects (including the
materialized hypertable and its chunks). This is done by direct insert
into `pg_class` instead of executing a `GRANT` statement in the update
script, since the latter will record the grant/ACL as an init
privilege (i.e., the system believes the GRANT is for an extension
object). The init privilege will prevent this ACL from being included
in future dump files, since `pg_dump` only includes non-init
privileges as it believes such privileges will be recreated with the
extension.
Fixes#2825
The update scripts now test extension updates for a multi-node setup
with one access node and one data node. Currently the following is
tested:
* A distributed hypertable is created in the database where the update
process is normally tested.
* A new database representing a data node is created. This database is
updated and restored in the same way as the other database. However,
no integrity or other validation checks are currently run on the
datanode database; only the ability to update the extension is
tested.
There are some flags that cannot be used in preprelease tests, which
this commit fixes. It uses the flag `-DWARNINGS_AS_ERRORS=OFF` to not
turn warnings into errors for release builds. In addition, explicit
checks for compiler flags are added instead of using a function, which
for some reason causes problems.
Note that the prerelease tests run on a regular schedule as well.
Some tests contain code that only work if the build is based on a Git
clone, which caused these tests to fail when Git was not available.
This commit splits out those tests and only enabling them if Git is
found and Git information can be retrieved.
The clang-format wrapper is always using `clang-format` even if a
different version of `clang-format` has been found and has the correct
version.
This commit fixes the issue by passing the name of the found version of
`clang-format` to the wrapper script using the `CLANG_FORMAT` varible.
It will default to use `clang-format` if `CLANG_FORMAT` is unset.
The command `get_git_commit` shows quite a little information and
especially if mistakes are made it is hard to verify what the binary is
actually based on.
This commit extends the function to provide some more information:
specifically the SHA of the HEAD regardless of whether there is a tag
on it or not and the time for the HEAD commit.
```
postgres=# select * from _timescaledb_internal.get_git_commit();
commit_tag | commit_hash | commit_time
--------------------------+-------------+------------------------
1.7.4-10-g09b0b778-dirty | 09b0b778 | 2020-09-13 17:50:38+02
(1 row)
```
If git is not installed, the function `get_git_commit` will return an
error indicating that no git information is available. If some of the
fields are available, they will be emitted and the remaining fields
will be NULL.
Fixes#2457
Disable NOTICE messages for time based tables in
continuous_aggs_policy test. The refresh sometimes
does not have anything to update as the intervals
are computed based off now(). This causes intermittent
failures due to NOTICEs that the cagg is up to date.
Remove continuous_aggs_policy from ignores list in CI
scripts.
