Update version numbers and add 2.4.2 to update tests.
We have to put the DROP FUNCTION back in latest-dev because
2.4.2 did not include the commit which removed the function
definitions.
When renaming a column on a continuous aggregate, only the user view
column was renamed. This commit changes this by applying the rename on
the materialized table, the user view, the direct view, and the partial
view, as well as the column name in the dimension table.
Since this also changes some of the table definitions, we need to
perform the same rename in the update scripts as well, which is done by
comparing the direct view and the user view to decide what columns that
require a rename and then executing that rename on the direct view,
partial view, and materialization table, as well as updating the column
name in the dimension table.
When renaming columns in tables with indexes, the column in the table
is renamed but not the column in the index, which keeps the same name.
When restoring from a dump, however, the column name of the table is
used, which create a diff in the update test. For that reason, we
change the update tests to not list index definitions as part of the
comparison. The existance of the indexes is still tracked and compared
since the indexes for a hypertable is listed as part of the output.
If a downgrade does not revert some changes, this will cause a diff in
the downgrade test. Since the rename is benign and not easy to revert,
this will cause test failure. Instead, we add a file to do extra
actions during a clean-rerun to prevent these diffs. In this case,
applying the same renames as the update script.
Fixes#3405
Since the isolationtester got modified to improve the query result
output which got backported in to PG12 and PG13 we have to adjust
all isolationtester output.
https://github.com/postgres/postgres/commit/4a05406
If tools like `pg_regress` or `pg_isolation_regress` are not found, an
error is generated telling you that you need to disable the tests
explicitly using `REGRESS_CHECKS`. This is an inconvenience for the
user since in most cases you just want to build the system with what
you have.
This commit changes that by automatically disabling regression tests,
isolation tests, TAP tests, and formatting targets when tools cannot be
found, which means that users do not normally have to disable tests
explicitly. A notice is still printed that the targets are not added.
In addition, the commit adds an option `REQUIRE_ALL_TESTS` for the rare
case when you want to make sure that all tests are indeed executed (for
example, in CI runs), which by default is off.
Add 2.3.1 to the update tests and update the downgrade target for the
downgrade version. This commit also fixes two issues that were fixed in
the release branch:
1. Drop `_timescaledb_internal.refresh_continuous_aggregate`
2. Update the changelog to match the release branch.
During an update, it is not possible to run the downgrade scripts until
the release has been tagged, but the update scripts can be run. This
means that we need to split the previous version into two different
fields: one for running the update tests and one for running the
downgrade tests.
This commit add functions and code to generate a downgrade script from
the current version to the previous version. This requires execution
from a Git repository since it retrieves the prolog and epilog for the
"downgrade" file from the version given by `update_from_version` in the
`version.config` file.
The commit adds several CMake functions that simplifies the composition
of script files, but these are not used to generate the update scripts.
A potential improvement is to use the scripts to also generate the
update scripts.
This commit supports generating a downgrade script from the
current version to the previous version. Other versions are handled
using a variable containing file names of reverse update
scripts and the source and target version is extracted from the file
names, which is assumed to be of the form
`<source-version>--<target-version>.sql`.
In addition to adding support for generating downgrade scripts, the
commit adds a downgrade test file that tests a release in a similar way
to the update script and adds it as a workflow.
Change sanitizer test to run on PG12 and make it use the same
infrastructure as the other linux regression tests.
Co-authored-by: Sven Klemm <sven@timescale.com>
The post-update script was handling preserving initprivs for newly
added catalog tables and views. However, newly added catalog sequences
need separate handling otherwise update tests start failing. We also
now grant privileges for all future sequences in the update tests.
In passing, default the PG_VERSION in the update tests to 12 since we
don't work with PG11 anymore.
Add a workflow to check that CMake files are correctly formatted as
well as a custom target to format CMake files in the repository. This
commit also runs the formatting on all CMake files in the repository.
Install the Perl prerequisites when building the image for ABI tests,
including the `prove` binary.
Although the ABI tests currently don't run TAP tests, CMake still
failed the configuration since it expects the prerequisites to be
installed unless it is run with `-DTAP_CHECKS=off`.
CMake now detects if the necessary prerequisites for running TAP
tests are installed on the local system. This includes perl
installation and other dependencies, such as the IPC::Run module and
prove binary.
GitHub removed default installation of clang-9 from the ubuntu
images but those are required to build postgres with llvm so we have
to explicitly install the required packages.
https://github.com/actions/virtual-environments/pull/3381
The TAP testing PG framework meets our requirements for doing
multinode/stand-alone-binaries testing for scenarios which need
individual multiple PG instances (multi-node testing is a prime
example).
This commit adds the necessary wrappers to allow the use of the TAP
testing framework in timescale code base. The README in "test/perl"
directory gives a fair idea of how to write tap tests. A simple tap
test has been added to provide a reference point for developers to
write new ones.
One can go to "build/tsl/test" and invoke "make checkprove" to see
the tap test in action.
Also includes changes for enabling github CI to run these taptests by
installing the requisite dependencies.
Includes changes to license checking scripts to handle new *.pl and
*.pm files.
Also added a new scripts/perltidyrc to aid in formatting of these
files.
Change CI tests to run against pg 11.12, 12.7 and 13.3.
The latest postgres versions adjusts the event_trigger test to
output the list of currently defined event triggers making the
test output different when timescaledb is installed since we
define our own event triggers. Due to this change we have to
ignore the result of this test in our CI.
https://github.com/postgres/postgres/commit/f3c45378
Grants and revokes where not propagated to compressed hypertables, if
the hypertable had a compressed hypertable, meaning that `pg_dump` and
`pg_restore` would not be able to dump nor restore the compressed part
of a hypertable unless they were owners and/or was a superuser.
This commit fixes this by propagating grants and revokes on a
hypertable to the associated compressed hypertable, if one exists,
which will then include the compressed hypertable and the associated
chunks in the grant and revoke execution.
It also adds code to fix the permissions of compressed hypertables and
all associated chunks in an update and adds an update test to check
that the permissions match.
Fixes#3209
Tables, indexes, and sequences that are recreated as part of an update
does not propagate permissions to the recreated object. This commit
fixes that by saving away the permissions in `pg_class` temporarily and
then copying them back into the `pg_class` table.
If internal objects are created or re-created, they get the wrong
initial privileges, which result in privileges not being dumped when
using `pg_dump`. Save away the privileges before starting the update
and restore them afterwards to make sure that the privileges are
maintained over the update.
For new objects, we use the initial privileges of the `chunk` metadata
table, which should always have correct initial privileges.
Fixes#3078
The update smoke test script can be used to test that an extension
works correctly. It will run the normal update tests, but do it against
a single server.
It also makes some minor refactorings of the update tests to make it
possible to run the tests against different versions of the server and
also support the smoke tests where some features cannot be tested.
Most places in the source tree use README.md as filename for
additional documentation. This patch changes the filenames of
files using a different capitalization.
Switch CI tasks currently running on Ubuntu 18.04 to 20.04 and
also switch some tasks that dont require specific version because
they run in docker containers to use ubuntu-latest instead.
All tasks switched to ubuntu-latest are expected to be ubuntu-version
agnostic because they run their actions in docker containers.
When building timescaledb on Ubuntu 20.04 mangle_path is present
as exported symbol in binaries, making the export prefix check fail.
This patch changes the export prefix check to ignore mangle_path.
In order to implement repair tests, changes are made to the
`constraint_check` table to simulate a broken dependency, which
requires the constraints on that table to be dropped. This means that
the repair runs without constraints, and a bug in the update test could
potentially not get caught.
This commit fixes this by factoring out the repair tests from the
update tests and run them as a separate pass. This means that the
contraints are not dropped in the update tests and bugs there will be
caught.
In addition, some bash functions are factored out into a separate file
to avoid duplication.
Due to a validation bug involving interval validation and the current
daylight savings time switch in the 1.7 releases the update tests that
includes 1.7 versions is currently failing. This patch changes CI
to run in GMT instead.
In order to support smoke-testing with a single server, the update
tests are refactored to not require a `postgres` user with full
privileges.
To support both running smoke tests and update tests, the following
changes where made:
- Database creation code was factored out of tests and is only executed
for update tests.
- Since the default for `docker_pgscript` was to use the `postgres`
database and the database creation code also switched database to
`single` as part of the exection, the default of `docker_pgscript` is
now changed to `single`.
- Parts of tests that changes roles during execution was removed since
it is more suitable for a regression test. Some `GRANT` statements
were kept for the update tests since they test that non-superuser
privileges are propagated correctly.
- Operations that require escalated privileges are factored out into
support functions that execute the original code for update tests and
are no-ops for smoke tests.
- A dedicated `test_update_smoke` script was added that can run a smoke
test against a single server.
This commit is part of a longer sequence to refactor the update tests
for use with smoke testing.
The multinode parts of the tests are broken out into a new version to
make the logic simpler.
- All multinode tests are now added to a new version, `v7` which allow
us to create multinode objects only for versions that support
multinode.
- Test script `test_updates_pg11` and `test_updates_pg12` are changed
to use `v6` for updates from versions preceeding 2.0 and `v7` for
versions 2.0 and later.
- Setup file `post.update.sql` is not needed any more since it was used
to "fix" a pre-2.0 updated version to add data nodes so that it
matched the clean setup. This is not necessary any more since v6 does
not add data nodes for some versions and not for others.
In order to support smoke-testing with a single server, the update
tests are refactored. This is part of a longer sequence of changes to
support both running smoke tests and update tests and contain the
following changes:
- All timestamp columns now have `not null` added to avoid spamming the
output file and help finding the real issue.
- Extension of dump files are changed from `sql` to `dump` since they
use the custom format and are not SQL files.
The sanitizer test is running on PG11.1 where the cluster test is
expected to fail. This patch changes the sanitizer test to ignore
the test result of the cluster test.
