This patch adds a new settings file for common github action settings.
Instead of repeating latest pg versions in every github workflow we
can read the settings from this central file.
The output format of the isolation tester got changed by pg upstream
and backported to earlier versions. This means PG versions before
the backport have different output format then latest PG version.
We used to add all the isolation tests to the IGNORE list for those
PG versions but that is error prone and often forgotten when new
isolation tests are added. This patch skips the isolation test on
PG versions with incompatible output format.
The dump_meta_data sql script used an external call to date to get
the current data which does not work on windows. This patch changes
the script to use now() instead to get the current time and date.
Fixes#3674
Unlike the existing memory leak test which tests a single INSERT
of 100.000.000 rows in a single transaction, this runs 1000000
SELECT, 10K UPDATE and 50K INSERT queries in the same backend.
The isolation tester in earlier PG versions is lacking features we
rely on in our tests so the results of isolation tests are ignored
when testing on the earliest version of a major pg version.
The commit-msg hook was counting the end-of-line characters in the 50-
and 72-character limit, so commit messages had to be wrapped slightly
shorter in order to avoid the hook complaint. Strip the EOL characters
during the check instead.
Since we use universal newline handling during the file read, stripping
LF ('\n') should be enough to handle both Windows and Unix development
environments.
This git hook is copied automatically during CMake, and breaks commits
immediately afterwards if Python 2 isn't installed. Since Python 2 has
been end-of-life for a while now, take this chance to upgrade.
The deadlock_recompress_chunk isolation test uses syntax not
supported by earlier versions of the isolation tester leading
to a parse failure when processing that file on PG13.2.
This patch skips that particular test on PG13.2.
Commit 7b9d8673 added an isolation test for telemetry but did not
add it to the ignore list for earlier postgres versions. PG14 changed
the output format for isolation tests which got backported to PG12
and PG13 which makes the output of earlier PG12 and PG13 different
from the latest one so we ignore isolation tests on those earlier
versions.
With recent refactorings the telemetry test seems to trigger the
same use-after-free bug that got triggered by tablespace test so
we skip that test on PG12.0 as well.
This is a *temporary* change to our workflows to ignore the test
because it is flaky and requires many jobs to be restarted before a
commit can be merged. Does not address the issue that causes the
flakiness, which needs to be investigated.
We don't do this currently, so some queries return the wrong ordering
of rows if there is an index on the compressed chunk. The fix only
works for the newly created chunks. We could add a migration that
corrects the old compressed chunks, but it seems to be too heavy and
not to lend itself well to automation -- we'll have to recreate the
indexes if there are any. So the old chunks continue to return a wrong
result.
Add the missing variables to the finalization view of Continuous
Aggregates and the corresponding columns to the materialization table.
Cover the case of targets that contain Aggref nodes and Var nodes
that are outside of the Aggref nodes at the same time.
Stop rebuilding the Continuous Aggregate view with ALTER MATERIALIZED
VIEW. Attempt to repair the view at post-update time instead, and fail
gracefully if it is not possible to do so without raw hypertable schema
or data modifications.
Stop rebuilding the Continuous Aggregate view when switching realtime
aggregation on and off. Instead, manipulate the User View by either:
1. removing the UNION ALL right-hand side and the WHERE clause when
disabling realtime aggregation
2. adding the Direct View to the right of a UNION ALL operator and
defining WHERE clauses with the relevant watermark checks when
enabling realtime aggregation
Fixes#3898
Due to a security vulnerability in git recent git versions now check
ownership of files and error out if it doesnt match current user.
Since we bind mount the source checkout into the build container
the user of the checkout is unlikely to match the user inside the
container. This patch configures git to skip the owner check for
the bind-mounted directory.
https://github.blog/2022-04-12-git-security-vulnerability-announced/
Add option `USE_TELEMETRY` that can be used to exclude telemetry from
the compile.
Telemetry-specific SQL is moved, which is only included when extension
is compiled with telemetry and the notice is changed so that the
message about telemetry is not printed when Telemetry is not compiled
in.
The following code is not compiled in when telemetry is not used:
- Cross-module functions for telemetry.
- Checks for telemetry job in job execution.
- GUC variables `telemetry_level` and `telemetry_cloud`.
Telemetry subsystem is not included when compiling without telemetry,
which requires some functions to be moved out of the telemetry
subsystem:
- Metadata handling is moved out of the telemetry module since it is
used not only with telemetry.
- UUID functions are moved into a separate module instead of being
part of the telemetry subsystem.
- Telemetry functions are either added or removed when updating from a
previous version.
Tests are updated to:
- Not use telemetry functions to get UUID or Metadata and instead use
the moved UUID and metadata functions.
- Not include telemetry information in tests that do not require it.
- Configuration files do not set telemetry variables when telemetry is
not compiled in.
- Replaced usage of telemetry functions in non-telemetry tests with
other sources of same information.
Fixes#3931
Smoke tests where missing critical files and some tests had changed
since last run and did not handle update smoke tests, so fixing all
necessary issues.
TimescaleDB was vulnerable to a privilege escalation attack in
the extension installation script. An attacker could precreate
objects normally owned by the extension and get those objects
used in the installation script since the script would only try
to create them if they did not already exist. Thanks to Pedro
Gallegos for reporting the problem.
This patch changes the schema, table and function creation to fail
and abort the installation when the object already exists instead
of using the existing object.
Security: CVE-2022-24128
On MacOS Monterey with LLVM 13.0.1 when executing export_prefix_check.sh
we get an error:
/usr/local/opt/llvm/bin/nm: error: : unknown argument '-defined-only',
did you mean '--defined-only'?
This patch adds a better check of which flag does `nm` expect.
This patch adds coccinelle scripts for detecting use-after-free
bugs in relation to ts_cache_release.
This will find code using the following pattern:
Use of the hypertable from cache after releasing the cache
and use of the dimension of the hypertable from a cache
after the cache was released.
Change the prefix for continuous aggregate tests from
continuous_aggs_ to cagg_. This is similar to commit 6a8c2b66
which did this adjustment for isolation tests because we were
running into length limitations for the spec name. This patch
adjusts the remaining tests to be consistent with the naming
used in isolation tests.
Upstream changed the explain output for the memoize node to include
an additional `Cache Mode` line. While we could adjust our test
runner to always ignore that line this would prevent us from testing
the cache mode in future tests.
6c32c09777
Bump year in copyright information to 2022 and adjust same scripts
to reference NOTICE that didn't have the reference yet.
This patch also removes orphaned test/expected/utils.out.
Enable ALTER MATERIALIZED VIEW (timescaledb.compress)
This enables compression on the underlying materialized
hypertable. The segmentby and orderby columns for
compression are based on the GROUP BY clause and time_bucket
clause used while setting up the continuous aggregate.
timescaledb_information.continuous_aggregate view defn
change
Add support for compression policy on continuous
aggregates
Move code from job.c to policy_utils.c
Add support functions to check compression
policy validity for continuous aggregates.
