This patch removes PG 9.6 and PG 10 test output files and merges the
output files where PG11 output does not differ from PG12 output.
Removing code support for these versions will be in a subsequent patch.
This change moves the custom_type, ddl, ddl_single, insert, and
partition tests under test/sql, as well as the move and reorder
tests under /tsl/test/sql to our test template framework. This
allows them to have different golden files for different version of
Postgres.
With the exception of the reorder test, all of these tests produced
new output in PG12 which only differed by the exclusion of append
nodes from plan explanations (this exclusion is a new feature of
PG12). The reorder test had some of these modified plans, but also
included a test using a table with OIDs, which is not valid in PG12.
This test was modified to allow the table creation to fail, and we
captured the expected output in the new golden file for PG12.
Replace hardcoded database name from regression tests with :TEST_DBNAME
Remove creation of database single_2 from test runner and add it to
bgw_launcher and loader test since no other tests used those
use SQL comments in test scripts
Introduce PG11 support by introducing compatibility functions for
any whose signatures have changed in PG11. Additionally, refactor
the structure of the compatibility functions found in compat.h by
breaking them out by function (or small set of similar functions)
so that it is easier to see what changed between versions and maintain
changes as more versions are supported.
In general, the philosophy has been to try for forward compatibility
wherever possible, so that we use the latest versions of function interfaces
where we can or where reasonably convenient and mimic the behavior
in older versions as much as possible.
TimescaleDB has always supported functions on closed (space)
dimension, i.e., for hash partitioning. However, functions have not
been supported on open (time) dimensions, instead requiring columns to
have a supported time type (e.g, integer or timestamp). This restricts
the tables that can be time partitioned. Tables with custom "time"
types, which can be transformed by a function expression into a
supported time type, are not supported.
This change generalizes partitioning so that both open and closed
dimensions can have an associated partitioning function that
calculates a dimensional value. Fortunately, since we already support
functions on closed dimensions, the changes necessary to support this
on any dimension are minimal. Thus, open dimensions now support an
(optional) partitioning function that transforms the input type to a
supported time type (e.g., integer or timestamp type). Any indexes on
such dimensional columns become expression indexes.
Tests have been added for chunk expansion and the hashagg and sort
transform optimizations on tables that are using a time partitioning
function.
Currently, not all of these optimizations are well supported, but this
could potentially be fixed in the future.
Previously, we did not intercept the ALTER SCHEMA [name] RENAME command, which meant all Timescale catalog tables that store a schema name were not getting updated properly. This caused problems when users tried to drop a hypertable in a renamed schema, but also would have also caused problems with other commands (which we now add tests for in this PR).
Postgres assumes that exprs will live for the entire duration of the SQL
expression that they are created from and stores pointers into them
based on that assumption. As a memory usage optimization our INSERT
dynamically creats chunk constraints which live for only an individual
chunk insert, causing segfaults when postgres tries to read through the
cleaned up memory. To fix this we now free the per_tuple_exprcontext
_before_ deleting a chunk insert state memory context, ensuring the
segfaulting callback is called on a valid state.
Previously, if a hypertable dimension type did not have a default
hash function, create_hypertable would throw an error.
However, this should not be the case if a custom partitioning
function is provided.
This commit addresses the issue making sure that arbitrary
custom types can be used as partitioning dimensions as long
as a valid partitioning function is provided.
Fixes#470.
If the argument column of add_dimension is not in all
of hypertable indexes that have UNIQUE, PRIMARY KEY
or EXCLUSION constraints, then add_dimension call
should fail.
This commit enforces the above.
This adds a simple check to enforce that partitioning functions
are IMMUTABLE. This is a requirement since a partitioning function
must always return the same value given the same input.
Previously, when the FmgrInfo was initialized for partitioning
functions, the type information was cached in the fn_extra
field. However, the contract for fn_extra is that it is to be set by
the called function and not prior to its invokation. Setting fn_extra
prior to function invokation is an issue for custom partitioning
functions implemented in SQL, as the SQL call manager expects to use
fn_extra for its own cache.
This change avoids setting fn_extra prior to function invokation, and
instead information is cached in fn_extra within our provided
partitioning functions.
The native partitioning functions now also support nested functions,
i.e., the input to the function is the output of another function.
The extension now works with PostgreSQL 10, while
retaining compatibility with version 9.6.
PostgreSQL 10 has numerous internal changes to functions and
APIs, which necessitates various glue code and compatibility
wrappers to seamlessly retain backwards compatiblity with older
versions.
Test output might also differ between versions. In particular,
the psql client generates version-specific output with `\d` and
EXPLAINs might differ due to new query optimizations. The test
suite has been modified as follows to handle these issues. First,
tests now use version-independent functions to query system
catalogs instead of using `\d`. Second, changes have been made to
the test suite to be able to verify some test outputs against
version-dependent reference files.
Users might want to implement their own partitioning function
or use the legacy one included with TimescaleDB. This change
adds support for setting the partitioning function in
create_hypertable() and add_dimension().
Hash partitioning previously relied on coercing (casting) values to
strings before calculating a hash value, including creating CHECK
constraints with casts. This approach is fairly suboptimal from a
performance perspective and might have issues related to different
character encodings depending on system.
Hash partitioning now instead uses a partitioning function that takes
an anyelement type that calls type-dependent hash functions internal
to PostgreSQL. This should provide more efficient hashing both by
avoiding unnecessary string conversions and by using more optimal
type-specific hash functions.
Support for the previous hash partitioning function is preserved for
backwards compatibility. Hypertables created with the previous
function will continue to use to old hashing strategy, while new
tables will default to the updated hash partitioning.
For safety, this change also blocks changing types on hash-partitioned
columns, since it seems hard to guarantee the same hash result between
different types.
