If the ANY construct contains a singleton NULL then the logic in
"dimension_values_create_from_array" barfs causing a crash. Fix it
appropriately in the caller "hypertable_restrict_info_add_expr"
function.
PostgreSQL 11 added support for query plans that do partitionwise
aggregation on partitioned tables. Such query plans push down
aggregates to individual partitions (either fully or partially) for
similar or better performance than regular plans due to, among other
things, improved locking.
The changes in this commit adds the corresponding partitionwise
aggregation functionality for hypertables. To enable this
functionality on hypertables, we add partitioning metadata at the
planning stage to make the regular PostgreSQL planner believe it is
planning a partitioned table. Alternatively, we could have added the
corresponding planner paths in our own code, e.g., in the
create_upper_paths_hook, but this would require copying or
re-implementing a large amount of PostgreSQL planning code.
Note that partitionwise aggregation will only work with PostgreSQL 11.
As a side effect of making hypertables look like partitioned tables
during planning, some append plans will differ because the planner
removes any Result projection nodes from such plans, knowing it can
push projections down to the partitions instead. This also affects a
number of query-related tests, so these have been split into
version-specific tests.
The extension now works with PostgreSQL 10, while
retaining compatibility with version 9.6.
PostgreSQL 10 has numerous internal changes to functions and
APIs, which necessitates various glue code and compatibility
wrappers to seamlessly retain backwards compatiblity with older
versions.
Test output might also differ between versions. In particular,
the psql client generates version-specific output with `\d` and
EXPLAINs might differ due to new query optimizations. The test
suite has been modified as follows to handle these issues. First,
tests now use version-independent functions to query system
catalogs instead of using `\d`. Second, changes have been made to
the test suite to be able to verify some test outputs against
version-dependent reference files.
Hash partitioning previously relied on coercing (casting) values to
strings before calculating a hash value, including creating CHECK
constraints with casts. This approach is fairly suboptimal from a
performance perspective and might have issues related to different
character encodings depending on system.
Hash partitioning now instead uses a partitioning function that takes
an anyelement type that calls type-dependent hash functions internal
to PostgreSQL. This should provide more efficient hashing both by
avoiding unnecessary string conversions and by using more optimal
type-specific hash functions.
Support for the previous hash partitioning function is preserved for
backwards compatibility. Hypertables created with the previous
function will continue to use to old hashing strategy, while new
tables will default to the updated hash partitioning.
For safety, this change also blocks changing types on hash-partitioned
columns, since it seems hard to guarantee the same hash result between
different types.
remove all murmur3-related source code. Alter regression tests
to reflect new hash values for inputs, and a slightly different
set of input data to ensure that sufficient chunks and partitions
are tested. Some changes to .sh scripts in sql/setup that seem
to be used only to power the "unit tests", which I cannot
yet run successfully.
Common time-series rollups are of the form SELECT date_trunc(const,
time) as t, aggregates... GROUP BY t ORDER BY t DESC LIMIT const2.
Previously, Postgres would not optimize such queries because it
could not match date_trunc(const, time) to any indexes on time, since
it has no way to know that a sort on time is always a valid sort on
date_trunc(const, time).
This commit implements this optimization for date_trunc but it could
apply to a wider range of functions, to be implemented later.
Previously, each test set their own (although mostly the same)
configuration for log output and error verbosity. This is now set
globally in the test runner so that tests only need to set these
configuration parameters if they need to override the defaults. The
log verbosity is also reduced so that errors aren't generated with the
line number of the source file that output the error. Line numbers in
the output can break tests when upgrading to a new PostgreSQL version
that outputs a different line number.
These tests show that non-aggregated queries that have an ORDER BY time
DESC LIMIT x structure perform well. Postgres processes such queries by
using a time DESC index and then performs a MergeAppend. Some further
optimization to avoid touching unnecessary tables due to constraints
can be added but they are not a priority. Much worse is that these
optimizations do not work when grouping by time (either through
integer division or using date_trunc).
In order to use the non-aggregated query optimizations we had
to create an index with time as the leading field and no
WHERE clause on the index.
Since create_hypertable() allows you to optionally specify a
partitioning column, it makes sense to default to one partition when
no column is specified and asking for the number of partitions when a
column is specified and the number of partitions is not (instead of
defaulting to one).
This patch also changes the order and type of partitioning-related
input arguments to create_hypertable() so that the number of
partitions can easily be specified alongside the partitioning column
and without type casting.
- Directory structure now matches common practices
- Regression tests now run with pg_regress via the PGXS infrastructure.
- Unit tests do not integrate well with pg_regress and have to be run
separately.
- Docker functionality is separate from main Makefile. Run with
`make -f docker.mk` to build and `make -f docker.mk run` to run
the database in a container.
