Remove user from regression tests so that the local test will work.
The tests itself will still be run with the appropriate user because
of the logic in runner.sh.
This PR fixes the handling of drop_chunks when the hypertable's
time field is a TIMESTAMP or DATE field. Previously, such
hypertables needed drop_chunks to be given a timestamptz in UTC.
Now, drop_chunks can take a DATE or TIMESTAMP. Also, the INTERVAL
version of drop_chunks correctly handles these cases.
A consequence of this change is that drop_chunks cannot be called
on multiple tables (with table_name = NULL or schema_name = NULL)
if the tables have different time column types.
Windows 64-bit binaries should now be buildable using the cmake
build system either from the command line or from Visual Studio.
Previous issues regarding unresolved symbols have been resolved
with compatibility header files to properly export symbols or
getting GUCs via normal APIs.
Plans that have a result relation set (DELETE or UPDATE) should
not optimize subplans that contain append nodes. This is because
the PostgreSQL must turn such plans on an inheritance table into
a set of similar plans on each subtable (i.e., it needs to apply
UPDATE and DELETE to each subtable and not just the parent).
Optimizing such plans with, e.g., ConstraintAwareAppend makes
the planner believe it only needs to do the DELETE or UPDATE
on the root table, which means rows in subtables won't be affected.
TimescaleDB cache invalidation happens as a side effect of doing a
full SQL statement (INSERT/UPDATE/DELETE) on a catalog table (via
table triggers). However, triggers aren't invoked when using
PostgreSQL's internal catalog API for updates, since PostgreSQL's
catalog tables don't have triggers that require full statement
parsing, planning, and execution.
Since we are now using the regular PostgreSQL catalog update API for
some TimescaleDB catalog operations, we need to do cache invalidation
also on such operations.
This change adds cache invalidation when updating catalogs using the
internal (C) API and also makes the cache invalidation more fine
grained. For instance, caches are no longer invalidated on some
INSERTS that do not affect the validity of objects already in the
cache, such as adding a new chunk.
Append plans that are empty (no children) are replaced at planning-
time with a Result plan node that returns nothing. Such Result nodes
occur typically when querying outside the time range covered by chunks.
However, the ConstraintAwareAppend plan node did not handle the
case of a Result child node, instead expecting either an Append or
MergeAppend node. This is now fixed so that encountering a Result node
means doing nothing.
This change reduces the usage of SECURITY DEFINER on SQL
functions and fixes related permissions issues. It also
properly checks hypertable permissions relative the current_user
instead of the session_user, which otherwise breaks SET ROLE,
among other things.
The reindex test outputs the OID of a cloned index. This OID might
change with the state of the database, added tests, etc., causing
frequent test failures. The test is now updated to output the name of
the index instead of the OID.
This changes the order in which subdirectories are processed in the
top-level CMakeLists.txt file, so that dependencies from the scripts/
dir are set when processing the src/ dir. Otherwise, the `pgindent`
target won't be enabled on the first CMake run.
reindex allows you to reindex the indexes of only certain chunks,
filtering by time. This is a common use case because a user may
want to reindex chunks after they are no longer getting new data once.
reindex also has a recreate option which will not use REINDEX
but will rather CREATE INDEX a new index and then
DROP INDEX / RENAME new_index to old_name. This approach has advantages
in terms of blocking reads for a much shorter period of time. However,
it does more work and will use more disk space during the operation.
Clustering a table means reordering it according to an index.
This operation requires an exclusive lock and extensive
processing time. On large hypertables with many chunks, CLUSTER
risks blocking a lot of operations by holding locks for a long time.
This is alleviated by processing each chunk in a new transaction,
ensuring locks are only held on one chunk at a time.
Previously, for timezones w/o tz. The range_end and range_start were
defined as UTC, but the constraints on the table were written as as
the local time at the time of chunk creation. This does not work well
if timezones change over the life of the hypertable.
This change removes the dependency on local time for all timestamp
partitioning. Namely, the range_start and range_end remain as UTC
but the constraints are now always written in UTC too. Since old
constraints correctly describe the data currently in the chunks, the
update script to handle this change changes range_start and range_end
instead of the constraints.
Fixes#300.
Functions marked IMMUTABLE should also be parallel safe, but
aren't by default. This change marks all immutable functions
as parallel safe and removes the IMMUTABLE definitions on
some functions that have been wrongly labeled as IMMUTABLE.
If functions that are IMMUTABLE does not have the PARALLEL SAFE
label, then some standard PostgreSQL regression tests will fail
(this is true for PostgreSQL >= 10).
Aggregate functions that have serialize and deserialize support
functions (histogram, last, first, etc.) should have these
support functions marked STRICT.
PostgreSQL's regular test suite will fail when the timescaledb
module is loaded without these functions being marked STRICT.
To verify that PostgreSQL operations on regular tables are unaffected by
the TimescaleDB extension, the regular PostgreSQL regression tests can be
run with the TimescaleDB extension loaded.
To enable the build target (`pginstallcheck`), one needs to set the
CMake `PG_SOURCE_DIR` variable to point to the PostgreSQL source code, which
needs to have at least the regression test suite compiled.
The PostgreSQL tests suite will run on every Travis build.
All partitioning functions now has the signature `int func(anyelement)`.
This cleans up some special handling that was necessary to support
the legacy partitioning function that expected text input.
The C source files can now be formatted with pgindent using
a new CMake build target.
The location of pgindent and dependencies are auto-discovered
by CMake or can be overridden on the command line.
We now use INT64_MAX and INT64_MIN as the max and min values for
dimension_slice ranges. If a dimension_slice has a range_start of
INT64_MIN or the range_end is INT64_MAX, we remove the corresponding
check constraint on the chunk since it signifies that this end of the
range is infinite. Closed ranges now always have INT64_MIN as range_end
of first slice and range_end of INT64_MAX for the last slice.
Also, points corresponding to INT64_MAX are always
put in the same slice as INT64_MAX-1 to avoid problems with the
semantics that coordinate < range_end.
The extension now works with PostgreSQL 10, while
retaining compatibility with version 9.6.
PostgreSQL 10 has numerous internal changes to functions and
APIs, which necessitates various glue code and compatibility
wrappers to seamlessly retain backwards compatiblity with older
versions.
Test output might also differ between versions. In particular,
the psql client generates version-specific output with `\d` and
EXPLAINs might differ due to new query optimizations. The test
suite has been modified as follows to handle these issues. First,
tests now use version-independent functions to query system
catalogs instead of using `\d`. Second, changes have been made to
the test suite to be able to verify some test outputs against
version-dependent reference files.
SQL source files that are joined together were
previously listed in the `load_order.txt` and
`extra_extension_files.txt` files. These SQL file list
were then read by CMake. However, since no dependencies
existed on these list files in Make targets, the project
was not automatically rebuilt if the lists were updated.
This change moves the lists directly to the CMakeLists.txt
file, ensuring that the project is properly rebuilt every
time the lists are updated.
This fixes a bug with an explain for ConstraintAware plans
when all chunks were excluded and a MergeAppend was used.
The problem was that the EXPLAIN output for MergeAppend nodes
expected at least one child to print the sorting column info.
When all chunks were excluded there are no children. This
PR removes all child execution states from the ConstraintAware
node when all chunks are excluded.
The chunk cache needs to free chunk memory as
it evicts chunks from the cache. This was previously
done by pfree:ing the chunk memory, but this didn't
account for sub-allocated objects, like the chunk's
hypercube. This lead to some chunk objects remaining
in the cache's memory context, thus inflating memory
usage, although the objects were no longer associated
with a chunk.
This change adds a per-chunk memory context in the cache
that allows all chunk memory to be easily freed when
the cache entry is evicted or when the chunk cache
is destroyed.
Add check that time dimensions are set as NOT NULL in the
main table that a hypertable is created from. If it is not
set, the constraint will be added.
When inserting tuples into a chunk, the tuples are converted to chunk
table format if the chunk's attributes differ from the parent
table's. This can happen if, e.g., the parent table has had one of its
columns removed, leading to a garbage attribute that is not inherited
by new chunks.
Unfortunately, PostgreSQL compares the typeid of tuples to decide if
tuple conversion is needed, while also creating a new typeid for every
table (including child tables). This results in tuple conversion
occurring between a parent and a child table, even if not strictly
needed.
This optimization adds an extra check that avoids tuple conversion
when not needed. Note that this updated check is also in recent
PostgreSQL versions (>= PG10), so it won't strictly be needed when
running on those versions.
This fixes two assertion statements in the subspace store. The first
fix is for an assigment that occurred in the assertion instead of an
equality comparison. The other is just a cleanup to remove an
unnecessary if-statement.
Everytime a new chunk insert state is created, a new range table entry
is also created in the executor's range table. This change ensures
that a range table entry is reused in case one already exists for a
chunk that has been closed and reopened again in the same transaction.
This can happen if the chunk insert state is evicted from its
associated cache due to out-of-order inserts. Thus, this will limit
the growth of the range table to at most the number of chunks in the
hypertable.
Users might want to implement their own partitioning function
or use the legacy one included with TimescaleDB. This change
adds support for setting the partitioning function in
create_hypertable() and add_dimension().
A recent change blocked changing types of space-partitioned hypertable
columns. However, this blocking should not apply to regular tables,
which otherwise causes a crash. This change fixes this issue by
properly checking that the the table is a hypertable.
Hash partitioning previously relied on coercing (casting) values to
strings before calculating a hash value, including creating CHECK
constraints with casts. This approach is fairly suboptimal from a
performance perspective and might have issues related to different
character encodings depending on system.
Hash partitioning now instead uses a partitioning function that takes
an anyelement type that calls type-dependent hash functions internal
to PostgreSQL. This should provide more efficient hashing both by
avoiding unnecessary string conversions and by using more optimal
type-specific hash functions.
Support for the previous hash partitioning function is preserved for
backwards compatibility. Hypertables created with the previous
function will continue to use to old hashing strategy, while new
tables will default to the updated hash partitioning.
For safety, this change also blocks changing types on hash-partitioned
columns, since it seems hard to guarantee the same hash result between
different types.
When creating an index on a chunk based on a index on a hypertable, it
is necessary to adjust the attribute numbers of referenced columns in
case the hypertable and the chunk have different number of attributes
(this can happen, e.g., due to removing columns on the hypertable that
aren't inherited by a newly created chunk). This adjustment was
handled for regular indexes, but not expression indexes, causing an
error to be raised. This change adds the proper handling of expression
indexes.
By default, Postgres places constraint expressions on the query memory
context. However, these expressions only apply to individual chunks
and cannot be re-used across chunks. Placing them on the
chunk_insert_state context (created as part of this PR) allow these
expressions to be freed along with the insert state of individual
chunks.
