To increase schema security we do not want to mix our own internal
objects with user objects. Since chunks are created in the
_timescaledb_internal schema our internal functions should live in
a different dedicated schema. This patch make the necessary
adjustments for the following functions:
- to_unix_microseconds(timestamptz)
- to_timestamp(bigint)
- to_timestamp_without_timezone(bigint)
- to_date(bigint)
- to_interval(bigint)
- interval_to_usec(interval)
- time_to_internal(anyelement)
- subtract_integer_from_now(regclass, bigint)
The constraint test fails when running with sanitizers (ASAN and
UBSAN) due to a tablespace conflict. Make sure tablespaces are cleaned
up prior to recreating.
This resolves an issue when using both constraints with index
tablespaces AND constraints that have no indexes on a single hypertable.
chunk_constraint_add_table_constraint() was attempting to add
`USING INDEX TABLESPACE` to all constraints when a given hypertable had
any constraint configured to use a tablespace, resulting in a SYNTAX
error and blocking the creation of new chunks.
The solution here is to limit index tablespace lookups to only the
constraint types which use indexes: primary key and unique so that only
those constraits will have `USING INDEX TABLESPACE` prepended when
necessary.
Unless otherwise listed, the TODO was converted to a comment or put
into an issue tracker.
test/sql/
- triggers.sql: Made required change
tsl/test/
- CMakeLists.txt: TODO complete
- bgw_policy.sql: TODO complete
- continuous_aggs_materialize.sql: TODO complete
- compression.sql: TODO complete
- compression_algos.sql: TODO complete
tsl/src/
- compression/compression.c:
- row_compressor_decompress_row: Expected complete
- compression/dictionary.c: FIXME complete
- materialize.c: TODO complete
- reorder.c: TODO complete
- simple8b_rle.h:
- compressor_finish: Removed (obsolete)
src/
- extension.c: Removed due to age
- adts/simplehash.h: TODOs are from copied Postgres code
- adts/vec.h: TODO is non-significant
- planner.c: Removed
- process_utility.c
- process_altertable_end_subcmd: Removed (PG will handle case)
Check constraints are propagated to child chunks by default, but they're
miggins from _timescaledb_catalog.chunk_constraint table. It can lead to
a situation, when such a constraint will be added as not valid, and then
at validation time validate_hypertable_constraint will construct a
chunk_cmd with a NULL name (since it was not there and
ts_chunk_constraint_get_name_from_hypertable_constraint returns NULL),
what eventually leads to a crash.
Replace hardcoded database name from regression tests with :TEST_DBNAME
Remove creation of database single_2 from test runner and add it to
bgw_launcher and loader test since no other tests used those
use SQL comments in test scripts
Previously, we did not intercept the ALTER SCHEMA [name] RENAME command, which meant all Timescale catalog tables that store a schema name were not getting updated properly. This caused problems when users tried to drop a hypertable in a renamed schema, but also would have also caused problems with other commands (which we now add tests for in this PR).
Previously the following commands would not throw
an error on hypertables and this PR makes sure that
they do in order to avoid corrupting hypertable
internals:
* ALTER TABLE hypertable ADD COLUMN new_device_id int UNIQUE;
* ALTER TABLE hypertable DROP COLUMN partitioning_column;
* ALTER TABLE hypertable DROP COLUMN time_open_dim;
* ALTER TABLE hypertable ALTER COLUMN time_open_dim TYPE TEXT;
Fixes#536.
Constraints with the NO INHERIT option does not make sense on a
hypertable's root table since these will not be enforced.
Previously, NO INHERIT constraints were blocked on chunks, and were
thus not enforced until chunk creation time, allowing creation of NO
INHERIT constraints on empty hypertables, but then causing failure at
chunk-creation time. Instead, NO INHERIT constraints are now properly
blocked at the hypertable level.
This Fixes at least two bugs:
1) A drop of a referenced table used to drop the associated
FK constraint but not the metadata associated with the constraint.
Fixes#43.
2) A drop of a column removed any indexes associated with the column
but not the metadata associated with the index.
Renaming constraints on hypertables is now supported. Using the ONLY
option with RENAME CONSTRAINT is blocked for hypertables. Renaming
constraints on chunks is also blocked.
This commit also fixes a possible bug in chunk_constraint code.
Previously, `chunk_constraint_add_from_tuple` used GETSTRUCT on the
tuple to convert the tuple to `FormData_chunk_constraint`. This
was incorrect since some fields can be NULL. Using GETSTRUCT for
tables with NULLABLE fields is unsafe and indeed was incorrect·
in this case.
This change is part of an effort to create a consistent way
of dealing with metadata catalog updates, which is currently
a mix of C API and INSERT/UPDATE/DELETE statements from SQL
code. This mix makes catalog handling unnecessarily complex as
there are multiple ways to update metadata, increasing the risk
of security issues with publically exposed SQL functions. It also
complicates things like cache invalidation, requiring different
mechanisms for C and SQL code. Catalog updates from SQL code
require triggers on metadata tables for cache invalidation that
do not work with native catalog updates.
The creation of chunks has been particularly messy in this regard,
making the code hard to follow. Especially the handling of a chunk's
constraints, where dimensional and other constraints were handled
differently. With this change, constraint handling is now consistent
across constraint types with a single API for updating metadata.
Reduce memory usage for out-of-order inserts
The chunk_result_relation_info should be put on the chunk memory
context. This will cause the rri constraint expr to also go onto
that context and be correctly freed when the chunk insert state
is destroyed.
The extension now works with PostgreSQL 10, while
retaining compatibility with version 9.6.
PostgreSQL 10 has numerous internal changes to functions and
APIs, which necessitates various glue code and compatibility
wrappers to seamlessly retain backwards compatiblity with older
versions.
Test output might also differ between versions. In particular,
the psql client generates version-specific output with `\d` and
EXPLAINs might differ due to new query optimizations. The test
suite has been modified as follows to handle these issues. First,
tests now use version-independent functions to query system
catalogs instead of using `\d`. Second, changes have been made to
the test suite to be able to verify some test outputs against
version-dependent reference files.
This commit moves a lot of test setup logic to runner.sh. Also
passes the right commands to the regression infrastructure to create
appropriate users and run tests as a regular user.
The ProcessUtility hook doesn't give any information on applied DDL
commands, which makes it hard to implement DDL processing that
requires the result of a DDL command on a hypertable (for instance,
adding a constraint or index without an explicit name).
This change splits the DDL processing over start and end hooks,
handling DDL commands before and after regular PostgreSQL processing,
respectively.
The start DDL hook is still based on the ProcessUtility hook, while
the end DDL hook is based on an event trigger that allows getting
information on the created/dropped/altered objects.
The current approach of handling alter table commands does not allow
getting the result of the command; for instance, the object address of
a created constraint on a hypertable. Thus there is no way to get the
auto-generated name of a constraint, which is needed when the
corresponding constraints are created on the hypertable's chunks.
Therefore, this change blocks the ability to create constraints
without an explicit name. When a better approach to handling alter
table is deviced, it is possible to remove this restriction.
Adding a constraint using an existing index changes the existing
index's name as a side effect (PostgreSQL wants the index name to
match the constraint name). This creates a mismatch between the
index's name and our metadata in the chunk_index table. Further, since
the name change happens as a result of an internal command (and not
via DDL command that we can capture) there is currently no way to sync
up this information.
This PR add support for primary-key, foreign-key, unique, and exclusion constraints.
Previously supported are CHECK and NOT NULL constraints. Now, foreign key
constraints where a hypertable references a plain table is support
(while vice versa, with a plain table references a hypertable, is still not).
