Calling `create_hypertable` with a space dimension silently succeeds
without actually creating the space dimension if `num_partitions` is
not specified.
This change ensures that we raise an appropriate error when a user
fails to specify the number of partitions.
Add bool created to return value of create_hypertable and add_dimension.
When if_not_exists is true and creation is skipped because the object
already exists created will be false, otherwise it will be true. This
modifies the functions to return meta data even when no object was
created.
Change create_hypertable to return a record consisting of
(hypertable_id, schema_name, table_name). This improves user feedback
about success of the operation but also gives the function an API
returning useful information for non-human consumption.
This is a continuation of prior efforts to refactor API functions in C
to:
- improve usage of proper error codes
- use error messages that better conform with the PostgreSQL standard.
- improve security by avoiding that lots of code run under SECURITY DEFINER
- move towards doing all metadata updates using a consistent catalog API
Most importantly, `create_hypertable()` has been refactored in C,
which simplifies a lot of code that previously required
upcalls/downcalls between C code and plpgsql code, or duplicated
functionality between the two environments.
The functions for adding and updating dimensions have been refactored
in C to:
- improve usage of proper error codes
- make messages that better conform with the PostgreSQL standard.
- improve security by avoiding that lots of code run under SECURITY DEFINER
A new if_not_exists option has also been added to add_dimension() and
a the number of partitions can now be set using the new
set_number_partitions() function.
A bug in the validation of smallint time intervals has been fixed. The
previous code didn't check for intervals > 0 and smallint intervals
accepted values up to UINT16_MAX instead of INT16_MAX.
This commit moves a lot of test setup logic to runner.sh. Also
passes the right commands to the regression infrastructure to create
appropriate users and run tests as a regular user.
All regression tests will now use a non-superuser unless superuser is
necessary. This PR is a meant to prevent things like issue #226.
This PR also fixes some more permission bugs found during this testing.
With standard PostgreSQL append plans, planning-time constraint
exclusion does not work if a query expression contains restrictions
that are non-constants (e.g., mutable functions like now()). This
leads to full scans of all chunks (child tables), significantly
increasing query time.
To fix this, we wrap append plans (Append and MergeAppend) in a
ConstraintAwareAppend plan, which excludes child relations at
execution time by pruning the child relations from the regular append
plan. This happens after first constifying the original restriction
clause every time the plan is executed.
This optimization only happens if there are non-constant restrictions
in the original query and the entire optimization can be disbled with
a GUC variable.
Previously the default chunk time in microseconds was too large
for a SMALLINT or INTEGER field. Now, we only assign a default
value if the type is TIMESTAMP or TIMESTAMPTZ. Integer timestamps,
such as SMALLINT, INTEGER, and BIGINT, need to be explicitly set
since only the user knows what units the numbers represent.
Further, we check to make sure the chunk time interval is not too
large for SMALLINT and INTEGER so as to avoid confusing problems
later when the user goes to insert.
A new public-facing API `add_dimension(table, column, ...)`
makes it possible to add additional dimensions (partitioning
columns) to a hypertable.
Currently, new dimension can only be added to empty tables.
The code has also been refactored with a corresponding
internal function that is called by both `add_dimension()`
and `create_hypertable()`.
Previously, catalog tables were not fully protected from malicious
non-superusers. This PR fixes permission handling be severely
restricting permissions to the catalog and instead using SECURITY
DEFINER functions to alter the catalog when needed without giving
users permission to do those same operations outside of these functions.
In addition, these functions check for proper permissions themselves
so are safe to use.
This PR also makes sure that chunk tables have the same owner as the
hypertable and correctly handles `ALTER TABLE...OWNER TO` commands to
keep this info in sync.
Clean up the table schema to get rid of legacy tables and functionality
that makes it more difficult to provide an upgrade path.
Notable changes:
* Get rid of legacy tables and code
* Simplify directory structure for SQL code
* Simplify table hierarchy: remove root table and make chunk tables
* inherit directly from main table
* Change chunk table suffix from _data to _chunk
* Simplify schema usage: _timescaledb_internal for internal functions.
* _timescaledb_catalog for metadata tables.
* Remove postgres_fdw dependency
* Improve code comments in sql code
This PR removes the need to run setup_timescaledb. It also fixes
pg_dump and pg_restore. Previously, the database would restore in
a broken state because trigger functions were never attached to
meta tables (since setup_timescaledb() was not run). However, attaching
those triggers at extension creation also causes problems since the data
copy happens after extension creation but we don't want triggers fired
on the data restored (that could cause duplicate rows, for example).
The solution to this chicken-and-egg problem in this PR is to have
a special configuration (GUC) variable `timescaledb.restoring` that,
if 'on', would prevent the extension from attaching triggers at
extension creation. Then, after restoration, you'd call
restore_timescaledb() to attach the triggers and unset the GUC above.
This procedure is documented in the README as part of this PR.
The dblink extension is blacklisted by some cloud-hosting providers and
is an unnecessary dependency for single-node operation. Since we don't plan
to use dblink to implement clustering this PR removes the dependency.
Previously, each test set their own (although mostly the same)
configuration for log output and error verbosity. This is now set
globally in the test runner so that tests only need to set these
configuration parameters if they need to override the defaults. The
log verbosity is also reduced so that errors aren't generated with the
line number of the source file that output the error. Line numbers in
the output can break tests when upgrading to a new PostgreSQL version
that outputs a different line number.
If a user attempts to setup a database while not connecting using
the network, port is NULL and thus fails constraint checks. Instead,
we now use the default Postgres port of 5432 when this happens.
Also, setup_db() is renamed to setup_timescaledb() for clarity in
the presence of other extensions.
Since create_hypertable() allows you to optionally specify a
partitioning column, it makes sense to default to one partition when
no column is specified and asking for the number of partitions when a
column is specified and the number of partitions is not (instead of
defaulting to one).
This patch also changes the order and type of partitioning-related
input arguments to create_hypertable() so that the number of
partitions can easily be specified alongside the partitioning column
and without type casting.
Setting up single node is now:
```
CREATE EXTENSION IF NOT EXISTS iobeamdb CASCADE;
select setup_single_node();
```
To setup a cluster do (on meta node):
```
CREATE EXTENSION IF NOT EXISTS iobeamdb CASCADE;
select set_meta();
```
on data node:
```
CREATE EXTENSION IF NOT EXISTS iobeamdb CASCADE;
select join_cluster('metadb', 'metahost');
```
This assumes that the commands are issued by the same user on both the
meta node and the data node. Otherwise the data node also has to
specify the user name to use when connecting to the meta node.
- Directory structure now matches common practices
- Regression tests now run with pg_regress via the PGXS infrastructure.
- Unit tests do not integrate well with pg_regress and have to be run
separately.
- Docker functionality is separate from main Makefile. Run with
`make -f docker.mk` to build and `make -f docker.mk run` to run
the database in a container.
