Previously, date was auto-cast to timestamptz when time_bucket was
called. This led to weird behavior with regards to timezones and
the return value was a timestamptz. This PR makes time_bucket return
a DATE on DATE input and avoids all timezone conversions.
Previously dimension_slice would only allow ranges to start and
end with values >= 0. However, this did not allow for timestamps
prior to 1970 because those times would be represented by negative
integers. With this PR we now support dates prior to 1970.
Addresses issue #153.
Previously the default chunk time in microseconds was too large
for a SMALLINT or INTEGER field. Now, we only assign a default
value if the type is TIMESTAMP or TIMESTAMPTZ. Integer timestamps,
such as SMALLINT, INTEGER, and BIGINT, need to be explicitly set
since only the user knows what units the numbers represent.
Further, we check to make sure the chunk time interval is not too
large for SMALLINT and INTEGER so as to avoid confusing problems
later when the user goes to insert.
With this change, hypertables no longer rely on an INSERT trigger to
dispatch tuples to chunks. While an INSERT trigger worked well for
both INSERTs and COPYs, it caused issues with supporting some regular
triggers on hypertables, and didn't support RETURNING statements and
upserts (ON CONFLICT DO UPDATE).
INSERTs are now handled by modifying the plan for INSERT statements. A
custom plan node is inserted as a subplan to a ModifyTable plan node,
taking care of dispatching tuples to chunks by setting the result
table for every tuple scanned.
COPYs are handled by modifying the regular copy code. Unfortunately,
this required copying a significant amount of regular PostgreSQL
source code since there are no hooks to add modifications. However,
since the modifications are small it should be fairly easy to keep the
code in sync with upstream changes.
To allow upgrading versions older than the most recent one,
update scripts from older versions need to be added to the
extension. This is now done by including all files in the
file extra_extension_files.txt to the extension. Currently
that file needs to be manually updated when a new release
is built.
Also includes some update infrastructure changes. Main difference
is that sql/updates/ scripts are now split into pre- and post- files.
This is needed because table updates need to happen before reloading
functions (some functions checked for accessing right schema at
definition time). Meanwhile trigger updates need to happen after
since triggers must refer to valid functions at definition time.
This adds support for all types of triggers on a hypertable except
INSERT AFTER. UPDATE and DELETE ROW triggers are automatically copied from
a hypertable onto the chunks. Therefore, any trigger defined on the
parent hypertable will apply to any row in any of the chunks as well.
STATEMENT level triggers and iNSERT triggers need not be copied in this
way.
SQL code is now split into setup, functions, and init files to
allow a subset to be run when the extension is updated. During
build, an update script is now also generated.
It is not an error for range_end of a new chunk to be equal to
range_start of an existing chunk because range_end is exclusive.
Fix logic not to throw an error in that case.
A new public-facing API `add_dimension(table, column, ...)`
makes it possible to add additional dimensions (partitioning
columns) to a hypertable.
Currently, new dimension can only be added to empty tables.
The code has also been refactored with a corresponding
internal function that is called by both `add_dimension()`
and `create_hypertable()`.
Previously, catalog tables were not fully protected from malicious
non-superusers. This PR fixes permission handling be severely
restricting permissions to the catalog and instead using SECURITY
DEFINER functions to alter the catalog when needed without giving
users permission to do those same operations outside of these functions.
In addition, these functions check for proper permissions themselves
so are safe to use.
This PR also makes sure that chunk tables have the same owner as the
hypertable and correctly handles `ALTER TABLE...OWNER TO` commands to
keep this info in sync.
Previously, the extension could end up in a bad state if it
was dropped as part of a cascade. This PR fixes that by
checking explicitly for the presence of the proxy table
to make sure we are not in the middle of an extension
drop. Fixes#73.
Tablespaces can now be associated with a hypertable
using the new user-facing API attach_tablespace().
Alternatively, if the main table, which is to be
converted into a hypertable, was created with an
associated tablespace, that tablespace will
automatically be associated also with the hypertable.
Whenever a chunk is created, a tablespace will be
chosen from the ones associated with the chunk's
hypertable (if any). This is done in a way that ensures
consistency in one dimension. I.e., if a hypertable
has a closed (space) dimension with a fixed number
of slices (ranges), it will ensure that chunks that
fall in the same slice will alsp be stored in the same
tablespace.
If a hypertable has more than one closed dimension,
the first one will be used to assign tablespaces
to chunks. If the table has no closed dimensions, but
one or more open (time) dimensions, then the first
time dimension will be used. However, since open
dimensions do not have a fixed number of slices,
tablespaces will be assigned in a round-robbin
fashion as new slices are created. Still, chunks
in the same time slice will be stored in the same
tablespace.
Also in this commit:
- Rename time/space to open/closed for more generality.
- Create a Point data type for mapping a tuple to an
N-dimensional space.
- Numerous fixes and cleanups.
This is the first stab at updating the table and data type
definitions in the catalog module in the C code. This also
adds functions for natively scanning the dimension and
dimension_slice tables.
The current schema and code support only one "time" and one "space"
dimension for partitioning. While this is probably good enough for
many applications, it won't allow partitioning along two "space"
dimensions, like longitude and latitude, for instance.
This commit is a first stab at refactoring the metadata schema and
internal functionality to support any number of partitioning
dimensions. The idea is to think of a hypertable as partitioned in N
dimensions, where a partition (chunk) is a hypercube in the
N-dimensional hyperspace. Each dimension is divided into a number of
"slices" (dimensional partitions) that each occupies a range along the
dimension's keyspace axis.
A dimension can either be "closed" or "open", indicating a space-like
or time-like dimension, respectively. Closed dimensions have a limited
number of partitions that cover the entire domain. Open dimensions have
an unlimited number of partitions and thus must be created on-demand
as needed for the inserted data.
Note that the open/closed notation is preferred over time/space, since
an open dimension could be used for something other than time (for
instance, a sequence number). Conversely, closed dimensions need not be
space-like as it is entirely possible to use time as a closed
dimension.
Another advantage of this refactoring is that it now unifies a lot
of the logic for time and space that used to be separate.
On a schema-level, this gets rid of partition/partitition_epoch tables and
replace them with dimension, dimension_slice, and chunk_constraint. It
also removes the time columns associated with a chunk and instead makes
this a separate dimension stored as a chunk_constraint.
Previously, chunks could end up unaligned betweeen partitions if the chunk_time_interval
was updated between chunk creations. Now there is a check when a chunk is created whether there is
a chunk matching the time point in another partition and if there is, the new chunk is created
with the same interval.
Clean up the table schema to get rid of legacy tables and functionality
that makes it more difficult to provide an upgrade path.
Notable changes:
* Get rid of legacy tables and code
* Simplify directory structure for SQL code
* Simplify table hierarchy: remove root table and make chunk tables
* inherit directly from main table
* Change chunk table suffix from _data to _chunk
* Simplify schema usage: _timescaledb_internal for internal functions.
* _timescaledb_catalog for metadata tables.
* Remove postgres_fdw dependency
* Improve code comments in sql code
Previously create_hypertable() would throw an error when called on
an already existing hypertable. This can now be skipped by setting
if_not_exists argument to true.
This PR removes the need to run setup_timescaledb. It also fixes
pg_dump and pg_restore. Previously, the database would restore in
a broken state because trigger functions were never attached to
meta tables (since setup_timescaledb() was not run). However, attaching
those triggers at extension creation also causes problems since the data
copy happens after extension creation but we don't want triggers fired
on the data restored (that could cause duplicate rows, for example).
The solution to this chicken-and-egg problem in this PR is to have
a special configuration (GUC) variable `timescaledb.restoring` that,
if 'on', would prevent the extension from attaching triggers at
extension creation. Then, after restoration, you'd call
restore_timescaledb() to attach the triggers and unset the GUC above.
This procedure is documented in the README as part of this PR.
Changed create_hypertable to add time DESC and partitioning_column, time DESC
indexes by default (the latter only if partitioning column not null).
Indexes only created if there is no index on time and
partitioning_column, time respectively. Index creation can be turned off
with the create_default_indexes parameter to create_hypertable (default true).
Remove unit tests because most of our testing is in regression tests
anyway and regression tests are better integrated with the Postgres
extension workflow. sql/setup scripts were old and not used except
for unit testing.
