When called with negative chunk_target_size_bytes
calculate_chunk_interval will throw an assertion. This patch adds
error handling for this condition. Found by sqlsmith.
PostgreSQL 12 introduced space optimizations for indexes, which caused
the adaptive chunking test to fail since its measure of chunk size
includes indexes that now report different sizes.
To fix this, the adaptive chunking test now has version-specific
output files.
Replace hardcoded database name from regression tests with :TEST_DBNAME
Remove creation of database single_2 from test runner and add it to
bgw_launcher and loader test since no other tests used those
use SQL comments in test scripts
Previously, we did not intercept the ALTER SCHEMA [name] RENAME command, which meant all Timescale catalog tables that store a schema name were not getting updated properly. This caused problems when users tried to drop a hypertable in a renamed schema, but also would have also caused problems with other commands (which we now add tests for in this PR).
The use of the incorrect index to find the hypertable that a
dimension belongs to caused adaptive chunking to break whenever
dimension_id != hypertable_id, which happened with hypertables that
have multiple dimensions where time is not the first one.
Previously our check that the index matched the type of the index
was incorrect, using attno instead of atttypid. However, this also
had a problem in that if there was another column with the same
type as the time column, e.g., two BIGINT columns, we could pick
the wrong index to use to find min and max. The check now makes
sure the type is correct AND the name is the same as the table
column name.
When configuring adaptive chunking and estimating the
chunk_target_size, we should use shared_buffers as an indication of
cache memory instead of trying to estimate based on
effective_cache_memory, system memory or other settings. This is
because shared_buffers should already have been set by the user based
on system memory, and also accurately reflects the cache memory
available to PostgreSQL.
We use a fraction (currently 0.9) of this cache memory as the
chunk_target_size. This assumes no concurrency with other hypertables.
When adaptive chunking is enabled, and no `chunk_time_interval` is
set, it is better to start with a small chunk size rather than a too
big, since it will adapt faster. This change sets the
`chunk_time_interval` to 1 day if adaptive chunking is enabled on a
hypertable. Note that this only happens if adaptive chunking is
enabled when `create_hypertable()` is called. Otherwise, the existing
`chunk_time_interval` will be used.
Adaptive chunking uses the min and max value of previous chunks
to estimate their "fill factor". Ideally, min and max should be
retreived using an index, but if no index exists we fall back
to a heap scan. A heap scan can be very expensive, so we now
raise a WARNING if no index exists.
This change also renames set_adaptive_chunk_sizing() to simply
set_adaptive_chunking().
Users can now (optionally) set a target chunk size and TimescaleDB
will try to adapt the interval length of the first open ("time")
dimension in order to reach that target chunk size. If a hypertable
has more than one open dimension, only the first one will have a
dynamically adapting interval.
Users can optionally specify their own function that calculates the
new dimension interval. They can also set a target size of 0 in order
to estimate a suitable target size for a chunk based on available
memory.
