This release contains performance improvements and bug fixes since
the 2.18.1 release. We recommend that you upgrade at the next
available opportunity.
**Bugfixes**
* #7686 Potential wrong aggregation result when using vectorized aggregation with hash grouping in reverse order
* #7694 Fix ExplainHook breaking call chain
* #7695 Block dropping internal compressed chunks with `drop_chunk()`
* #7711 License error when using hypercore handler
* #7712 Respect other extensions' ExecutorStart hooks
**Thanks**
* @davidmehren and @jflambert for reporting an issue with extension hooks
* @jflambert for reporting a bug with license errors shown in autovacuum
This release introduces the ability to add secondary indexes to the columnstore, improves group by and filtering performance through columnstore vectorization, and contains the highly upvoted community request of transition table support. We recommend that you upgrade at the next available opportunity.
**Highlighted features in TimescaleDB v2.18.0**
* The ability to add secondary indexes to the columnstore through the new hypercore table access method.
* Significant performance improvements through vectorization (`SIMD`) for aggregations using a group by with one column and/or using a filter clause when querying the columnstore.
* Hypertables support triggers for transition tables, which is one of the most upvoted community feature requests.
* Updated methods to manage Timescale's hybrid row-columnar store (hypercore) that highlight the usage of the columnstore which includes both an optimized columnar format as well as compression.
**Dropping support for Bitnami images**
After the recent change in Bitnami’s [LTS support policy](https://github.com/bitnami/containers/issues/75671), we are no longer building Bitnami images for TimescaleDB. We recommend using the [official TimescaleDB Docker image](https://hub.docker.com/r/timescale/timescaledb-ha)
**Deprecation Notice**
We are deprecating the following parameters, functions, procedures and views. They will be removed with the next major release of TimescaleDB. Please find the replacements in the table below:
| Deprecated | Replacement | Type |
| --- | --- | --- |
| decompress_chunk | convert_to_rowstore | Procedure |
| compress_chunk | convert_to_columnstore | Procedure |
| add_compression_policy | add_columnstore_policy | Function |
| remove_compression_policy | remove_columnstore_policy | Function |
| hypertable_compression_stats | hypertable_columnstore_stats | Function |
| chunk_compression_stats | chunk_columnstore_stats | Function |
| hypertable_compression_settings | hypertable_columnstore_settings | View |
| chunk_compression_settings | chunk_columnstore_settings | View |
| compression_settings | columnstore_settings | View |
| timescaledb.compress | timescaledb.enable_columnstore | Parameter |
| timescaledb.compress_segmentby | timescaledb.segmentby | Parameter |
| timescaledb.compress_orderby | timescaledb.orderby | Parameter |
**Features**
* #7341: Vectorized aggregation with grouping by one fixed-size by-value compressed column (such as arithmetic types).
* #7104: Hypercore table access method.
* #6901: Add hypertable support for transition tables.
* #7482: Optimize recompression of partially compressed chunks.
* #7458: Support vectorized aggregation with aggregate `filter` clauses that are also vectorizable.
* #7433: Add support for merging chunks.
* #7271: Push down `order by` in real-time continuous aggregate queries.
* #7455: Support `drop not null` on compressed hypertables.
* #7295: Support `alter table set access method` on hypertable.
* #7411: Change parameter name to enable hypercore table access method.
* #7436: Add index creation on `order by` columns.
* #7443: Add hypercore function and view aliases.
* #7521: Add optional `force` argument to `refresh_continuous_aggregate`.
* #7528: Transform sorting on `time_bucket` to sorting on time for compressed chunks in some cases.
* #7565: Add hint when hypertable creation fails.
* #7390: Disable custom `hashagg` planner code.
* #7587: Add `include_tiered_data` parameter to `add_continuous_aggregate_policy` API.
* #7486: Prevent building against PostgreSQL versions with broken ABI.
* #7412: Add [GUC](https://www.postgresql.org/docs/current/acronyms.html#:~:text=GUC) for the `hypercore_use_access_method` default.
* #7413: Add GUC for segmentwise recompression.
**Bugfixes**
* #7378: Remove obsolete job referencing `policy_job_error_retention`.
* #7409: Update `bgw_job` table when altering procedure.
* #7410: Fix the `aggregated compressed column not found` error on aggregation query.
* #7426: Fix `datetime` parsing error in chunk constraint creation.
* #7432: Verify that the heap tuple is valid before using.
* #7434: Fix the segfault when internally setting the replica identity for a given chunk.
* #7488: Emit error for transition table trigger on chunks.
* #7514: Fix the error: `invalid child of chunk append`.
* #7517: Fix the performance regression on the `cagg_migrate` procedure.
* #7527: Restart scheduler on error.
* #7557: Fix null handling for in-memory tuple filtering.
* #7566: Improve transaction check in CAGG refresh.
* #7584: Fix NaN-handling for vectorized aggregation.
* #7598: Match the Postgres NaN comparison behavior in WHERE clause over compressed tables.
**Thanks**
* @bharrisau for reporting the segfault when creating chunks.
* @jakehedlund for reporting the incompatible NaN behavior in WHERE clause over compressed tables.
* @k-rus for suggesting that we add a hint when hypertable creation fails.
* @staticlibs for sending the pull request that improves the transaction check in CAGG refresh.
* @uasiddiqi for reporting the `aggregated compressed column not found` error.
This release contains performance improvements and bug fixes since
the 2.17.0 release. We recommend that you upgrade at the next
available opportunity.
**Features**
* #7360 Add chunk skipping GUC
**Bugfixes**
* #7335 Change log level used in compression
* #7342 Fix collation for in-memory tuple filtering
**Thanks**
* @gmilamjr for reporting an issue with the log level of compression messages
* @hackbnw for reporting an issue with collation during tuple filtering
This release adds support for PostgreSQL 17, significantly improves the
performance of continuous aggregate refreshes,
and contains performance improvements for analytical queries and delete
operations over compressed hypertables.
We recommend that you upgrade at the next available opportunity.
**Highlighted features in TimescaleDB v2.17.0**
* Full PostgreSQL 17 support for all existing features. TimescaleDB
v2.17 is available for PostgreSQL 14, 15, 16, and 17.
* Significant performance improvements for continuous aggregate
policies: continuous aggregate refresh is now using
`merge` instead of deleting old materialized data and re-inserting.
This update can decrease dramatically the amount of data that must be
written on the continuous aggregate in the
presence of a small number of changes, reduce the `i/o` cost of
refreshing a continuous aggregate, and generate fewer
Write-Ahead Logs (`WAL`).
Overall, continuous aggregate policies will be more lightweight, use
less system resources, and complete faster.
* Increased performance for real-time analytical queries over compressed
hypertables:
we are excited to introduce additional Single Instruction, Multiple Data
(`SIMD`) vectorization optimization to our
engine by supporting vectorized execution for queries that group by
using the `segment_by` column(s) and
aggregate using the basic aggregate functions (`sum`, `count`, `avg`,
`min`, `max`).
Stay tuned for more to come in follow-up releases! Support for grouping
on additional columns, filtered aggregation,
vectorized expressions, and `time_bucket` is coming soon.
* Improved performance of deletes on compressed hypertables when a large
amount of data is affected.
This improvement speeds up operations that delete whole segments by
skipping the decompression step.
It is enabled for all deletes that filter by the `segment_by` column(s).
**PostgreSQL 14 deprecation announcement**
We will continue supporting PostgreSQL 14 until April 2025. Closer to
that time, we will announce the specific
version of TimescaleDB in which PostgreSQL 14 support will not be
included going forward.
**Features**
* #6882: Allow delete of full segments on compressed chunks without
decompression.
* #7033: Use `merge` statement on continuous aggregates refresh.
* #7126: Add functions to show the compression information.
* #7147: Vectorize partial aggregation for `sum(int4)` with grouping on
`segment by` columns.
* #7204: Track additional extensions in telemetry.
* #7207: Refactor the `decompress_batches_scan` functions for easier
maintenance.
* #7209: Add a function to drop the `osm` chunk.
* #7275: Add support for the `returning` clause for `merge`.
* #7200: Vectorize common aggregate functions like `min`, `max`, `sum`,
`avg`, `stddev`, `variance` for compressed columns
of arithmetic types, when there is grouping on `segment by` columns or
no grouping.
**Bug fixes**
* #7187: Fix the string literal length for the `compressed_data_info`
function.
* #7191: Fix creating default indexes on chunks when migrating the data.
* #7195: Fix the `segment by` and `order by` checks when dropping a
column from a compressed hypertable.
* #7201: Use the generic extension description when building `apt` and
`rpm` loader packages.
* #7227: Add an index to the `compression_chunk_size` catalog table.
* #7229: Fix the foreign key constraints where the index and the
constraint column order are different.
* #7230: Do not propagate the foreign key constraints to the `osm`
chunk.
* #7234: Release the cache after accessing the cache entry.
* #7258: Force English in the `pg_config` command executed by `cmake` to
avoid the unexpected building errors.
* #7270: Fix the memory leak in compressed DML batch filtering.
* #7286: Fix the index column check while searching for the index.
* #7290: Add check for null offset for continuous aggregates built on
top of continuous aggregates.
* #7301: Make foreign key behavior for hypertables consistent.
* #7318: Fix chunk skipping range filtering.
* #7320: Set the license specific extension comment in the install
script.
**Thanks**
* @MiguelTubio for reporting and fixing the Windows build error.
* @posuch for reporting the misleading extension description in the generic loader packages.
* @snyrkill for discovering and reporting the issue with continuous
aggregates built on top of continuous aggregates.
---------
Signed-off-by: Pallavi Sontakke <pallavi@timescale.com>
Signed-off-by: Yannis Roussos <iroussos@gmail.com>
Signed-off-by: Sven Klemm <31455525+svenklemm@users.noreply.github.com>
Co-authored-by: Yannis Roussos <iroussos@gmail.com>
Co-authored-by: atovpeko <114177030+atovpeko@users.noreply.github.com>
Co-authored-by: Sven Klemm <31455525+svenklemm@users.noreply.github.com>
This release contains bug fixes since the 2.16.0 release. We recommend
that you upgrade at the next available opportunity.
**Bugfixes**
* #7182 Fix untier_chunk for hypertables with foreign keys
This release contains significant performance improvements when working with compressed data, extended join
support in continuous aggregates, and the ability to define foreign keys from regular tables towards hypertables.
We recommend that you upgrade at the next available opportunity.
In TimescaleDB v2.16.0 we:
* Introduce multiple performance focused optimizations for data manipulation operations (DML) over compressed chunks.
Improved upsert performance by more than 100x in some cases and more than 1000x in some update/delete scenarios.
* Add the ability to define chunk skipping indexes on non-partitioning columns of compressed hypertables
TimescaleDB v2.16.0 extends chunk exclusion to use those skipping (sparse) indexes when queries filter on the relevant columns,
and prune chunks that do not include any relevant data for calculating the query response.
* Offer new options for use cases that require foreign keys defined.
You can now add foreign keys from regular tables towards hypertables. We have also removed
some really annoying locks in the reverse direction that blocked access to referenced tables
while compression was running.
* Extend Continuous Aggregates to support more types of analytical queries.
More types of joins are supported, additional equality operators on join clauses, and
support for joins between multiple regular tables.
**Highlighted features in this release**
* Improved query performance through chunk exclusion on compressed hypertables.
You can now define chunk skipping indexes on compressed chunks for any column with one of the following
integer data types: `smallint`, `int`, `bigint`, `serial`, `bigserial`, `date`, `timestamp`, `timestamptz`.
After you call `enable_chunk_skipping` on a column, TimescaleDB tracks the min and max values for
that column. TimescaleDB uses that information to exclude chunks for queries that filter on that
column, and would not find any data in those chunks.
* Improved upsert performance on compressed hypertables.
By using index scans to verify constraints during inserts on compressed chunks, TimescaleDB speeds
up some ON CONFLICT clauses by more than 100x.
* Improved performance of updates, deletes, and inserts on compressed hypertables.
By filtering data while accessing the compressed data and before decompressing, TimescaleDB has
improved performance for updates and deletes on all types of compressed chunks, as well as inserts
into compressed chunks with unique constraints.
By signaling constraint violations without decompressing, or decompressing only when matching
records are found in the case of updates, deletes and upserts, TimescaleDB v2.16.0 speeds
up those operations more than 1000x in some update/delete scenarios, and 10x for upserts.
* You can add foreign keys from regular tables to hypertables, with support for all types of cascading options.
This is useful for hypertables that partition using sequential IDs, and need to reference those IDs from other tables.
* Lower locking requirements during compression for hypertables with foreign keys
Advanced foreign key handling removes the need for locking referenced tables when new chunks are compressed.
DML is no longer blocked on referenced tables while compression runs on a hypertable.
* Improved support for queries on Continuous Aggregates
`INNER/LEFT` and `LATERAL` joins are now supported. Plus, you can now join with multiple regular tables,
and you can have more than one equality operator on join clauses.
**PostgreSQL 13 support removal announcement**
Following the deprecation announcement for PostgreSQL 13 in TimescaleDB v2.13,
PostgreSQL 13 is no longer supported in TimescaleDB v2.16.
The Currently supported PostgreSQL major versions are 14, 15 and 16.
This release contains bug fixes since the 2.15.2 release.
Best practice is to upgrade at the next available opportunity.
**Migrating from self-hosted TimescaleDB v2.14.x and earlier**
After you run `ALTER EXTENSION`, you must run [this SQL
script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql).
For more details, see the following pull request
[#6797](https://github.com/timescale/timescaledb/pull/6797).
If you are migrating from TimescaleDB v2.15.0, v2.15.1 or v2.15.2, no
changes are required.
**Bugfixes**
* #7061: Fix the handling of multiple unique indexes in a compressed
INSERT.
* #7080: Fix the `corresponding equivalence member not found` error.
* #7088: Fix the leaks in the DML functions.
* #7035: Fix the error when acquiring a tuple lock on the OSM chunks on
the replica.
**Thanks**
* @Kazmirchuk for reporting the issue about leaks with the functions in
DML.
This release contains performance improvements and bug fixes since
the 2.15.0 release. Best practice is to upgrade at the next
available opportunity.
**Migrating from self-hosted TimescaleDB v2.14.x and earlier**
After you run `ALTER EXTENSION`, you must run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql). For more details, see the following pull request [#6797](https://github.com/timescale/timescaledb/pull/6797).
If you are migrating from TimescaleDB v2.15.0 or v2.15.1, no changes are required.
**Bugfixes**
* #6975: Fix sort pushdown for partially compressed chunks.
* #6976: Fix removal of metadata function and update script.
* #6978: Fix segfault in compress_chunk with primary space partition.
* #6993: Disallow hash partitioning on primary column.
**Thanks**
* @gugu for reporting the issue with catalog corruption due to update.
* @srieding for reporting an issue with partially compressed chunks and ordering on joined columns.
This release contains bug fixes since the 2.15.0 release.
We recommend that you upgrade at the next available opportunity.
**Bugfixes**
* #6540 Segmentation fault when backfilling data with COPY into a compressed chunk
* #6858 Before update trigger not working correctly
* #6908 Fix gapfill with timezone behaviour around dst switches
* #6911 Fix dropped chunk metadata removal in update script
* #6940 Fix `pg_upgrade` failure by removing `regprocedure` from catalog table
* #6957 Fix segfault in UNION queries with ordering on compressed chunks
**Thanks**
* @DiAifU, @kiddhombre and @intermittentnrg for reporting issues with gapfill and daylight saving time
* @edgarzamora for reporting issue with update triggers
* @hongquan for reporting an issue with the update script
* @iliastsa and @SystemParadox for reporting an issue with COPY into a compressed chunk
This release contains performance improvements and bug fixes since
the 2.14.2 release. We recommend that you upgrade at the next
available opportunity.
In addition, it includes these noteworthy features:
* Support `time_bucket` with `origin` and/or `offset` on Continuous Aggregate
* Compression improvements:
- Improve expression pushdown
- Add minmax sparse indexes when compressing columns with btree indexes
- Make compression use the defaults functions
- Vectorize filters in WHERE clause that contain text equality operators and LIKE expressions
**Deprecation warning**
* Starting on this release will not be possible to create Continuous Aggregate using `time_bucket_ng` anymore and it will be completely removed on the upcoming releases.
* Recommend users to [migrate their old Continuous Aggregate format to the new one](https://docs.timescale.com/use-timescale/latest/continuous-aggregates/migrate/) because it support will be completely removed in next releases prevent them to migrate.
* This is the last release supporting PostgreSQL 13.
**For on-premise users and this release only**, you will need to run [this SQL script](https://github.com/timescale/timescaledb-extras/blob/master/utils/2.15.X-fix_hypertable_foreign_keys.sql) after running `ALTER EXTENSION`. More details can be found in the pull request [#6786](https://github.com/timescale/timescaledb/pull/6797).
**Features**
* #6382 Support for time_bucket with origin and offset in CAggs
* #6696 Improve defaults for compression segment_by and order_by
* #6705 Add sparse minmax indexes for compressed columns that have uncompressed btree indexes
* #6754 Allow DROP CONSTRAINT on compressed hypertables
* #6767 Add metadata table `_timestaledb_internal.bgw_job_stat_history` for tracking job execution history
* #6798 Prevent usage of deprecated time_bucket_ng in CAgg definition
* #6810 Add telemetry for access methods
* #6811 Remove no longer relevant timescaledb.allow_install_without_preload GUC
* #6837 Add migration path for CAggs using time_bucket_ng
* #6865 Update the watermark when truncating a CAgg
**Bugfixes**
* #6617 Fix error in show_chunks
* #6621 Remove metadata when dropping chunks
* #6677 Fix snapshot usage in CAgg invalidation scanner
* #6698 Define meaning of 0 retries for jobs as no retries
* #6717 Fix handling of compressed tables with primary or unique index in COPY path
* #6726 Fix constify cagg_watermark using window function when querying a CAgg
* #6729 Fix NULL start value handling in CAgg refresh
* #6732 Fix CAgg migration with custom timezone / date format settings
* #6752 Remove custom autovacuum setting from compressed chunks
* #6770 Fix plantime chunk exclusion for OSM chunk
* #6789 Fix deletes with subqueries and compression
* #6796 Fix a crash involving a view on a hypertable
* #6797 Fix foreign key constraint handling on compressed hypertables
* #6816 Fix handling of chunks with no contraints
* #6820 Fix a crash when the ts_hypertable_insert_blocker was called directly
* #6849 Use non-orderby compressed metadata in compressed DML
* #6867 Clean up compression settings when deleting compressed cagg
* #6869 Fix compressed DML with constraints of form value OP column
* #6870 Fix bool expression pushdown for queries on compressed chunks
**Thanks**
* @brasic for reporting a crash when the ts_hypertable_insert_blocker was called directly
* @bvanelli for reporting an issue with the jobs retry count
* @djzurawsk For reporting error when dropping chunks
* @Dzuzepppe for reporting an issue with DELETEs using subquery on compressed chunk working incorrectly.
* @hongquan For reporting a 'timestamp out of range' error during CAgg migrations
* @kevcenteno for reporting an issue with the show_chunks API showing incorrect output when 'created_before/created_after' was used with time-partitioned columns.
* @mahipv For starting working on the job history PR
* @rovo89 For reporting constify cagg_watermark not working using window function when querying a CAgg
This release contains bug fixes since the 2.14.1 release.
We recommend that you upgrade at the next available opportunity.
**Bugfixes**
* #6655 Fix segfault in cagg_validate_query
* #6660 Fix refresh on empty CAgg with variable bucket
* #6670 Don't try to compress osm chunks
**Thanks**
* @kav23alex for reporting a segfault in cagg_validate_query
This release contains bug fixes since the 2.14.0 release.
We recommend that you upgrade at the next available opportunity.
**Features**
* #6630 Add views for per chunk compression settings
**Bugfixes**
* #6636 Fixes extension update of compressed hypertables with dropped columns
* #6637 Reset sequence numbers on non-rollup compression
* #6639 Disable default indexscan for compression
* #6651 Fix DecompressChunk path generation with per chunk settings
**Thanks**
* @anajavi for reporting an issue with extension update of compressed hypertables
This release contains performance improvements and bug fixes since
the 2.13.1 release. We recommend that you upgrade at the next
available opportunity.
In addition, it includes these noteworthy features:
* Ability to change compression settings on existing compressed hypertables at any time.
New compression settings take effect on any new chunks that are compressed after the change.
* Reduced locking requirements during chunk recompression
* Limiting tuple decompression during DML operations to avoid decompressing a lot of tuples and causing storage issues (100k limit, configurable)
* Helper functions for determining compression settings
**For this release only**, you will need to restart the database before running `ALTER EXTENSION`
**Multi-node support removal announcement**
Following the deprecation announcement for Multi-node in TimescaleDB 2.13,
Multi-node is no longer supported starting with TimescaleDB 2.14.
TimescaleDB 2.13 is the last version that includes multi-node support. Learn more about it [here](docs/MultiNodeDeprecation.md).
If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB, read the
[migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/).
**Deprecation notice: recompress_chunk procedure**
TimescaleDB 2.14 is the last version that will include the recompress_chunk procedure. Its
functionality will be replaced by the compress_chunk function, which, starting on TimescaleDB 2.14,
works on both uncompressed and partially compressed chunks.
The compress_chunk function should be used going forward to fully compress all types of chunks or even recompress
old fully compressed chunks using new compression settings (through the newly introduced recompress optional parameter).
**Features**
* #6325 Add plan-time chunk exclusion for real-time CAggs
* #6360 Remove support for creating Continuous Aggregates with old format
* #6386 Add functions for determining compression defaults
* #6410 Remove multinode public API
* #6440 Allow SQLValueFunction pushdown into compressed scan
* #6463 Support approximate hypertable size
* #6513 Make compression settings per chunk
* #6529 Remove reindex_relation from recompression
* #6531 Fix if_not_exists behavior for CAgg policy with NULL offsets
* #6545 Remove restrictions for changing compression settings
* #6566 Limit tuple decompression during DML operations
* #6579 Change compress_chunk and decompress_chunk to idempotent version by default
* #6608 Add LWLock for OSM usage in loader
* #6609 Deprecate recompress_chunk
* #6609 Add optional recompress argument to compress_chunk
**Bugfixes**
* #6541 Inefficient join plans on compressed hypertables.
* #6491 Enable now() plantime constification with BETWEEN
* #6494 Fix create_hypertable referenced by fk succeeds
* #6498 Suboptimal query plans when using time_bucket with query parameters
* #6507 time_bucket_gapfill with timezones doesn't handle daylight savings
* #6509 Make extension state available through function
* #6512 Log extension state changes
* #6522 Disallow triggers on CAggs
* #6523 Reduce locking level on compressed chunk index during segmentwise recompression
* #6531 Fix if_not_exists behavior for CAgg policy with NULL offsets
* #6571 Fix pathtarget adjustment for MergeAppend paths in aggregation pushdown code
* #6575 Fix compressed chunk not found during upserts
* #6592 Fix recompression policy ignoring partially compressed chunks
* #6610 Ensure qsort comparison function is transitive
**Thanks**
* @coney21 and @GStechschulte for reporting the problem with inefficient join plans on compressed hypertables.
* @HollowMan6 for reporting triggers not working on materialized views of
CAggs
* @jbx1 for reporting suboptimal query plans when using time_bucket with query parameters
* @JerkoNikolic for reporting the issue with gapfill and DST
* @pdipesh02 for working on removing the old Continuous Aggregate format
* @raymalt and @martinhale for reporting very slow query plans on realtime CAggs queries
This release contains bug fixes since the 2.13.0 release.
We recommend that you upgrade at the next available opportunity.
**Bugfixes**
* #6365 Use numrows_pre_compression in approximate row count
* #6377 Use processed group clauses in PG16
* #6384 Change bgw_log_level to use PGC_SUSET
* #6393 Disable vectorized sum for expressions.
* #6408 Fix groupby pathkeys for gapfill in PG16
* #6428 Fix index matching during DML decompression
* #6439 Fix compressed chunk permission handling on PG16
* #6443 Fix lost concurrent CAgg updates
* #6454 Fix unique expression indexes on compressed chunks
* #6465 Fix use of freed path in decompression sort logic
**Thanks**
* @MA-MacDonald for reporting an issue with gapfill in PG16
* @aarondglover for reporting an issue with unique expression indexes on compressed chunks
* @adriangb for reporting an issue with security barrier views on pg16
This release contains performance improvements, an improved hypertable DDL API
and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next
available opportunity.
In addition, it includes these noteworthy features:
* Full PostgreSQL 16 support for all existing features
* Vectorized aggregation execution for sum()
* Track chunk creation time used in retention/compression policies
**Deprecation notice: Multi-node support**
TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node
support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it
[here](docs/MultiNodeDeprecation.md).
If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the
[migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/).
**PostgreSQL 13 deprecation announcement**
We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will
announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be
included going forward.
**Starting from TimescaleDB 2.13.0**
* No Amazon Machine Images (AMI) are published. If you previously used AMI, please
use another [installation method](https://docs.timescale.com/self-hosted/latest/install/)
* Continuous Aggregates are materialized only (non-realtime) by default
**Features**
* #5575 Add chunk-wise sorted paths for compressed chunks
* #5761 Simplify hypertable DDL API
* #5890 Reduce WAL activity by freezing compressed tuples immediately
* #6050 Vectorized aggregation execution for sum()
* #6062 Add metadata for chunk creation time
* #6077 Make Continous Aggregates materialized only (non-realtime) by default
* #6177 Change show_chunks/drop_chunks using chunk creation time
* #6178 Show batches/tuples decompressed during DML operations in EXPLAIN output
* #6185 Keep track of catalog version
* #6227 Use creation time in retention/compression policy
* #6307 Add SQL function cagg_validate_query
**Bugfixes**
* #6188 Add GUC for setting background worker log level
* #6222 Allow enabling compression on hypertable with unique expression index
* #6240 Check if worker registration succeeded
* #6254 Fix exception detail passing in compression_policy_execute
* #6264 Fix missing bms_del_member result assignment
* #6275 Fix negative bitmapset member not allowed in compression
* #6280 Potential data loss when compressing a table with a partial index that matches compression order.
* #6289 Add support for startup chunk exclusion with aggs
* #6290 Repair relacl on upgrade
* #6297 Fix segfault when creating a cagg using a NULL width in time bucket function
* #6305 Make timescaledb_functions.makeaclitem strict
* #6332 Fix typmod and collation for segmentby columns
* #6339 Fix tablespace with constraints
* #6343 Enable segmentwise recompression in compression policy
**Thanks**
* @fetchezar for reporting an issue with compression policy error messages
* @jflambert for reporting the background worker log level issue
* @torazem for reporting an issue with compression and large oids
* @fetchezar for reporting an issue in the compression policy
* @lyp-bobi for reporting an issue with tablespace with constraints
* @pdipesh02 for contributing to the implementation of the metadata for chunk creation time,
the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time
* @lkshminarayanan for all his work on PG16 support
This release contains bug fixes since the 2.12.1 release.
We recommend that you upgrade at the next available opportunity.
**Bugfixes**
* #6155 Align gapfill bucket generation with time_bucket
* #6181 Ensure fixed_schedule field is populated
* #6210 Fix EXPLAIN ANALYZE for compressed DML
This release contains bug fixes since the 2.12.0 release.
We recommend that you upgrade at the next available opportunity.
**Bugfixes**
* #6113 Fix planner distributed table count
* #6117 Avoid decompressing batches using an empty slot
* #6123 Fix concurrency errors in OSM API
* #6142 Do not throw an error when deprecation GUC cannot be read
**Thanks**
* @symbx for reporting a crash when selecting from empty hypertables
This release contains performance improvements for compressed hypertables
and continuous aggregates and bug fixes since the 2.11.2 release.
We recommend that you upgrade at the next available opportunity.
This release moves all internal functions from the _timescaleb_internal
schema into the _timescaledb_functions schema. This separates code from
internal data objects and improves security by allowing more restrictive
permissions for the code schema. If you are calling any of those internal
functions you should adjust your code as soon as possible. This version
also includes a compatibility layer that allows calling them in the old
location but that layer will be removed in 2.14.0.
**PostgreSQL 12 support removal announcement**
Following the deprecation announcement for PostgreSQL 12 in TimescaleDB 2.10,
PostgreSQL 12 is not supported starting with TimescaleDB 2.12.
Currently supported PostgreSQL major versions are 13, 14 and 15.
PostgreSQL 16 support will be added with a following TimescaleDB release.
**Features**
* #5137 Insert into index during chunk compression
* #5150 MERGE support on hypertables
* #5515 Make hypertables support replica identity
* #5586 Index scan support during UPDATE/DELETE on compressed hypertables
* #5596 Support for partial aggregations at chunk level
* #5599 Enable ChunkAppend for partially compressed chunks
* #5655 Improve the number of parallel workers for decompression
* #5758 Enable altering job schedule type through `alter_job`
* #5805 Make logrepl markers for (partial) decompressions
* #5809 Relax invalidation threshold table-level lock to row-level when refreshing a Continuous Aggregate
* #5839 Support CAgg names in chunk_detailed_size
* #5852 Make set_chunk_time_interval CAggs aware
* #5868 Allow ALTER TABLE ... REPLICA IDENTITY (FULL|INDEX) on materialized hypertables (continuous aggregates)
* #5875 Add job exit status and runtime to log
* #5909 CREATE INDEX ONLY ON hypertable creates index on chunks
**Bugfixes**
* #5860 Fix interval calculation for hierarchical CAggs
* #5894 Check unique indexes when enabling compression
* #5951 _timescaledb_internal.create_compressed_chunk doesn't account for existing uncompressed rows
* #5988 Move functions to _timescaledb_functions schema
* #5788 Chunk_create must add an existing table or fail
* #5872 Fix duplicates on partially compressed chunk reads
* #5918 Fix crash in COPY from program returning error
* #5990 Place data in first/last function in correct mctx
* #5991 Call eq_func correctly in time_bucket_gapfill
* #6015 Correct row count in EXPLAIN ANALYZE INSERT .. ON CONFLICT output
* #6035 Fix server crash on UPDATE of compressed chunk
* #6044 Fix server crash when using duplicate segmentby column
* #6045 Fix segfault in set_integer_now_func
* #6053 Fix approximate_row_count for CAggs
* #6081 Improve compressed DML datatype handling
* #6084 Propagate parameter changes to decompress child nodes
**Thanks**
* @ajcanterbury for reporting a problem with lateral joins on compressed chunks
* @alexanderlaw for reporting multiple server crashes
* @lukaskirner for reporting a bug with monthly continuous aggregates
* @mrksngl for reporting a bug with unusual user names
* @willsbit for reporting a crash in time_bucket_gapfill
This release contains bug fixes since the 2.11.1 release.
We recommend that you upgrade at the next available opportunity.
**Features**
* #5923 Feature flags for TimescaleDB features
**Bugfixes**
* #5680 Fix DISTINCT query with JOIN on multiple segmentby columns
* #5774 Fixed two bugs in decompression sorted merge code
* #5786 Ensure pg_config --cppflags are passed
* #5906 Fix quoting owners in sql scripts.
* #5912 Fix crash in 1-step integer policy creation
**Thanks**
* @mrksngl for submitting a PR to fix extension upgrade scripts
* @ericdevries for reporting an issue with DISTINCT queries using
segmentby columns of compressed hypertable
This release contains bug fixes since the 2.11.0 release. We recommend
that you upgrade at the next available opportunity.
**Features**
* #5679 Teach loader to load OSM extension
**Bugfixes**
* #5705 Scheduler accidentally getting killed when calling `delete_job`
* #5742 Fix Result node handling with ConstraintAwareAppend on
compressed chunks
* #5750 Ensure tlist is present in decompress chunk plan
* #5754 Fixed handling of NULL values in bookend_sfunc
* #5798 Fixed batch look ahead in compressed sorted merge
* #5804 Mark cagg_watermark function as PARALLEL RESTRICTED
* #5807 Copy job config JSONB structure into current MemoryContext
* #5824 Improve continuous aggregate query chunk exclusion
**Thanks**
* @JamieD9 for reporting an issue with a wrong result ordering
* @xvaara for reporting an issue with Result node handling in
ConstraintAwareAppend
Adds a simple check to ensure that the PR number is present at least once
in the added changelog file.
Also fixes an earlier PR which introduced a typo.
This patch does following:
1. Planner changes to create ChunkDispatch node when MERGE command
has INSERT action.
2. Changes to map partition attributes from a tuple returned from
child node of ChunkDispatch against physical targetlist, so that
ChunkDispatch node can read the correct value from partition column.
3. Fixed issues with MERGE on compressed hypertable.
4. Added more testcases.
5. MERGE in distributed hypertables is not supported.
6. Since there is no Custom Scan (HypertableModify) node for MERGE
with UPDATE/DELETE on compressed hypertables, we don't support this.
Fixes#5139
Previously when adding equivalence class members for the compressed
chunk's variables, we would only consider Vars. This led us to ignore
cases where the Var was wrapped in a RelabelType,
returning inaccurate results.
Fixed the issue by accepting Vars
with RelabelType for segmentby equivalence class.
Fixes#5585
Add table `_timescaledb_catalog.telemetry_event` table containing
events that should be sent out with telemetry reports. The table will
be truncated after reporting being generated.
When executing a subtransaction using `BeginInternalSubTransaction` the
memory context switches from the current context to
`CurTransactionContext` and when the transaction is aborted or
committed using `ReleaseCurrentSubTransaction` or
`RollbackAndReleaseCurrentSubTransaction` respectively, it will not
restore to the previous memory context or resource owner but rather use
`TopTransactionContext`. Because of this, both the memory context and
the resource owner will be wrong when executing
`calculate_next_start_on_failure`, which causes `run_job` to generate
an error when used with the telemetry job.
This commit fixes this by saving both the resource owner and the memory
context before starting the internal subtransaction and restoring it
after finishing the internal subtransaction.
Since the `ts_bgw_job_run_and_set_next_start` was incorrectly reading
the wrong result from the telemetry job, this commit fixes this as
well. Note that `ts_bgw_job_run_and_set_next_start` is only used when
running the telemetry job, so it does not cause issues for other jobs.
Use a per-tuple memory context when receiving chunk statistics from
data nodes. Otherwise memory usage is proportional to the number of
chunks and columns.
Running ALTER TABLE SET with multiple SET clauses on a regular PostgreSQL table
produces irrelevant error when timescaledb extension is installed.
Fix#5641
If a hypertable uses a non default tablespace, the compressed hypertable
and its corresponding toast table and index is still created in the
default tablespace.
This PR fixes this unexpected behavior and creates the compressed
hypertable and its toast table and index in the same tablespace as
the hypertable.
Fixes#5520
This patch adds an optimization to the DecompressChunk node. If the
query 'order by' and the compression 'order by' are compatible (query
'order by' is equal or a prefix of compression 'order by'), the
compressed batches of the segments are decompressed in parallel and
merged using a binary heep. This preserves the ordering and the sorting
of the result can be prevented. Especially LIMIT queries benefit from
this optimization because only the first tuples of some batches have to
be decompressed. Previously, all segments were completely decompressed
and sorted.
Fixes: #4223
Co-authored-by: Sotiris Stamokostas <sotiris@timescale.com>
On compressed hypertables 3 schema levels are in use simultaneously
* main - hypertable level
* chunk - inheritance level
* compressed chunk
In the build_scankeys method all of them appear - as slot have their
fields represented as a for a row of the main hypertable.
Accessing the slot by the attribut numbers of the chunks may lead to
indexing mismatches if there are differences between the schemes.
Fixes: #5577
Since telemetry job has a special code path to be able to be used both
from Apache code and from TSL code, trying to execute the telemetry job
with run_job() will fail.
This code will allow run_job() to be used with the telemetry job to
trigger a send of telemetry data. You have to belong to the group that
owns the telemetry job (or be the owner of the telemetry job) to be
able to use it.
Closes#5605
There were no permission checks when calling run_job(), so it was
possible to execute any job regardless of who owned it. This commit
adds such checks.
The internal `cagg_rebuild_view_definition` function was trying to cast
a pointer to `RangeTblRef` but it actually is a `RangeTblEntry`.
Fixed it by using the already existing `direct_query` data struct to
check if there are JOINs in the CAgg to be repaired.
All children of an append path are required to have the same parameterization
so we have to reparameterize when the selected path does not have the right
parameterization.
Inserting multiple rows into a compressed chunk could have bypassed
constraint check in case the table had segment_by columns.
Decompression is narrowed to only consider candidates by the actual
segment_by value.
Because of caching - decompression was skipped for follow-up rows of
the same Chunk.
Fixes#5553
When inserting into a compressed chunk with constraints present,
we need to decompress relevant tuples in order to do speculative
inserting. Usually we used segment by column values to limit the
amount of compressed segments to decompress. This change expands
on that by also using segment metadata to further filter
compressed rows that need to be decompressed.
