This release contains performance improvements and bug fixes since
the 2.18.1 release. We recommend that you upgrade at the next
available opportunity.
**Bugfixes**
* #7686 Potential wrong aggregation result when using vectorized aggregation with hash grouping in reverse order
* #7694 Fix ExplainHook breaking call chain
* #7695 Block dropping internal compressed chunks with `drop_chunk()`
* #7711 License error when using hypercore handler
* #7712 Respect other extensions' ExecutorStart hooks
**Thanks**
* @davidmehren and @jflambert for reporting an issue with extension hooks
* @jflambert for reporting a bug with license errors shown in autovacuum
This will help update the pending backport PR to the latest version of
the release branch.
GitHub doesn't provide a built-in possibility to trigger a workflow on
the main branch when another branch has changed. For example, if we had
a "push: ?.*.x" condition, it would trigger the backport workflow on the
release branch itself, which is not what we need. So we have to use a
helper workflow instead.
The problem with the test in compression_unti_test.c was that
tsl_bool_compressor_finish() expects a BoolCompressor and it
should get it from its pair: tsl_bool_compressor_append().
The test supplied an incompatible type (ExtendedCompressor)
coming from bool_compressor_for_type() and sanitizer caught it.
Disable-check: force-changelog-file
Dropping internal compressed chunks is blocked on DDL level, but
is still possible with `_timescaledb_functions.drop_chunk` function.
This patch adds a simple check to prevent that.
get_relmergeinfo macro produced the following warning:
macro argument should be enclosed in parentheses [bugprone-macro-parentheses,-warnings-as-errors]
Reusing the existing Simple8bRLE algorithm for bools. I added a new
compression type specifically for this case called 'bool'.
A new GUC is introduced so we can revert to the previous, array
compression for bools: `timescaledb.enable_bool_compression`.
It defaults to `false`.
To enable bool compression set the GUC:
`timescaledb.enable_bool_compression=true`
Fixes#7233
The arrow cache is used to cache decompressed arrow segments and is
mostly useful for index scans that access values across segments in
non-segment order.
The fixed cache size was set too small for index scans on bigger data
sets. Therefore, increase the default size and make it configurable via
a GUC.
If a crossmodule wrapper function is called as part of a query on a
fresh backend, it will generate a license error even if the license is
set correctly. This happens because the module load happens very late
(in `post_parse_analyze_hook`) but the function call is quite early.
This is fixed by eagerly loading the TSL module in the hypercore
handler function and hypercore index proxy handler function.
Enables the segmentwise recompression flow to be used for chunks without
segmentby columns.
This should be more performant than doing a full recompression.
This patch adds support for ALTER TABLE ALTER COLUMN SET NOT NULL
to compressed chunks. The statement will be allowed when no NULL
values for the specific column are present in compressed chunks.
Add support for running VectorAgg on top of scans on Hypercore
TAM. Currently, only ColumnarScan can run below VectorAgg when
Hypercore TAM is used. In theory, a SeqScan or IndexScan reading from
Hypercore TAM should also work because they would produce Arrow
slots. However, only ColumnarScan performs vectorized filtering, which
is currently assumed to happen before the VectorAgg node.
In ColumnarScan, it is necessary to turn off projection when VectorAgg
is used. Otherwise, it would project the arrow slot into a virtual
slot, thus losing the vector data. Ideally, a projection should never
be planned to begin with, but this isn't possible since VectorAgg
modifies existing non-vectorized Agg plans that already includes
projections.
For the remaining nightly runs.
Remove reference to non-existent trigger/homebrew_test branch.
---------
Signed-off-by: Pallavi Sontakke <pallavi@timescale.com>
Co-authored-by: Philip Krauss <35487337+philkra@users.noreply.github.com>
Hooks in postgres are supposed to be chained and call the previous
hook in your own added hooks. Not calling previous hook will prevent
other extensions from having working hooks.
Vectorized aggregation assumes that it runs on top of a
DecompressChunk child node, which makes it difficult to support other
child plans; including those that fetch data via Hypercore TAM.
Most of the DecompressChunk-specific code for planning VectorAgg
relates to identifying vectorizable columns. This code is moved to a
separate source file so that the main planning code is mostly
child-node independent.
Currently while a job is running we set `pid = SchedulerPid`,
`succeed = false` and `execution_finish=NOW()` and it leads to
confusion when querying either `timescaledb_information.job_errors`
or `timescaledb_information.job_history` views showing in the
`err_message = job crash detected, see server logs`. This information
is wrong and create confusion.
Fixed it by setting `succeed=NULL` and `pid=NULL` when the scheduler
launch the job and then when the job worker start to work then set
`pid=MyProcPid` (the worker PID) meaning that the job started and
didn't finished yet, and at the end of the execution we set
`succeed=TRUE or FALSE` and the `execution_finish=NOW()` to mark the
end of the job execution. Also adjusted the views to expose the
information properly.
It should not be possible to merge a frozen chunk since it is used to
tier chunks. If the chunk is merged, it might no longer exist when the
tiering happens.
When an INSERT with ON CONFLICT DO NOTHING hits the first conflicts
it would abort additional INSERTS following the INSERT triggering
the DO NOTHING clause leading to missed INSERTs.
Fixes#7672
Since we split the fixes and thanks messages into separate sections
in the changelog the context between fix and thanks will be lost so
the thanks note should repeat any required context
Since we depend on the openssl version of the postgres installer
to match the openssl version we built against and we can't ensure
stability of that version in the postgres installer we only test
windows package against the latest available postgres version.
When filtering arrow slots in ColumnarScan, quals on segmentby columns
should be executed separately from those on other columns because they
don't require decompression and might filter the whole arrow slot in
one go. Furthermore, the quals only need to be applied once per arrow
slot since the segmentby value is the same for all compressed rows in
the slot.
This will speed up scans when filters on segmentby columns cannot be
pushed down to Hypercore TAM as scankeys. For example, "<column> IN
(1, 2, 3)" won't be pushed down as a scankey because only index scans
support scankeys with such scalar array expressions.
Before compressed chunks were mutable, adding a compression policy
to a continuous aggregate that could include portions of the refrsh
window were blocked. When a continuous aggregate is refreshed, the
underlying chunks needed to allow DELETEs and INSERTs, so they
could not be compressed. Now, compressed chunks allow both operations
and there is no longer a need to prevent the refresh window and
compression window from overlapping.
Quals on orderby columns can be pushed down to Hypercore TAM and be
transformed to the corresponding min/max scankeys on the compressed
relation. Previously, only quals on non-compressed segmentby columns
were pushed down as scankeys.
Pushing down orderby scan keys seem to give a good performance boost
for columnar scans when no index exists.
The scankey push down can be disabled with a new GUC:
`timescaledb.enable_hypercore_scankey_pushdown=false`
Depending on the branch protection settings, we might have to update
them before they can be merged, so do this automatically to minimize the
required manual work.
