Dropping internal compressed chunks is blocked on DDL level, but
is still possible with `_timescaledb_functions.drop_chunk` function.
This patch adds a simple check to prevent that.
get_relmergeinfo macro produced the following warning:
macro argument should be enclosed in parentheses [bugprone-macro-parentheses,-warnings-as-errors]
Reusing the existing Simple8bRLE algorithm for bools. I added a new
compression type specifically for this case called 'bool'.
A new GUC is introduced so we can revert to the previous, array
compression for bools: `timescaledb.enable_bool_compression`.
It defaults to `false`.
To enable bool compression set the GUC:
`timescaledb.enable_bool_compression=true`
Fixes#7233
The arrow cache is used to cache decompressed arrow segments and is
mostly useful for index scans that access values across segments in
non-segment order.
The fixed cache size was set too small for index scans on bigger data
sets. Therefore, increase the default size and make it configurable via
a GUC.
If a crossmodule wrapper function is called as part of a query on a
fresh backend, it will generate a license error even if the license is
set correctly. This happens because the module load happens very late
(in `post_parse_analyze_hook`) but the function call is quite early.
This is fixed by eagerly loading the TSL module in the hypercore
handler function and hypercore index proxy handler function.
Enables the segmentwise recompression flow to be used for chunks without
segmentby columns.
This should be more performant than doing a full recompression.
This patch adds support for ALTER TABLE ALTER COLUMN SET NOT NULL
to compressed chunks. The statement will be allowed when no NULL
values for the specific column are present in compressed chunks.
Add support for running VectorAgg on top of scans on Hypercore
TAM. Currently, only ColumnarScan can run below VectorAgg when
Hypercore TAM is used. In theory, a SeqScan or IndexScan reading from
Hypercore TAM should also work because they would produce Arrow
slots. However, only ColumnarScan performs vectorized filtering, which
is currently assumed to happen before the VectorAgg node.
In ColumnarScan, it is necessary to turn off projection when VectorAgg
is used. Otherwise, it would project the arrow slot into a virtual
slot, thus losing the vector data. Ideally, a projection should never
be planned to begin with, but this isn't possible since VectorAgg
modifies existing non-vectorized Agg plans that already includes
projections.
For the remaining nightly runs.
Remove reference to non-existent trigger/homebrew_test branch.
---------
Signed-off-by: Pallavi Sontakke <pallavi@timescale.com>
Co-authored-by: Philip Krauss <35487337+philkra@users.noreply.github.com>
Hooks in postgres are supposed to be chained and call the previous
hook in your own added hooks. Not calling previous hook will prevent
other extensions from having working hooks.
Vectorized aggregation assumes that it runs on top of a
DecompressChunk child node, which makes it difficult to support other
child plans; including those that fetch data via Hypercore TAM.
Most of the DecompressChunk-specific code for planning VectorAgg
relates to identifying vectorizable columns. This code is moved to a
separate source file so that the main planning code is mostly
child-node independent.
Currently while a job is running we set `pid = SchedulerPid`,
`succeed = false` and `execution_finish=NOW()` and it leads to
confusion when querying either `timescaledb_information.job_errors`
or `timescaledb_information.job_history` views showing in the
`err_message = job crash detected, see server logs`. This information
is wrong and create confusion.
Fixed it by setting `succeed=NULL` and `pid=NULL` when the scheduler
launch the job and then when the job worker start to work then set
`pid=MyProcPid` (the worker PID) meaning that the job started and
didn't finished yet, and at the end of the execution we set
`succeed=TRUE or FALSE` and the `execution_finish=NOW()` to mark the
end of the job execution. Also adjusted the views to expose the
information properly.
It should not be possible to merge a frozen chunk since it is used to
tier chunks. If the chunk is merged, it might no longer exist when the
tiering happens.
When an INSERT with ON CONFLICT DO NOTHING hits the first conflicts
it would abort additional INSERTS following the INSERT triggering
the DO NOTHING clause leading to missed INSERTs.
Fixes#7672
Since we split the fixes and thanks messages into separate sections
in the changelog the context between fix and thanks will be lost so
the thanks note should repeat any required context
Since we depend on the openssl version of the postgres installer
to match the openssl version we built against and we can't ensure
stability of that version in the postgres installer we only test
windows package against the latest available postgres version.
When filtering arrow slots in ColumnarScan, quals on segmentby columns
should be executed separately from those on other columns because they
don't require decompression and might filter the whole arrow slot in
one go. Furthermore, the quals only need to be applied once per arrow
slot since the segmentby value is the same for all compressed rows in
the slot.
This will speed up scans when filters on segmentby columns cannot be
pushed down to Hypercore TAM as scankeys. For example, "<column> IN
(1, 2, 3)" won't be pushed down as a scankey because only index scans
support scankeys with such scalar array expressions.
Before compressed chunks were mutable, adding a compression policy
to a continuous aggregate that could include portions of the refrsh
window were blocked. When a continuous aggregate is refreshed, the
underlying chunks needed to allow DELETEs and INSERTs, so they
could not be compressed. Now, compressed chunks allow both operations
and there is no longer a need to prevent the refresh window and
compression window from overlapping.
Quals on orderby columns can be pushed down to Hypercore TAM and be
transformed to the corresponding min/max scankeys on the compressed
relation. Previously, only quals on non-compressed segmentby columns
were pushed down as scankeys.
Pushing down orderby scan keys seem to give a good performance boost
for columnar scans when no index exists.
The scankey push down can be disabled with a new GUC:
`timescaledb.enable_hypercore_scankey_pushdown=false`
Depending on the branch protection settings, we might have to update
them before they can be merged, so do this automatically to minimize the
required manual work.
The EquivalenceMember lookup is the most costly part, so share it
between different uses.
Switch batch sorted merge to use the generic pathkey matching code.
Also cache some intermediate data in the CompressionInfo struct.
When pushing down expressions into the compressed scan we assumed
all valid expressions use btree operators and dropped any that weren't.
This patch changes the behaviour to keep those expressions and use
them as heap filter on the compressed scan for UPDATE and DELETE
on compressed chunks.
The VectorAgg exec loop reads tuples directly from a compressed
relation, thus bypassing the DecompressChunk child node. This won't
work with arrow slots, which are read via a table access method.
To make the VectorAgg exec code similar to the standard pattern of
reading slots from child nodes, code specific to decompressing batches
is moved out of the main VectorAgg exec loop so that the loop only
deals with the final compressed batch slot instead of the raw
compressed slot. The code is instead put in a "get_next_slot"
function, which is called from the loop.
Also move the code to initialize vectorized filters to its own
"init_vector_qual" function, since it is specific to compressed
batches.
With these two function interfaces, it is possible to provide
implementations of the functions for handling arrow slots.
The downgrade script for 2.17.2 that was merged into main branch
was different from the one in 2.18.0 due to merge error. This
patch syncs the downgrade script with the version used in 2.18.0
release.
