Until PG 17.1 is released.
Currently shows a diff in PG 17.0 regression run as:
```
select jsonb_path_query_tz('"12:34:56"', '$.time_tz().string()');
jsonb_path_query_tz
---------------------
- "12:34:56-07:00"
+ "12:34:56-08:00"
```
Add GUC option to enable or disable segmentwise recompression. If
disabled, then a full recompression is done instead when recompression
is attempted through `compress_chunk`. If `recompress_chunk_segmentwise`
is used when GUC is disabled, then an error is thrown.
Closes#7381.
Coverity assumes that casting a void pointer to anything else is
tainted, which gives warning when casting the `rd_amcache` field to
anything else. Since this is what the field is used for, we mark the
cast from `void*` to `HypercoreInfo*` as a false positive.
Due to a typo, we were misusing OIDs and putting them
into a bitmapset which caused plan time errors. Switching
to using RTE index and adding a test with OID that is
greater than INT32_MAX.
A [previous
commit](d7513b26f6)
I made added support for MERGE ... RETURNING ....
While working on the coverity fix, I realized that this code handling
oldtuple was not added; that's what this fixes.
Disable-check: force-changelog-file
Currently if a PR that is elegible for backporting touches a workflow
file then the automatic backport fails.
Now adding the label `force-auto-backport-workflow` to a PR that touches
workflow files will proceed with the automatic backporting. This is
useful because sometimes we need to fix a workflow and backport it to
the current release branch or when we're adding support to a new
Postgres major version that requires workflow changes and it should be
backported to the release branch in case of creating patch releases.
policy_job_error_retention was removed in 2.15.0 but we did not
get rid of the job that called it back then. This patch removes
the defunct job definition calling that function.
Coverity detected a possible null pointer dereference. It doesn't seem
like this can be triggered, so added an `Ensure` clause.
Disable-check: force-changelog-file
This release contains performance improvements and bug fixes since
the 2.17.0 release. We recommend that you upgrade at the next
available opportunity.
**Features**
* #7360 Add chunk skipping GUC
**Bugfixes**
* #7335 Change log level used in compression
* #7342 Fix collation for in-memory tuple filtering
**Thanks**
* @gmilamjr for reporting an issue with the log level of compression messages
* @hackbnw for reporting an issue with collation during tuple filtering
The join test could sometimes pick a seqscan+sort instead of an
indexscan when doing a MergeAppend+MergeJoin. Disabling seqscan should
make it deterministic.
Having multiple indexes that include same prefix of columns caused the
planner to sometimes pick a different index for one of the querires,
which led to different test output. Temporarily remove the alternative
index to make the test predictible.
When queried from within action context .authorAssociation is not
filled in as MEMBER but CONTRIBUTOR instead so adjust to query to
take that into account.
Previously ordered queries on realtime caggs would always lead to full
table scan as the query plan would have a sort with the limit on top.
With this patch this gets changed so that the ORDER BY can be pushed
down so the query can benefit from the ordered append optimization and
does not require full table scan.
Since the internal structure is different on PG 14 and 15 this
optimization will only be available on PG 16 and 17.
Fixes#4861
All PRs except trivial ones should require 2 approvals, since this a global setting
we cannot allow trivial PRs to only have 1 from github configuration alone. So we set
the required approvals in github to 1 and make this check required which will enforce
2 approvals unless overwritten or only CI files are touched.
The Hypercore table access method (TAM) wraps TimescaleDB's columnar
compression engine in a table access method. The TAM API enables
sevaral features that were previously not available on compressed
data, including (but not limited to):
- Ability to build indexes on compressed data (btree, hash).
- Proper statistics, including column stats via ANALYZE
- Better support for vacuum and vacuum full
- Skip-scans on top of compressed data
- Better support for DML (copy/insert/update) directly on compressed
chunks
- Ability to dynamically create constraints (check, unique, etc.)
- Better lock handling including via CURSORs
PG17 changed the TAM API to use the new `ReadStream` API instead of the
previous block-oriented API. This commit port the existing
block-oriented solution to use the new `ReadStream` API by setting up
separate read streams for the two relations and using the provided read
stream as a block sampler, fetching the apropriate block from either
the non-compressed or compressed relation.
The macro `TS_WITH_MEMORY_CONTEXT` was used to switch memory context
for a block of code and restore it afterwards. This is checked using
Coccinelle rules instead and the macro is removed.
Truncate the compressed relation when truncating a hyperstore relation.
This can happen in two situations: either a non-transactional context
or in a transactional context.
For the transactional context, `relation_set_new_filelocator`
will be called to replace the file locator. If this happens, we need to
replace the file locator for the compressed relation as well, if there
is one.
For the non-transactional case, `relation_nontransactional_truncate`
will be called, and we will just forward the call to the compressed
relation as well, if it exists.
If an index scan is on segment-by columns only, the index is optimized
to only contain references to complete segments. However, deciding if a
scan is only on segment-by columns requires checking all columns used
in the index scan and since this does not change during a scan, but
needs to be checked for each tuple, we cache this information for the
duration of the scan.
The index build function didn't properly handle the case when all
rolled-up values in a compressed column were null, thus having a
null-segment. The code has been slightly refactored to handle this
case.
A test is also added for this case.
Dropped columns need to be included in a tuple table slot's values
array after having called slot_getsomeattrs(). The arrow slot didn't
do this and instead skipped dropped columns, which lead to assertion
errors in some cases.
When recompressing a Hyperstore after changing compression settings,
the compressed chunk could be created twice, leading to a conflict
error when inserting two compression chunk size rows.
The reason this happened was that Hyperstore creates a compressed
chunk on-demand if it doesn't exist when the relation is opened. And,
the recompression code had not yet associated the compressed chunk
with the main chunk when compressing the data.
Fix this by associating the compressed chunk with the main chunk
before opening the main chunk relation to compress the data.
When populating an Arrow slot's tts_values array with values in the
getsomeattrs() function, the function set_attr_value() is called. This
function requires passing in an ArrowArray which is acquired via a
compression cache lookup. However, that lookup is not necassary for
segmentby columns (which aren't compressed) and, to avoid it, a
special fast-path was created for segmentby columns outside
set_attr_value(). That, unfortunately, created som code duplication.
This change moves the cache lookup into set_attr_value() instead,
where it can be performed only for the columns that need it. This
leads to cleaner code and less code duplication.
When copying from a non-arrow slot to a arrow slot, we should always
copy the data into the non-compressed slot and never to the compressed
slot.
The previous check for matching number of attributes fail when you drop
one column from the hyperstore.
We disable hash aggregation in favor of group aggregation to get stable
test. It was flaky because it could pick either group aggregate or hash
aggregate.
If you set the table access method for a hypertable all new chunks will
use `ArrowTupleTableSlot` but the copy code assumes that the parent
table has a virtual tuple table slot. This causes a crash when copying
a heap tuple since the values are stored in the "main" slot and not in
either of the child tuple table slots.
Fix this issue by storing the values in the uncompressed slot when it
is empty.
If an attempt is made to use the hyperstore table access method with a
plain table during creation, throw an error instead of allowing the
table access method to be used.
The table access method currently only supports hypertables and expect
chunks to exist for the table.
When a columnar scan needs to return a subset of the columns in a scan
relation, it is possible to do a "simple" projection that just copies
the column values to the projection result slot. This avoids a more
costly projection done by PostgreSQL.
The tableOid was not set in an Arrow slot when hyperstore was
delivering the next arrow value from the same compressed child slot,
assuming that the tableOid would remain the same since since
delivering the previous value.
This is not always the case, however, as the same slot can be used in
a CREATE TABLE AS or similar statement that inserts the data into
another table. In that case, the insert function of that table will
change the slot's tableOid.
To fix this, hyperstore will always set the tableOid on the slot when
delivering new values.
