On compressed hypertables 3 schema levels are in use simultaneously
* main - hypertable level
* chunk - inheritance level
* compressed chunk
In the build_scankeys method all of them appear - as slot have their
fields represented as a for a row of the main hypertable.
Accessing the slot by the attribut numbers of the chunks may lead to
indexing mismatches if there are differences between the schemes.
Fixes: #5577
Since telemetry job has a special code path to be able to be used both
from Apache code and from TSL code, trying to execute the telemetry job
with run_job() will fail.
This code will allow run_job() to be used with the telemetry job to
trigger a send of telemetry data. You have to belong to the group that
owns the telemetry job (or be the owner of the telemetry job) to be
able to use it.
Closes#5605
We defined some paths to ignore regression test workflows when, for
example, we change the CHANGELOG.md and others. But in fact it was not
happening because we didn't define a proper name in the fake regression
workflow that fake to the CI that some required status passed.
Fixed it defining a proper regression name like we do for the regular
regression workflow.
There were no permission checks when calling run_job(), so it was
possible to execute any job regardless of who owned it. This commit
adds such checks.
The internal `cagg_rebuild_view_definition` function was trying to cast
a pointer to `RangeTblRef` but it actually is a `RangeTblEntry`.
Fixed it by using the already existing `direct_query` data struct to
check if there are JOINs in the CAgg to be repaired.
When updating or deleting tuples from a compressed chunk, we first
need to decompress the matching tuples then proceed with the operation.
This optimization reduces the amount of data decompressed by using
compressed metadata to decompress only the affected segments.
All children of an append path are required to have the same parameterization
so we have to reparameterize when the selected path does not have the right
parameterization.
This block was removed by accident, in order to support this we
need to ensure the uniqueness in the compressed data which is
something we should do in the future thus removing this block.
Inserting multiple rows into a compressed chunk could have bypassed
constraint check in case the table had segment_by columns.
Decompression is narrowed to only consider candidates by the actual
segment_by value.
Because of caching - decompression was skipped for follow-up rows of
the same Chunk.
Fixes#5553
When inserting into a compressed chunk with constraints present,
we need to decompress relevant tuples in order to do speculative
inserting. Usually we used segment by column values to limit the
amount of compressed segments to decompress. This change expands
on that by also using segment metadata to further filter
compressed rows that need to be decompressed.
## 2.10.2 (2023-04-20)
**Bugfixes**
* #5410 Fix file trailer handling in the COPY fetcher
* #5446 Add checks for malloc failure in libpq calls
* #5233 Out of on_proc_exit slots on guc license change
* #5428 Use consistent snapshots when scanning metadata
* #5499 Do not segfault on large histogram() parameters
* #5470 Ensure superuser perms during copy/move chunk
* #5500 Fix when no FROM clause in continuous aggregate definition
* #5433 Fix join rte in CAggs with joins
* #5556 Fix duplicated entries on timescaledb_experimental.policies view
* #5462 Fix segfault after column drop on compressed table
* #5543 Copy scheduled_jobs list before sorting it
* #5497 Allow named time_bucket arguments in Cagg definition
* #5544 Fix refresh from beginning of Continuous Aggregate with variable time bucket
* #5558 Use regrole for job owner
* #5542 Enable indexscan on uncompressed part of partially compressed chunks
**Thanks**
* @nikolaps for reporting an issue with the COPY fetcher
* @S-imo-n for reporting the issue on Background Worker Scheduler crash
* @geezhu for reporting issue on segfault in historgram()
* @mwahlhuetter for reporting the issue with joins in CAggs
* @mwahlhuetter for reporting issue with duplicated entries on timescaledb_experimental.policies view
* @H25E for reporting error refreshing from beginning of a Continuous Aggregate with variable time bucket
This was previously disabled as no data resided on the
uncompressed chunk once it was compressed, but this is not
the case anymore with partially compressed chunks, so we
enable indexscan for the uncompressed chunk again.
Fixes#5432
Co-authored-by: Ante Kresic <ante.kresic@gmail.com>
Instead of using a user name to register the owner of a job, we use
regrole. This allows renames to work properly since the underlying OID
does not change when the owner name changes.
We add a check when calling `DROP ROLE` that there is no job with that
owner and generate an error if there is.
Our cost model should be self-consistent, and the relative values for
the remote tuple and startup costs should reflect their real cost,
relative to costs of other operations like CPU tuple cost.
For example, now remote costs are set even lower than the parallel tuple
and startup cost. Contrary to that, their real world cost is going to be
an order of magnitude higher or more, because parallel tuples are sent
through shared memory, and remote tuples are sent over the network.
Increasing these costs leads to query plan improvements, e.g. we start
to favor the GROUP BY pushdown in some cases.
Commit 3f9cb3c2 introduced new repair tests for broken Continuous
Aggregates with JOIN clause but in the post.repair.sql we not properly
calling the post.repair.cagg_joins.sql because a wrong usage of psql
`if` statement.
Whenever we create a template sql file (*.sql.in) we should add the
respective .gitignore entry for the generated test files.
So added a CI check to check for missing gitignore entries for generated
test files.
The start_scheduled_jobs function mistakenly sorts the scheduled_jobs
list in-place. As a result, when the ts_update_scheduled_jobs_list
function compares the updated list of scheduled jobs with the existing
scheduled jobs list, it is comparing a list that is sorted by job_id to
one that is sorted by next_start time. Fix that by properly copying the
scheduled_jobs list into a new list and use that for sorting.
Fixes#5537
The cmake call to "find_package(OpenSSL)" on windows may set the
variable OPENSSL_LIBRARIES to a non-standard list which might include
non path items such as "debug", "optimized". These non-standard
elements in the list would cause the link failure with error
"LNK1104".
Fix:
Check and retain only valid paths in OPENSSL_LIBRARIES list.
closes#407
With recent changes, we enabled analyze on uncompressed chunk tables
for compressed chunks. This change includes analyzing the compressed
chunks table when analyzing the hypertable and its chunks,
enabling us to remove the generating stats when compressing chunks.
In case of joins in the continuous aggregates, pass the required
structs to the new rte created. These values are required by the
planner to finally query the materialized view.
Fixes#5433
Commit 16fdb6ca5e introduced `timescaledb_experimental.policies` view
to expose the Continuous Aggregate policies but the current JOINS over
our catalog are not accurate.
Fixed it by properly JOIN the underlying catalog tables to expose the
correct information without duplicates about the Continuous Aggregate
policies.
Fixes#5492
When refreshing from the beginning (window_start=NULL) of a
Continuous Aggregate with variable time bucket we were getting a
`timestamp out of range` error.
Fixed it by setting `-Infinity` when passing `window_start=NULL` when
refreshing a Continuous Aggregate with variable time bucket.
Fixes#5474, #5534
Several error messages for continuous aggregates are not following the
error message style guidelines at
https://www.postgresql.org/docs/current/error-style-guide.html
In particular, they do not write the hints and detailed messages as
full sentences.
This is mostly a cosmetic change. When only 1 child is present there
is no need for ordered append. In this situation we might still
benefit from a ChunkAppend node here due to runtime chunk exclusion
when we have non-immutable constraints, so we still add the ChunkAppend
node in that situation even with only 1 child.
This patch drops the following internal SQL functions which were
unused:
_timescaledb_internal.is_main_table(regclass);
_timescaledb_internal.is_main_table(text, text);
_timescaledb_internal.hypertable_from_main_table(regclass);
_timescaledb_internal.main_table_from_hypertable(integer);
_timescaledb_internal.time_literal_sql(bigint, regtype);
While executing compression operations in parallel with
inserting into chunks (both operations which can potentially
change the chunk status), we could get into situations where
the chunk status would end up inconsistent. This change re-reads
the chunk status after locking the chunk to make sure it can
decompress data when handling ON CONFLICT inserts correctly.
Verify that insertion into compressed chunks does not block
each other if the chunks is already partially compressed. Also
check that using the RETURNING clause works the same.
These tests try to verify that changing physical layout
of chunks (either compressed or uncompressed) should
yield consistent results. They also verify index mapping
on compressed chunks is handled correctly.
Commit 8afdddc2da added the first step for deprecating the old format
of Continuous Aggregate but just for PostgreSQL 15 and later versions.
During the extension update we emit a message about the deprecation but
this has being emited even if the user is using PostgreSQL versions
before 15.
Fixed it by emiting the WARNING just when PostgreSQL version is greater
or equal to 15.
This patch moves the support functions for histogram, first and last
into the _timescaledb_functions schema. Since we alter the schema
of the existing functions in upgrade scripts and do not change the
aggregates this should work completely transparently for any user
objects using those aggregates.
Commit 57fde383b3dddd0b52263218e65a0135981c2d34 changed the
messaging but did not format the error hint correctly.
This patch fixes the error hint.
Fixes#5490
