4561 Commits

Author SHA1 Message Date
Ante Kresic
24ae38b651 Fix flaky SQL test
During dropping of the database used in the various tests
we can encounter an error if something is still connected
to the database, making the output unpredicatable. This
change should reduce the possibility of that happening.
2023-12-19 15:53:41 +01:00
Alexander Kuzmenkov
a64ae61c56 Use a stateful detoaster for compressed data
The normal Postgres detoasting code locks and opens toast tables and
indexes for each toast value, which can take a large percentage CPU time
on simple queries. Since in decompression we're working with one table
at a time, the toast table and index are the same for every datum, so we
don't have to redo this work.
2023-12-19 13:23:20 +01:00
Sven Klemm
301612af55 Allow SQLValueFunction pushdown into compressed scan
Since all SQLValueFunction are STABLE we can safely push them
down into the scan of the compressed chunk.
The following expressions are pushed down:
- CURRENT_DATE
- CURRENT_TIME
- CURRENT_TIME(p)
- CURRENT_TIMESTAMP
- CURRENT_TIMESTAMP(p)
- LOCALTIME
- LOCALTIME(p)
- LOCALTIMESTAMP
- LOCALTIMESTAMP(p)
- USER
- CURRENT_USER
- SESSION_USER
- CURRENT_CATALOG
- CURRENT_SCHEMA
2023-12-19 09:17:07 +01:00
Sven Klemm
a1f7d3530b Fix permission handling for compressed chunks on PG16
Before this patch we required the user to have select permissions
on the compressed chunks in addition to permissions on the hypertable.
This patch changes our code to not require permission on the
compressed chunk when querying through the uncompressed hypertable
or chunk similar to how we handle this on PG < 16.
This fixes views with security_barrier that have constraints
on the user.

Fixes: #6425
2023-12-18 17:32:43 +01:00
Jan Nidzwetzki
6e76702ca0 Enhance stale-bot configuration
Our stale bot currently processes only bugs and enhancements. Adding a
further label for uncategorized issues.
2023-12-18 14:41:47 +01:00
Ante Kresic
384bcaa7ad Fix index matching during DML decompression
Prior to version 2.8, compressed chunks had a different format
for indexing. This change attempts to find the best index
it can actually use during scanning of compressed data
and moves filters around based on the selected index. If
no index exists, it should fallback to doing a sequencial
scan.
2023-12-18 12:30:36 +01:00
Sven Klemm
8f73f95c2a Remove replication_factor field from _timescaledb_catalog.hypertable 2023-12-18 10:53:27 +01:00
Sven Klemm
11dd9af847 Remove multinode catalog objects
This patch removes the following objects:

tables:
- _timescaledb_catalog.chunk_data_node
- _timescaledb_catalog.dimension_partition
- _timescaledb_catalog.hypertable_data_node
- _timescaledb_catalog.remote_txn

views:
- timescaledb_information.data_nodes

functions:
- _timescaledb_functions.hypertable_remote_size
- _timescaledb_functions.chunks_remote_size
- _timescaledb_functions.indexes_remote_size
- _timescaledb_functions.compressed_chunk_remote_stats
2023-12-18 10:53:27 +01:00
Sven Klemm
d5dc251405 Remove multinode drop database detection from loader 2023-12-18 10:53:27 +01:00
Sven Klemm
ce90fde526 Alter pre-update handling during downgrade script generation
Before this change during downgrade script generation we would
always fetch the pre-update script from the previous version and
prepend it to the generated scripts. This limits what can be
referenced in the pre-update script and also what is possible
within the downgrade itself.
This patch splits the pre-update script into a generic part that
is used for update/downgrade and an update specific part. We could
later also add a downgrade specific part but currently it is not
needed. This change is necessary because we reference a timescaledb
view in the pre-update script which prevents changes to that view.
2023-12-18 10:13:03 +01:00
Sven Klemm
6395b249a9 Remove remote connection handling code
Remove the code used by multinode to handle remote connections.
This patch completely removes tsl/src/remote and any remaining
distributed hypertable checks.
2023-12-15 19:13:08 +01:00
Alexander Kuzmenkov
3d8ec1e43d Make the bulk decompression function depend on PG type
This is a refactoring to enable bulk decompression of array and
dictionary compressed text columns, but not other types. Currently has
no effect.
2023-12-15 15:26:14 +01:00
Fabrízio de Royes Mello
37ece578df Remove useless AggPartCxt data structure
This data structure is a leftover of 78490c47 where we removed the
support for creating Continuous Aggregate in the old format.
2023-12-15 10:37:54 -03:00
Nikhil Sontakke
449e0ab261 Fix cagg_migrate test on MacOSx
Need to use "zcat < filename"
2023-12-15 16:31:56 +05:30
Nikhil Sontakke
987435032c Fix MacOSx build issues
The upload script needs access to the "psql" binary with its proper
path. Fix that.
2023-12-15 16:31:56 +05:30
Alexander Kuzmenkov
9a4eb12eb6 Do not produce partial aggregation plans for the only chunk
It doesn't make much sense.
2023-12-14 22:12:57 +01:00
Alexander Kuzmenkov
e0a3e309a7 Fix integer overflow in batch sorted merge costs
Also remove a leftover debug print.
2023-12-14 21:45:31 +01:00
Sven Klemm
06867af966 Remove multinode functions from crossmodule struct
This commit removes the multinode specific entries from the cross
module function struct. It also removes the function
set_chunk_default_data_node
2023-12-14 21:32:14 +01:00
Sven Klemm
4b51b8ca96 Disable failing cagg query on PG13 update test
The update test fails on PG13 on the statement
`\d+ cagg_joins_upgrade_test_with_realtime_using` with
the error message `ERROR:  invalid memory alloc request size 13877216128`.

To unblock CI and allow other PRs to get merged we temporarily
skip the offending query on PG13.
2023-12-14 21:06:05 +01:00
Nikhil Sontakke
7d98613f28 Remove warning on MacOSx
The -Werror,-Wunused-but-set-variable parameters while compiling on
MacOSx were causing compilation to fail on CI. Fixed that now.
2023-12-14 18:04:43 +05:30
Sven Klemm
11df1dd648 Remove experimental multinode functions
This commit removes the following functions:
- timescaledb_experimental.block_new_chunks
- timescaledb_experimental.allow_new_chunks
- timescaledb_experimental.subscription_exec
- timescaledb_experimental.move_chunk
- timescaledb_experimental.copy_chunk
- timescaledb_experimental.cleanup_copy_chunk_operation
2023-12-13 23:38:32 +01:00
Sven Klemm
83cf5605a5 Remove multinode functionality from compression 2023-12-13 23:38:32 +01:00
Sven Klemm
8a2029f569 Remove rxid type and distributed size util functions 2023-12-13 23:38:32 +01:00
Sven Klemm
b1159eba68 Remove AsyncAppend custom node 2023-12-13 23:38:32 +01:00
Sven Klemm
dde86ac116 Remove process_utility hooks used by multinode 2023-12-13 23:38:32 +01:00
Sven Klemm
19f1395191 Remove internal multinode ddl functions
This commit removes the following functions:
- _timescaledb_functions.create_chunk_replica_table
- _timescaledb_functions.chunk_drop_replica
- _timescaledb_functions.wait_subscription_sync
- _timescaledb_functions.health
- _timescaledb_functions.drop_stale_chunks
2023-12-13 23:38:32 +01:00
Sven Klemm
a4f336fd7f Remove healthcheck code 2023-12-13 23:38:32 +01:00
Sven Klemm
5f4e6f3006 Remove multinode specific postgres configuration
Remove the multinode specific things from the postgres configuration
used for TSL tests.
2023-12-13 23:38:32 +01:00
Sven Klemm
3be4f21b0e Remove multinode from telemetry 2023-12-13 23:38:32 +01:00
Fabrízio de Royes Mello
78490c47b7 Remove support for creating CAggs with old format
Timescale 2.7 released a new version of Continuous Aggregate (#4269)
that store the final aggregation state instead of the byte array of
the partial aggregate state, offering multiple opportunities of
optimizations as well a more compact form.

In 2.10.0 released on February 2023 the Continuous Aggregate old format
deprecation was announced.

With this PR the ability of creating Continuous Aggregate in the old
format was removed, but we still support migrate from the old to the
new format by running the `cagg_migrate` procedure.

This is the continuation of the PR #5977 started by @pdipesh02.

References:
https://docs.timescale.com/api/latest/continuous-aggregates/cagg_migrate/
https://github.com/timescale/timescaledb/releases/tag/2.10.0
https://github.com/timescale/timescaledb/releases/tag/2.7.0
https://github.com/timescale/timescaledb/pull/5977
2023-12-13 18:48:31 -03:00
Nikhil Sontakke
13c4f08f16 Fix OpenSSL issues by using latest MacOSx
The latest OpenSSL 3.2.0 version has known issues with Postgres. MacOSX
CI runs were failing because of this. We now use macos-13 visavis the
earlier macos-11. That seems to solve this OpenSSL issue.
2023-12-13 21:33:05 +05:30
Sven Klemm
250c6c1dcd Speed up bgw scheduler tests
Speed up bgw tests by reducing the amount of time we wait in
pg_sleep.
2023-12-13 13:10:15 +01:00
Ante Kresic
0e2ff92ec7 Fix groupby pathkeys for gapfill in PG16
With PG16, group pathkeys can include column which are
not in the GROUP BY clause when dealing with ordered
aggregates. This means we have to exclude those columns
when checking and creating the gapfill custom scan subpath.

Fixes #6396
2023-12-13 12:47:39 +01:00
Sven Klemm
c914d19fac Remove the timescaledb_fdw foreign data wrapper
This is the fdw implementation that was used for communication
between multinode instances.
2023-12-13 09:48:03 +01:00
Mats Kindahl
4e17247e76 Add Coccinelle rule to use NameStr
Direct access of the `.data` member of `NameData` structures are
discuraged and `NameStr` should be used instead.

Also adding one instance that was missed in #5336.
2023-12-12 22:30:05 +01:00
Sven Klemm
36c36564a8 Refactor compression setting storage
This patch drops the catalog table _timescaledb_catalog.hypertable_compression
and stores those settings in _timescaledb_catalog.compression_settings instead.
The storage format is changed and the new table will have 1 entry per relation
instead of 1 entry per column and has no dependancy on hypertables.
All other aspects of compression will remain the same. This is refactoring is
to enable per chunk compression settings in a follow-up patch.
2023-12-12 21:45:33 +01:00
Sven Klemm
bc935ab2ca Remove multinode public API
This patch removes the following functions/procedures:
- add_data_node
- alter_data_node
- attach_data_node
- create_distributed_hypertable
- create_distributed_restore_point
- delete_data_node
- detach_data_node
- distributed_exec
- set_replication_factor
- _timescaledb_functions.ping_data_node
- _timescaledb_functions.remote_txn_heal_data_node
- _timescaledb_functions.set_dist_id
- _timescaledb_functions.set_peer_dist_id
- _timescaledb_functions.show_connection_cache
- _timescaledb_functions.validate_as_data_node
- _timescaledb_internal.ping_data_node
- _timescaledb_internal.remote_txn_heal_data_node
- _timescaledb_internal.set_dist_id
- _timescaledb_internal.set_peer_dist_id
- _timescaledb_internal.show_connection_cache
- _timescaledb_internal.validate_as_data_node
2023-12-12 20:37:35 +01:00
Sven Klemm
985c1d2beb fixup 2023-12-12 20:37:35 +01:00
Sven Klemm
5d3ab06295 Remove multinode from update/downgrade test 2023-12-12 20:37:35 +01:00
Fabrízio de Royes Mello
76fead4d1e Remove multinode cagg regression tests 2023-12-12 15:15:44 -03:00
Sven Klemm
320c6a8de5 Remove multinode dist_copy code 2023-12-12 14:48:28 +01:00
Sven Klemm
d84df25611 Remove DataNodeCopy and DataNodeDispatch custom node
This patch removes DataNodeCopy and DataNodeDispatch custom nodes which
were only used in multinode context.
2023-12-12 14:48:28 +01:00
Erik Nordström
e8b81c2ebe Use table scan API in compression code
Refactor the compression code to only use the table scan API when
scanning relations instead of a mix of the table and heap scan
APIs. The table scan API is a higher-level API and recommended as it
works for any type of relation and uses table slots directly, which
means that in some cases a full heap tuple need not be materialized.
2023-12-12 14:40:52 +01:00
Sven Klemm
0f60f88621 Remove multinode C test files
The previous multinode PR only removes the regression test,
isolation test and TAP test files but not the C files. This patch
also removes the deparse test which was missed in the previous
PR.
2023-12-12 09:40:42 +01:00
Sven Klemm
c5bed532ac Remove multinode tests
This tests removes all the multinode test files to remove the
burden to keep them update with PRs. The actual code will be removed
in followup PRs.
2023-12-11 16:37:55 +01:00
Sven Klemm
3151b0054d Pin Alpine version to 3.18
Pin Alpine version to 3.18 since 3.19 does not have openssl 1.1 at
all which we need for some of the ABI checks against older versions.
2023-12-11 15:05:06 +01:00
Sven Klemm
3ae2dbc2ef Allow disabling loader-change check in PRs
This PR adds capabilities to allow individual PRs to override the
loader-change check. Not all changes in the loader directories
will require a new loader version so the check can sometimes have
false positives. It also mentions to bump loader version for
loader changes.
2023-12-11 12:15:17 +01:00
Sven Klemm
887a39d7ef Remove unneeded multinode test files for PG16
Since multinode is not supported on PG16 none of these files is needed.
2023-12-11 10:31:35 +01:00
Sven Klemm
cfd65680cd Use #pragma once instead of include guards
We used a mixture of include guards and pragma once in our header
files. This patch changes our headers to always use pragma once
because it is less error prone, can be copy/pasted, doesnt require
a unique identifier and is also shorter.
2023-12-11 09:56:32 +01:00
Sven Klemm
e1a2c64743 Remove unused compression policy helper functions
This patch removes the following unused functions:
policy_compression_get_compress_after_int(const Jsonb *config)
policy_compression_get_compress_after_interval(const Jsonb *config)
policy_compression_get_compress_created_before_interval(const Jsonb *config)
2023-12-10 20:46:23 +01:00