3780 Commits

Author SHA1 Message Date
Konstantina Skovola
014b7b9664 Move .gitattributes to root and remove build-13
Commit effc8efe148c4ec0048bd7c1dfe0ca01df2afdc9
accidentally placed .gitattributes inside a build-13 directory
instead of the root of the project. This commit removes build-13 and
fixes the structure.
2023-01-20 10:21:11 +02:00
Jan Nidzwetzki
28dbeaa2ca Add cost estimation for remote joins
This patch adds the missing functionality to estimate the costs of
remote joins. Most of the code is a backport from PG Upstream.
2023-01-19 15:52:37 +01:00
Konstantina Skovola
effc8efe14 Add workflow to check for CHANGELOG update 2023-01-19 13:04:59 +02:00
Erik Nordström
1676259840 Fix repartition behavior when attaching data node
When attaching a data node and specifying `repartition=>false`, the
current number of partitions should remain instead of recalculating
the partitioning based on the number of data nodes.

Fixes #5157
2023-01-18 16:20:49 +01:00
Jan Nidzwetzki
19065bbdf3 Introduce a FDW option to mark reference tables
With this patch, the ability to mark reference tables (tables that exist
on all data nodes of a multi-node installation) via an FDW option has
been added.
2023-01-18 15:11:02 +01:00
Bharathy
9a2cbe30a1 Fix ChunkAppend, ConstraintAwareAppend child subplan
When TidRangeScan is child of ChunkAppend or ConstraintAwareAppend node, an
error is reported as "invalid child of chunk append: Node (26)". This patch
fixes the issue by recognising TidRangeScan as a valid child.

Fixes: #4872
2023-01-18 18:06:30 +05:30
Mats Kindahl
8f4fa8e4cc Add build matrix to Windows and Linux builds
Build matrix is missing from the ignore workflows for the Windows and
Linux builds, so this commit adds them.
2023-01-18 09:41:11 +01:00
Alexander Kuzmenkov
6aa3d6e2fc Don't run the APT package test on PRs
We run them nightly on main.
2023-01-17 14:57:17 +04:00
Jan Nidzwetzki
1d3d81491b Update issue workflow actions and used token
This patch updates the used issue workflow actions and ensures that the
'secrets.ORG_AUTOMATION_TOKEN' is used by all actions.
2023-01-17 11:47:22 +01:00
Dmitry Simonenko
5c897ff75d Fix default data node availability status
Function alter_data_node() return uninitialized value for "available"
option when it is not presented in the option list.

Fix #5154
2023-01-16 16:33:10 +02:00
Sven Klemm
dbe89644b5 Remove no longer used compression code
The recent refactoring of INSERT into compression chunk made this
code obsolete but forgot to remove it in that patch.
2023-01-16 14:18:56 +01:00
Fabrízio de Royes Mello
73df496c75 Fix CAgg on CAgg variable bucket size validation
Previous attempt to fix it (PR #5130) was not entirely correct because
the bucket width calculation for interval width was wrong.

Fixed it by properly calculate the bucket width for intervals using the
Postgres internal function `interval_part` to extract the epoch of the
interval and execute the validations. For integer widths use the already
calculated bucket width.

Fixes #5158, #5168
2023-01-13 12:38:58 -03:00
Mats Kindahl
ca9d508ede Add missing ignore files
The workflow ignore files for 32-bit Linux builds and Windows was
missing from #5156 so these are added here.
2023-01-13 14:02:27 +01:00
Erik Nordström
1e7b9bc558 Fix issue with deleting data node and dropping database
When deleting a data node with the option `drop_database=>true`, the
database is deleted even if the command fails.

Fix this behavior by dropping the remote database at the end of the
delete data node operation so that other checks fail first.
2023-01-13 13:54:27 +01:00
Fabrízio de Royes Mello
4118a72575 Remove parallel safe from partialize_agg
Previous PR #4307 mark `partialize_agg` and `finalize_agg` as parallel
safe but this change is leading to incorrect results in some cases.

Those functions are supposed work in parallel but seems is not the case
and it is not evident yet the root cause and how to properly use it in
parallel queries so we decided to revert this change and provide correct
results to users.

Fixes #4922
2023-01-13 07:31:55 -03:00
Mats Kindahl
f36db10826 Do not run regress workflows on benign changes
If only documentation is changed, the full regression check workflow
will still be executed, so this commit will instead skip running the
regression workflows if there are only changes to files that will not
affect the success of the workflow.
2023-01-11 22:42:32 +01:00
Jan Nidzwetzki
396bc6def7 Add join functionality to the deparser code
This patch adds the missing functionality to handle joins to the
deparser. Most of the code is a backport from PG Upstream.
2023-01-10 16:55:51 +01:00
Sven Klemm
cfd34f2752 Restructure chunk_dispatch
This patch adjusts the code layout for chunk_dispatch to be similar
to the other custom nodes. All the files related to chunk_dispatch
are moved into dedicated nodes/chunk_dispatch directory.
2023-01-10 16:54:43 +01:00
Lakshmi Narayanan Sreethar
06eca172bd Fix telemetry_stats test failure in PG15
The telemetry_stats testcase uses random() with seed(1) to generate the
column values on which the hypertable is partitioned. The Postgres commit
postgres/postgres@3804539e48 updates the random() implementation to use a
better algorithim causing the test to generate a different set of rows in
PG15. Due to this the test failed in PG15 as the distrubution stats of
the tuples have now changed. Fixed that by creating separate test
outputs for PG15 and other releases.

Fixes #5037
2023-01-10 21:04:49 +05:30
Lakshmi Narayanan Sreethar
7d3d260afb Skip auto assigning reviewers for draft PRs
Reviewers should be assigned only when the PR is ready for review.
2023-01-10 15:34:57 +05:30
Sven Klemm
7a6101a441 Bump pgspot version used in CI 2023-01-09 11:32:53 +01:00
Fabrízio de Royes Mello
cd48553de5 Fix CAgg on CAgg variable bucket size validation
During the creation of a CAgg on top of another CAgg we check if the
bucket width is multiple of the parent and for this arithmetic we made
an assumption that picking just the `month` part of the `interval` for
variable bucket size was enought.

This assumption was wrong for bucket sizes that doesn't have the month
part (i.e: using timezones) leading to division by zero error because in
that case the `month` part of an `interval` is equal to 0 (zero).

Fixed it by properly calculating the bucket width for variable sized
buckets.

Fixes #5126
2023-01-06 16:39:04 -03:00
Fabrízio de Royes Mello
73c97524a0 Fix CAgg on CAgg different column order
Creating a CAgg on a CAgg where the time column is in a different
order of the original hypertable was raising the following exception:

`ERROR:  time bucket function must reference a hypertable dimension
column`

The problem was during the validation we're initializing internal data
structure with the wrong hypertable metadata.

Fixes #5131
2023-01-06 13:04:06 -03:00
Fabrízio de Royes Mello
41d6a1f142 Fix adding column with NULL constraint
Adding new column with NULL constraint to a compressed hypertable is
raising an error but it make no sense because NULL constraints in
Postgres does nothing, I mean it is useless and exist just for
compatibility with other database systems:
https://www.postgresql.org/docs/current/ddl-constraints.html#id-1.5.4.6.6

Fixed it by ignoring the NULL constraints when we check for `ALTER TABLE
.. ADD COLUMN ..` to a compressed hypertable.

Fixes #5151
2023-01-06 11:15:12 -03:00
Jan Nidzwetzki
1751efbaea Improve the PR workflow actions
This patch includes two changes to the PR handling workflow:

(1) It changes the trigger for the workflow to pull_request_target. So,
    PRs can now also be assigned to reviewers if the PR is opened from
    external sources.

(2) A workflow has been added that automatically assigns PRs to the
    author.
2023-01-04 11:11:01 +01:00
Alexander Kuzmenkov
2c70dc622e Run all the tests on every commit in main
If we test every commit in master, we can allow GitHub to merge the PRs
automatically without requiring a manual rebase on the current master.
These rebases are a real time sink.
2023-01-03 16:12:46 +04:00
Sven Klemm
64e8e31c6e Improve ASSERT_IS_VALID_CHUNK macro
Currently when ASSERT_IS_VALID_CHUNK fails it is impossible to tell
which of the conditions fails without opening the coredump in debugger
as all the conditions are ANDed in a single Assert. This patch splits
the conditions into individual Asserts so you can immediately see
from stacktrace which condition failed.
2023-01-02 23:14:40 +01:00
Bharathy
3a8d294d58 SELECT from partial compressed chunks crashes
SELECT from partially compressed chunk crashes due to reference to NULL
pointer. When generating paths for DecompressChunk, uncompressed_partial_path
is null which is not checked, thus causing a crash. This patch checks for NULL
before calling create_append_path().

Fixes #5134
2023-01-02 20:40:30 +05:30
Sven Klemm
1eacb35107 Show postgres log in apt package test 2022-12-27 10:13:35 +01:00
Sven Klemm
b92f36d765 Add 2.9.1 to update test scripts 2022-12-27 09:24:57 +01:00
Sven Klemm
93667df7d8 Release 2.9.1
This release contains bug fixes since the 2.9.0 release.
This release is high priority for upgrade. We strongly recommend that you
upgrade as soon as possible.

**Bugfixes**
* #5072 Fix CAgg on CAgg bucket size validation
* #5101 Fix enabling compression on caggs with renamed columns
* #5106 Fix building against PG15 on Windows
* #5117 Fix postgres server restart on background worker exit
* #5121 Fix privileges for job_errors in update script
2022-12-23 14:38:45 +01:00
Konstantina Skovola
0a3615fc70 Fix privileges for job_errors table in update script 2022-12-23 14:05:19 +02:00
Konstantina Skovola
cdf8676689 Fix postgres server restart on background worker exit
Fixed by removing the croak signal handler that was introduced
to enable the silent exit of the telemetry job.
2022-12-23 11:50:59 +02:00
Sven Klemm
ca609696d6 Fix Windows package test 2022-12-22 15:40:39 +01:00
Sven Klemm
b1314e63f2 Fix RPM package test for PG15 on centos 7
Installing PG15 on Centos 7 requires the EPEL repository to satisfy
the dependencies.
2022-12-22 13:30:06 +01:00
Sven Klemm
4527f51e7c Refactor INSERT into compressed chunks
This patch changes INSERTs into compressed chunks to no longer
be immediately compressed but stored in the uncompressed chunk
instead and later merged with the compressed chunk by a separate
job.

This greatly simplifies the INSERT-codepath as we no longer have
to rewrite the target of INSERTs and on-the-fly compress leading
to a roughly 2x improvement on INSERT rate into compressed chunk.
Additionally this improves TRIGGER-support for INSERTs into
compressed chunks.

This is a necessary refactoring to allow UPSERT/UPDATE/DELETE on
compressed chunks in follow-patches.
2022-12-21 12:53:29 +01:00
Sven Klemm
bd20afccbb Add test for PG15 windows package 2022-12-21 10:44:07 +01:00
Sven Klemm
1d51672336 Skip package downgrade test for PG15
Since we currently only have 1 released version that supports PG15
we cannot test downgrade with PG15.
2022-12-20 20:19:15 +01:00
Lakshmi Narayanan Sreethar
a4806827dc Enable PG15 in various CI checks
This commit enables PG15 in the following workflows:
 - Regression Linux
 - ABI test
 - Memory tests
 - Coverity
 - SQLSmith
 - Additional cron tests

Co-authored-by: Bharathy Satish <bharathy@timescale.com>
2022-12-20 23:39:47 +05:30
Fabrízio de Royes Mello
024b1e1f30 Fix CAgg on CAgg bucket size validation
The bucket size of a Continuous Aggregate should be greater or equal to
the parent Continuous Aggregate because there are many cases where you
actually want to roll up on another dimension.
2022-12-20 14:03:45 -03:00
Bharathy
c5e496a554 Fix column ordering in compressed table index.
When defining compression segment by parameter using multiple columns,
the parameter ordering is not respected for index creation.

This patch fixes the issue by maintaining the same order in which user
has defined columns in segment by clause.

Fixes #5104
2022-12-20 21:11:35 +05:30
Sven Klemm
7d1b74a8c6 Use rand() instead of random()
Use rand() instead of random() cause the latter is not available
on Windows and postgres stopped backporting it with PG15.
Ideally we switch to the crypto functions added in PG15 but that
requires a bit more work and this is the minimal change required
to get it to build against PG15 on Windows.
2022-12-20 15:12:24 +01:00
Sven Klemm
b74f563a33 Run regression tests on pg15 windows 2022-12-20 15:12:24 +01:00
Sven Klemm
08bb21f7e6 2.9.0 Post-release adjustments
Add 2.9.0 to update test scripts and adjust downgrade scripts for
2.9.0. Additionally adjust CHANGELOG to sync with the actual release
CHANGELOG and add PG15 to CI.
2022-12-19 19:10:24 +01:00
Alexander Kuzmenkov
27310470be Allow AsyncAppend under IncrementalSort
We forgot to add a case for it.
2022-12-19 21:16:14 +04:00
Sven Klemm
c0e9bb4a30 Fix enabling compression on caggs with renamed columns
On caggs with realtime aggregation changing the column name does
not update all the column aliases inside the view metadata.
This patch changes the code that creates the compression
configuration for caggs to get the column name from the materialization
hypertable instead of the view internals.

Fixes #5100
2022-12-19 10:46:37 +01:00
Fabrízio de Royes Mello
317f8f1a99 Refactor CAggs on CAggs regression tests
When CAggs on CAggs was introduced in commit 3749953 the regression
tests was splited into 6 (six) different test suites.

Simplyfied it grouping tests and reduced to just 2 (two) different test
suites. It save resources and time because each suite test spawn it own
Postgres instance.
2022-12-17 17:19:43 -03:00
Fabrízio de Royes Mello
854b677749 Fix broken postgres install on Windows CI
The last minor versions for PG14 (14.6) and PG15 (15.1) were unlisted by
chocolatey maintainers due to some issues.

Fixed it by hardcoding the 14.5 until the packages become available
again.
2022-12-17 12:10:27 -03:00
Fabrízio de Royes Mello
4694c7d798 Refactor CAgg migrate regression tests
When CAggs migration was introduced in commit e34218ce the regression
tests was splited into 6 (six) different test suites.

Simplyfied it grouping tests and reduced to just 2 (two) different test
suites. It save resources and time because each suite test spawn it own
Postgres instance.
2022-12-16 14:39:44 -03:00
Mats Kindahl
e5843dddd3 Link development and design documentation
The tree contains a lot of design and architecture documents, but they
are not linked together, so this commits adds a few additional README
and build a basic structure for the documentation.
2022-12-15 21:35:36 +01:00