Alex Kasko d36d34a244 Improve transaction check in CAgg refresh
Procedures that use multiple transactions cannot be run in a transaction
block (from a function, from dynamic SQL) or in a subtransaction (from a
procedure block with an EXCEPTION clause). Such procedures use
PreventInTransactionBlock function to check whether they can be run.

Though currently such checks are incompete, because
PreventInTransactionBlock requires isTopLevel argument to throw a
consistent error when the call originates from a function. This
isTopLevel flag (that is a bit poorly named - see below) is not readily
available inside C procedures. The source of truth for it -
ProcessUtilityContext parameter is passed to ProcessUtility hooks, but
is not included with the function calls. There is an undocumented
SPI_inside_nonatomic_context function, that would have been sufficient
for isTopLevel flag, but it currently returns false when SPI connection
is absent (that is a valid scenario when C procedures are called from
top-lelev SQL instead of PLPG procedures or DO blocks) so it cannot be
used.

To work around this the value of ProcessUtilityContext parameter is
saved when TS ProcessUtility hook is entered and can be accessed from
C procedures using new ts_process_utility_is_context_nonatomic function.
The result is called "non-atomic" instead of "top-level" because the way
how isTopLevel flag is determined from the ProcessUtilityContext value
in standard_ProcessUtility is insufficient for C procedures - it
excludes PROCESS_UTILITY_QUERY_NONATOMIC value (used when called from
PLPG procedure without an EXCEPTION clause) that is a valid use case for
C procedures with transactions. See details in the description of
ExecuteCallStmt function.

It is expected that calls to C procedures are done with CALL and always
pass though the ProcessUtility hook. The ProcessUtilityContext
parameter is set to PROCESS_UTILITY_TOPLEVEL value by default. In
unlikely case when a C procedure is called without passing through
ProcessUtility hook and the call is done in atomic context, then
PreventInTransactionBlock checks will pass, but SPI_commit will fail
when checking that all current active snapshots are portal-owned
snapshots (the same behaviour that was observed before this change).
In atomic context there will be an additional snapshot set in
_SPI_execute_plan, see the snapshot handling invariants description
in that function.

Closes #6533.

(cherry picked from commit 85437c0b2f25d9cab0d5c263f248a6c59572afb6)
2025-01-15 15:38:27 +00:00
2024-11-21 17:59:20 -03:00
2024-06-03 18:13:51 -04:00
2024-10-07 18:10:34 -03:00

Linux/macOS Linux i386 Windows Coverity Code Coverage OpenSSF
Build Status Linux/macOS Build Status Linux i386 Windows build status Coverity Scan Build Status Code Coverage OpenSSF Best Practices

TimescaleDB

TimescaleDB is an open-source database designed to make SQL scalable for time-series data. It is engineered up from PostgreSQL and packaged as a PostgreSQL extension, providing automatic partitioning across time and space (partitioning key), as well as full SQL support.

If you prefer not to install or administer your instance of TimescaleDB, try the 30 day free trial of Timescale, our fully managed cloud offering. Timescale is pay-as-you-go. We don't charge for storage you dont use, backups, snapshots, ingress or egress.

To determine which option is best for you, see Timescale Products for more information about our Apache-2 version, TimescaleDB Community (self-hosted), and Timescale Cloud (hosted), including: feature comparisons, FAQ, documentation, and support.

Below is an introduction to TimescaleDB. For more information, please check out these other resources:

For reference and clarity, all code files in this repository reference licensing in their header (either the Apache-2-open-source license or Timescale License (TSL) ). Apache-2 licensed binaries can be built by passing -DAPACHE_ONLY=1 to bootstrap.

Contributors welcome.

(To build TimescaleDB from source, see instructions in Building from source.)

Using TimescaleDB

TimescaleDB scales PostgreSQL for time-series data via automatic partitioning across time and space (partitioning key), yet retains the standard PostgreSQL interface.

In other words, TimescaleDB exposes what look like regular tables, but are actually only an abstraction (or a virtual view) of many individual tables comprising the actual data. This single-table view, which we call a hypertable, is comprised of many chunks, which are created by partitioning the hypertable's data in either one or two dimensions: by a time interval, and by an (optional) "partition key" such as device id, location, user id, etc.

Virtually all user interactions with TimescaleDB are with hypertables. Creating tables and indexes, altering tables, inserting data, selecting data, etc., can (and should) all be executed on the hypertable.

From the perspective of both use and management, TimescaleDB just looks and feels like PostgreSQL, and can be managed and queried as such.

Before you start

PostgreSQL's out-of-the-box settings are typically too conservative for modern servers and TimescaleDB. You should make sure your postgresql.conf settings are tuned, either by using timescaledb-tune or doing it manually.

Creating a hypertable

-- Do not forget to create timescaledb extension
CREATE EXTENSION timescaledb;

-- We start by creating a regular SQL table
CREATE TABLE conditions (
  time        TIMESTAMPTZ       NOT NULL,
  location    TEXT              NOT NULL,
  temperature DOUBLE PRECISION  NULL,
  humidity    DOUBLE PRECISION  NULL
);

-- Then we convert it into a hypertable that is partitioned by time
SELECT create_hypertable('conditions', 'time');

Inserting and querying data

Inserting data into the hypertable is done via normal SQL commands:

INSERT INTO conditions(time, location, temperature, humidity)
  VALUES (NOW(), 'office', 70.0, 50.0);

SELECT * FROM conditions ORDER BY time DESC LIMIT 100;

SELECT time_bucket('15 minutes', time) AS fifteen_min,
    location, COUNT(*),
    MAX(temperature) AS max_temp,
    MAX(humidity) AS max_hum
  FROM conditions
  WHERE time > NOW() - interval '3 hours'
  GROUP BY fifteen_min, location
  ORDER BY fifteen_min DESC, max_temp DESC;

In addition, TimescaleDB includes additional functions for time-series analysis that are not present in vanilla PostgreSQL. (For example, the time_bucket function above.)

Installation

Timescale, a fully managed TimescaleDB in the cloud, is available via a free trial. Create a PostgreSQL database in the cloud with TimescaleDB pre-installed so you can power your application with TimescaleDB without the management overhead.

TimescaleDB is also available pre-packaged for several platforms such as Linux, Windows, MacOS, Docker, and Kubernetes. For more information, see Install TimescaleDB.

To build from source, see Building from source.

Resources

Architecture documents

Useful tools

  • timescaledb-tune: Helps set your PostgreSQL configuration settings based on your system's resources.
  • timescaledb-parallel-copy: Parallelize your initial bulk loading by using PostgreSQL's COPY across multiple workers.

Additional documentation

Community & help

Releases & updates

  • Timescale Release Notes: see detailed information about current and past versions and subscribe to get notified about new releases, fixes, and early access/beta programs.

Contributing

Description
An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
Readme 77 MiB
Languages
C 68.9%
PLpgSQL 24.4%
CMake 1.9%
Ruby 1.6%
Python 1.4%
Other 1.8%