Add support for dumping and restoring hypertables that have chunks that use the Hypercore TAM. Dumping a Hypercore table requires special consideration because its data is internally stored in two separate relations: one for compressed data and one for non-compressed data. The TAM returns data from both relations, but they may be dumped as separate tables. This risks dumping the compressed data twice: once via the TAM and once via the compressed table in compressed format. The `pg_dump` tool uses `COPY TO` to create dumps of each table, and, to avoid data duplication when used on Hypercore tables, this change introduces a GUC that allows selecting one of these two behaviors: 1. A `COPY TO` on a Hypercore table returns all data via the TAM, including data stored in the compressed relation. A `COPY TO` on the internal compressed relation returns no data. 2. A `COPY TO` on a Hypercore returns only non-compressed data, while a `COPY TO` on the compressed relation returns compressed data. A `SELECT` still returns all the data as normal. The second approach is the default because it is consistent with compression when Hypercore TAM is not used. It will produce a `pg_dump` archive that includes data in compressed form (if data was compressed when dumped). Conversely, option (1) will produce an archive that looks identical to a dump from an non-compressed table. There are pros and cons of each dump format. A non-compressed archive is a platform-agnostic logical dump that can be restored to any platform and architecture, while a compressed archive includes data that is compressed in a platform-dependent way and needs to be restored to a compatible system. A test is added that tests both these settings and corresponding dumping and restoring.
Linux/macOS | Linux i386 | Windows | Coverity | Code Coverage | OpenSSF |
---|---|---|---|---|---|
TimescaleDB
TimescaleDB is an open-source database designed to make SQL scalable for time-series data. It is engineered up from PostgreSQL and packaged as a PostgreSQL extension, providing automatic partitioning across time and space (partitioning key), as well as full SQL support.
If you prefer not to install or administer your instance of TimescaleDB, try the 30 day free trial of Timescale Cloud, our fully managed cloud offering. Timescale is pay-as-you-go. We don't charge for storage you dont use, backups, snapshots, ingress or egress.
To determine which option is best for you, see Timescale Products for more information about our Apache-2 version, TimescaleDB Community (self-hosted), and Timescale Cloud (hosted), including: feature comparisons, FAQ, documentation, and support.
Below is an introduction to TimescaleDB. For more information, please check out these other resources:
- Developer Documentation
- Slack Channel
- Timescale Community Forum
- Timescale Release Notes & Future Plans
For reference and clarity, all code files in this repository reference
licensing in their header (either the Apache-2-open-source license
or Timescale License (TSL)
). Apache-2 licensed binaries can be built by passing -DAPACHE_ONLY=1
to bootstrap
.
(To build TimescaleDB from source, see instructions in Building from source.)
Using TimescaleDB
TimescaleDB scales PostgreSQL for time-series data via automatic partitioning across time and space (partitioning key), yet retains the standard PostgreSQL interface.
In other words, TimescaleDB exposes what look like regular tables, but are actually only an abstraction (or a virtual view) of many individual tables comprising the actual data. This single-table view, which we call a hypertable, is comprised of many chunks, which are created by partitioning the hypertable's data in either one or two dimensions: by a time interval, and by an (optional) "partition key" such as device id, location, user id, etc.
Virtually all user interactions with TimescaleDB are with hypertables. Creating tables and indexes, altering tables, inserting data, selecting data, etc., can (and should) all be executed on the hypertable.
From the perspective of both use and management, TimescaleDB just looks and feels like PostgreSQL, and can be managed and queried as such.
Before you start
PostgreSQL's out-of-the-box settings are typically too conservative for modern
servers and TimescaleDB. You should make sure your postgresql.conf
settings are tuned, either by using timescaledb-tune
or doing it manually.
Creating a hypertable
-- Do not forget to create timescaledb extension
CREATE EXTENSION timescaledb;
-- We start by creating a regular SQL table
CREATE TABLE conditions (
time TIMESTAMPTZ NOT NULL,
location TEXT NOT NULL,
temperature DOUBLE PRECISION NULL,
humidity DOUBLE PRECISION NULL
);
-- Then we convert it into a hypertable that is partitioned by time
SELECT create_hypertable('conditions', 'time');
Inserting and querying data
Inserting data into the hypertable is done via normal SQL commands:
INSERT INTO conditions(time, location, temperature, humidity)
VALUES (NOW(), 'office', 70.0, 50.0);
SELECT * FROM conditions ORDER BY time DESC LIMIT 100;
SELECT time_bucket('15 minutes', time) AS fifteen_min,
location, COUNT(*),
MAX(temperature) AS max_temp,
MAX(humidity) AS max_hum
FROM conditions
WHERE time > NOW() - interval '3 hours'
GROUP BY fifteen_min, location
ORDER BY fifteen_min DESC, max_temp DESC;
In addition, TimescaleDB includes additional functions for time-series
analysis that are not present in vanilla PostgreSQL. (For example, the time_bucket
function above.)
Installation
Installation options are:
-
Timescale Cloud: A fully-managed TimescaleDB in the cloud, is available via a free trial. Create a PostgreSQL database in the cloud with TimescaleDB pre-installed so you can power your application with TimescaleDB without the management overhead.
-
Platform packages: TimescaleDB is also available pre-packaged for several platforms such as Linux, Windows, MacOS, Docker, and Kubernetes. For more information, see Install TimescaleDB.
-
Build from source: See Building from source.
We recommend not using TimescaleDB with PostgreSQL 17.1, 16.5, 15.9, 14.14, 13.17, 12.21.
These minor versions introduced a breaking binary interface change that, once identified, was reverted in subsequent minor PostgreSQL versions 17.2, 16.6, 15.10, 14.15, 13.18, and 12.22. When you build from source, best practice is to build with PostgreSQL 17.2, 16.6, etc and higher. Users of Timescale Cloud and Platform packages built and distributed by Timescale are unaffected.
Resources
Architecture documents
Useful tools
- timescaledb-tune: Helps set your PostgreSQL configuration settings based on your system's resources.
- timescaledb-parallel-copy:
Parallelize your initial bulk loading by using PostgreSQL's
COPY
across multiple workers.
Additional documentation
- Why use TimescaleDB?
- Migrating from PostgreSQL
- Writing data
- Querying and data analytics
- Tutorials and sample data
Community & help
- Slack Channel
- Github Issues
- Timescale Support: see support options (community & subscription)
Releases & updates
- Timescale Release Notes: see detailed information about current and past versions and subscribe to get notified about new releases, fixes, and early access/beta programs.