The tests do not expect that autovacuum is run. The most of the tests
finishes before it runs, but some ARM tests might fail due if autovacuum
is executed.
Adds test infrastructure for deleting bgw jobs. We do this
using isolation tests. We add infrastructure for running bgws
in isolation tests. We also add infrastructure to control which
user runs the bgw scheduler and jobs during the tests. We
test the bgw delete as non-superuser.
Add all entries from _timescaledb_catalog.installation_metadata
to the telemetry report under the key `db_metadata`.
Add new GUC timescaledb_telemetry.cloud which is sent in telemetry
under instance_metadata.cloud when set.
Set random_page_cost to 1.0 to produce more reasonable plans.
This is similar to what timescaledb-tune would set.
This patch also adds test/pgtest.conf which is only used as
postgresql.conf for running the PostgreSQL test suite, because
the random_page_cost change lead to test failures in the PostgreSQL
test suite.
Reloading shared libraries (which is done to start parallel
workers) is done is the order in which load_file completes.
If we try to load the TSL from PG_init, this causes postgres to
attempt to load the TSL before timescale itself, which causes an
error.
Also disables the warning on normal TSL startup.
This commit adds support for dynamically loaded submodules to timescaledb
as well an initial license-key implementation in the tsl subdirectory.
Dynamically loaded modules allow our users to determine which licenses they
wish to use for their version of timescaledb; if they wish to only use
Apache-Licensed code, they do not load the Timescale-Licensed submodule. Calls
from the Apache-Licensed code into the Timescale-Licensed submodule are
handled via dynamicaly-set function pointers; see tsl/src/Readme.module.md for
more details.
This commit also adds code for license keys for the ApacheOnly, Community, and
Enterprise editions. The license key determines which features are enabled,
and controls loading the submodule: when a license key that requires the
sub-module is installed, the module is automatically loaded.
Currently the ApacheOnly and Community license-keys are hardcoded to be
"ApacheOnly" and "Community" respectively. The first version of the enterprise
license-key is described in tsl/src/Readme.module.md
This adds the telemetry job to the job scheduler. Telemetry is
scheduled to run every 24 hours with a 1 hour exponential backoff
retry period. Additional fixes related to the telemetry job:
- Add separate ID space to the bgw_job table for default jobs. We do not dump this ID space inside pg_dump to prevent job insertion conflicts.
- Add check to update scripts for default jobs.
- Change shmem_callback so that it doesn't modify state since state transitions are not atomic with respect to interrupts and interrupt callbacks.
- Disable default telemetry job in regression and docker tests.
TimescaleDB will want to run multiple background jobs. This PR
adds a simple scheduler so that jobs inserted into a jobs table
could be run on a schedule. This first implementation has two limitations:
1) The list of jobs to be run is read from the database when the scheduler
is first started. We do not update this list if the jobs table changes.
2) There is no prioritization for when to run jobs.
There design of the scheduler is as follows:
The scheduler itself is a background job that continuously runs and waits
for a time when jobs need to be scheduled. It then launches jobs as new
background workers that it controls through the background worker handle.
Aggregate statistics about a job are kept in the job_stat catalog table.
These statistics include the start and finish times of the last run of the job
as well as whether or not the job succeeded. The next_start is used to
figure out when next to run a job after a scheduler is restarted.
The statistics table also tracks consecutive failures and crashes for the job
which is used for calculating the exponential backoff after a crash or failure
(which is used to set the next_start after the crash/failure). Note also that
there is a minimum time after the db scheduler starts up and a crashed job
is restarted. This is to allow the operator enough time to disable the job
if needed.
Note that the number of crashes is an overestimate of the actual number of crashes
for a job. This is so that we are conservative and never miss a crash and fail to
use the appropriate backoff logic. Note that there is some complexity
in ensuring that all crashes are counted since a crash in Postgres causes /all/
processes to SIGQUIT: we must commit changes to the stats
table /before/ a job starts so that we can then deduce after a job has crashed
and the scheduler comes back up that a job was started, and not finished before
the crash (meaning that it could have been the crashing process).
Moving the build system to CMake allows easy cross-platform
compiles, dependency checks, and more. In particular, CMake
allows us to easily build on Windows, and Visual Studio now
has native CMake support.
Tests are now run on a temporary Postgres instance, which is launched
by the `pg_regress` test runner. This allows running tests without
having an existing running instance with a matching configuration and
also obviates the need for preloading the TimescaleDB extension. As a
result, tests are simpler to setup and run, and are more reliable and
consistent.
