This change adds a compiler-agnostic annotation for fall-throughs in
switch statements. The definition is put in `annotations.h`, which can
be expanded to hold definitions for similar functionality in the
future.
The `clang` compiler (as of version 12) seems to have have dropped
support for the previous comment-based annotations to allow
fall-throughs in favor of native annotations or GCC-style attributes.
It is incorrect to forward relcache invalidations as syscache
invalidations, like cacheid = InvalidOid which is not possible
condition at the moment. Allow syscache invalidations only for
FOREIGNSERVEROIDs.
This change refactors how connections are handled during remote
transactions. In particular, the connection cache now stays consistent
during transactions, even during rollbacks. Previously, the connection
cache was replaced on every rollback, even if the rollback was
intentional (i.e, not due to an error). This made it hard to debug
connections since the cache became completely empty.
Connections could also be left in the cache in a bad state after
failed transactions. This has been fixed by moving connection checks
to the cache and tying transaction state changes to each
connection. This ensures that such checks are done in one canonical
place instead of being spread out throughout the code.
Given how tightly coupled a remote transaction is with its connection,
it might make sense to remove the separate remote transaction store
and instead put this information in each connection. This is left to a
future change, however.
In addition to the above changes, this commit includes:
* Showing transaction depth and invalidation in the transaction store
* Invalidation on individual connections instead of replacing the
whole cache
* Closing of connections to a local database that is being dropped to
prevent "in use" errors.
* Ability to add callbacks to async requests that are executed when a
response is received. This is used by remote transactions to mark
connections as having successfully completed a transaction. Thus, on
errors, it is easy to detect connections that are in bad states.
* Error checks on each connection instead of having global error
tracking for each remote transaction. This change removes the global
error state for distributed transactions.
This change replace UserMappings with newly introduced TSConnectionId
object, which represent a pair of foreign server id and local user id.
Authentication has been moved to non-password based, since original
UserMappings were used to store a data node user passwords as
well. This is a temporary step, until introduction of certificate
based authentication.
List of changes:
* add_data_node() password and bootstrap_password arguments removed
* introduced authentication using pgpass file
* RemoteTxn format string which represents tx changed to
tx-version-xid-server_id-user_id
* data_node_dispatch, remote transaction cache, connection cache hash
tables keys switched to TSConnectionId instead of user mappings
* remote_connection_open() been rework to exclude user options
* Tests upgraded, user mappings and passwords usage has been excluded
This commit adds the ability to cache remote connections
across commands and transactions. This is needed since establishing
new connections is expensive. The cache is invalidated
when the foreign server or user mapping is changed. Because of this
the cache is tied to a user mapping (it is keyed by the user mapping's
oid and requires a user_mapping for invalidation).
We use the syscache invalidation mechanism since foreign servers and
user mappings are already invalidated using this mechanism. This
requires some extra plumbing in our cache invalidation handling.
This cache will be used in txn callback handling and so the regular
auto-release of caches on (sub)txn commits/aborts that happens
with most caches is inappropriate. Therefore we added a new flag
to the cache called `handle_txn_callbacks` that allows a cache
to turn off the auto-release mechanism
Future proofing: if we ever want to make our functions available to
others they’d need to be prefixed to prevent name collisions. In
order to avoid having some functions with the ts_ prefix and
others without, we’re adding the prefix to all non-static
functions now.
Previously, the scheduler only populated its jobs list once at start time. This commit enables the scheduler to receive notifications for updates (insert, update, delete) to the bgw_job table. Notifications are sent via the cache invalidation framework. Whenever the scheduler receives a notification, it re-reads the bgw_job table. For each job currently in the bgw_job table, it either instantiates new scheduler state for the job or copies over any existing scheduler state, for persisting jobs. For jobs that have disappeared from the bgw_job table, the scheduler deletes any local state it has.
Note that any updates to the bgw_job table must now go through the C, so that the cache invalidation framework in catalog.c can run. In particular, this commit includes a rudimentary API for interacting with the bgw_job table, for testing purposes. This API will be rewritten in the future.
We've decided to adopt the ts_ prefix on all exported C functions in
order to avoid having symbol conflicts with future postgres functions.
We've already started using this prefix on new functions and this commit
adds the prefix to to the old functions.
The cache invalidation triggers on our catalog tables
aren't used anymore as all modifications to catalog tables
happen using the C API, which doesn't invoke triggers and
has its own cache invalidation functionality.
Windows 64-bit binaries should now be buildable using the cmake
build system either from the command line or from Visual Studio.
Previous issues regarding unresolved symbols have been resolved
with compatibility header files to properly export symbols or
getting GUCs via normal APIs.
TimescaleDB cache invalidation happens as a side effect of doing a
full SQL statement (INSERT/UPDATE/DELETE) on a catalog table (via
table triggers). However, triggers aren't invoked when using
PostgreSQL's internal catalog API for updates, since PostgreSQL's
catalog tables don't have triggers that require full statement
parsing, planning, and execution.
Since we are now using the regular PostgreSQL catalog update API for
some TimescaleDB catalog operations, we need to do cache invalidation
also on such operations.
This change adds cache invalidation when updating catalogs using the
internal (C) API and also makes the cache invalidation more fine
grained. For instance, caches are no longer invalidated on some
INSERTS that do not affect the validity of objects already in the
cache, such as adding a new chunk.
For all exported functions the macro PGDLLEXPORT needs to be pre-
pended. Additionally, on Windows `open` is a macro that needed to
be renamed. A few other small changes are done to make Visual
Studio's compiler happy / get rid of warnings (e.g. adding return
statements after elog).
This PR add support for primary-key, foreign-key, unique, and exclusion constraints.
Previously supported are CHECK and NOT NULL constraints. Now, foreign key
constraints where a hypertable references a plain table is support
(while vice versa, with a plain table references a hypertable, is still not).
Previously, the extension could end up in a bad state if it
was dropped as part of a cascade. This PR fixes that by
checking explicitly for the presence of the proxy table
to make sure we are not in the middle of an extension
drop. Fixes#73.
Sometimes postgres wants to invalidate the entire cache. In this case it
send a cache invalidation message with relid == InvalidOid. This should
invalidate all the caches. Previously, it did not effect the cache of
the extension state. This fixes that problem and creates one cache reset
code path for both extension dropping and whole-cache reset, cleaning
up some of the logic in the process.
DROP EXTENSION didn't properly reset caches and other saved state
causing various errors related to bad state when the extension was
dropped and/or recreated later.
This patch adds functionality to track the state of the extension and
also signals DROP EXTENSION to other backends that might be running,
allowing them to reset their internal extension state.
This patch refactors the source code so that a bunch of unrelated code
for the planner, process utilities and transaction management, which
was previously located in the common file timescaledb.c, is now broken
up into separate source files.
Previously, chunk replicas were retreived with an SPI query. Now, all
catalog items are retrieved with native scans, with the exception of
newly created chunks.
This commit also refactors the chunk (replica) cache, removing some
data structures that were duplicating information. Now chunks are
cached by their ID (including their replicas) instead of just the
set of replicas. This removes the need for additional data structures,
such as the replica set which looked like a chunk minus time info,
and the cache entry wrapper struct. Another upside is that chunks
can be retrieved from the cache directly by ID.
