This commit changes the code to only perform cis switch actions
when the cis changes instead of on every tuple. Several recent
bug fix changes have increase the amount of work that needs to be
done in switch actions and so this commit is a performance improvement
that avoids doing unnecessary work.
Previously, the slots for upserts were only updated when
a ChunkInsertState was created, but not when it was found in the
ChunkDispatch cache. This was wrong -- we need to update the slots
every time we switch to the chunk.
Right now the switch function is called on every tuple. We will
optimize that later to only be called when the previous chunk was
different. But, that is not part of this bug-fix PR.
Future proofing: if we ever want to make our functions available to
others they’d need to be prefixed to prevent name collisions. In
order to avoid having some functions with the ts_ prefix and
others without, we’re adding the prefix to all non-static
functions now.
Previously, the chunk dispatch node used arbiter index and other
information from the top-level Query node. This did not work with CTEs
(and probably subqueries) because the info for this insert was not at
the top-level. This commit changes the logic to use that same info from the
parent ModifyTable node (and ModifyTableState).
This commit also changes the logic to translate the arbiter indexes
from the hypertable index to the chunk index directly using our catalog
tables instead of re-inferring the index on the chunk.
We add better accounting for number of items stored in a subspace
to allow better pruning. Instead of pruning based on the number of
dimension_slices in subsequent dimensions we now track number of total
items in the subspace store and prune based on that.
We add two GUC variables:
1) max_open_chunks_per_insert (default work_mem in bytes / 512. This
assumes an entry is 512 bytes)
2) max_cached_chunks_per_hypertable (default 100). Maximum cached chunks per
hypertable.
Previously, the cache in chunk_dispatch was limited to only hold
the chunk_insert_state for the last time dimension as a consequence
of logic in subspace_store. This has now been relaxed so that a
chunk_dispatch holds the cache for any chunk_insert_states that it
encounters. Logic for the hypertable chunk cache has not been changed.
The rule that we should follow is to limit the subspace store size for
caches that survive across commands. But caches within commands can be
allowed to grow.
The extension now works with PostgreSQL 10, while
retaining compatibility with version 9.6.
PostgreSQL 10 has numerous internal changes to functions and
APIs, which necessitates various glue code and compatibility
wrappers to seamlessly retain backwards compatiblity with older
versions.
Test output might also differ between versions. In particular,
the psql client generates version-specific output with `\d` and
EXPLAINs might differ due to new query optimizations. The test
suite has been modified as follows to handle these issues. First,
tests now use version-independent functions to query system
catalogs instead of using `\d`. Second, changes have been made to
the test suite to be able to verify some test outputs against
version-dependent reference files.
When new chunks are created, the calculated chunk hypercube might
collide or not align with existing chunks when partitioning has
changed in one or more dimensions. In such cases, the chunk should be
cut to fit the alignment criteria and any collisions should be
resolved. Unfortunately, alignment and collision detection wasn't
properly handled.
This refactoring adds proper axis-aligned bounding box collision
detection generalized to N dimensions. It also correctly handles
dimension alignment.
Upserts are now supported as long as there is a unique index that
includes all hypertable dimensions. Upserts using ON CONFLICT ON
CONSTRAINT statements are currently unsupported, although this should
be possible in the future.
With this change, hypertables no longer rely on an INSERT trigger to
dispatch tuples to chunks. While an INSERT trigger worked well for
both INSERTs and COPYs, it caused issues with supporting some regular
triggers on hypertables, and didn't support RETURNING statements and
upserts (ON CONFLICT DO UPDATE).
INSERTs are now handled by modifying the plan for INSERT statements. A
custom plan node is inserted as a subplan to a ModifyTable plan node,
taking care of dispatching tuples to chunks by setting the result
table for every tuple scanned.
COPYs are handled by modifying the regular copy code. Unfortunately,
this required copying a significant amount of regular PostgreSQL
source code since there are no hooks to add modifications. However,
since the modifications are small it should be fairly easy to keep the
code in sync with upstream changes.
