New cluster-like command which writes to a new index than swaps,
much like is done for the data table, and only acquires
exclusive locks for said swap. This trades off disk usage for
lower contention: we hold locks for a much lower period of time,
allowing reads to work concurrently, but we have both the old
and new versions of the table existing at once, approximately
doubling storage usage while reorder is running.
Currently only works on chunks.
Future proofing: if we ever want to make our functions available to
others they’d need to be prefixed to prevent name collisions. In
order to avoid having some functions with the ts_ prefix and
others without, we’re adding the prefix to all non-static
functions now.
Cache pins are allocated on the CacheMemoryContext in order to survive
subtransactions, and, optionally, main transactions (in case of, e.g.,
clustering or vaccuming). However, cache pins that are released also
needs to free memory on the CacheMemoryContext in order to avoid
leaking memory in a session.
This change ensures cache pin memory is freed when a cache pin is
released. It also allocates cache pins on a separate memory context,
which is a child of the CacheMemoryContext. This separate memory
context makes it easier to track the memory used by cache pins and
also release it when necessary.
Previously, cache lookups were run on the cache's memory
context. While simple, this risked allocating transient (work) data on
that memory context, e.g., when scanning for new cache data during
cache misses.
This change makes scan functions take a memory context, which the
found data should be allocated on. All other data is allocated on the
current memory (typically the transaction's memory context). With this
functionality, a cache can pass its memory context to the scan, thus
avoiding taking on unnecessary memory allocations.
Previously, all Subtxn aborts released all cache pins. This is
wrong because that can release pins that are still in use by
higher-level subtxns and top-level txns. Now, we release only
pins corresponding to the appropriate subtxn. That means that we
now track the subtxn a pin was created in.
This commit adds logic for cache pinning to handle subtxn. It also makes
it easier to find cache pinning leaks. Finally, it fixes handling of
cross-commit operations like VACUUM and CLUSTER. Previously, such
operations incorrectly released that cache pin on the first commit
even though the object was used after that.
Currently, if a transaction ends (normally or abnormally) while a
cache is pinned, that cache will never be released, leading to a
memory leak. This change ensures that all taken cache pins are
released when a transaction ends by registering all pins in a list and
cleaning it up in a transaction end callback. This makes the use of
cache_release() optional. However, timely calls to cache_release() is
still encouraged since it is better to release memory as soon as
possible during long-running transactions.
Previously, the extension could end up in a bad state if it
was dropped as part of a cascade. This PR fixes that by
checking explicitly for the presence of the proxy table
to make sure we are not in the middle of an extension
drop. Fixes#73.
In case new allocations are made during a cache entry update, we
should use the cache's internal memory context to ensure that new data
is allocated and freed in a way consistent with the rest of the cache.
