mirror of
https://github.com/timescale/timescaledb.git
synced 2025-05-14 01:10:37 +08:00
This release introduces the ability to add secondary indexes to the columnstore, improves group by and filtering performance through columnstore vectorization, and contains the highly upvoted community request of transition table support. We recommend that you upgrade at the next available opportunity. **Highlighted features in TimescaleDB v2.18.0** * The ability to add secondary indexes to the columnstore through the new hypercore table access method. * Significant performance improvements through vectorization (`SIMD`) for aggregations using a group by with one column and/or using a filter clause when querying the columnstore. * Hypertables support triggers for transition tables, which is one of the most upvoted community feature requests. * Updated methods to manage Timescale's hybrid row-columnar store (hypercore) that highlight the usage of the columnstore which includes both an optimized columnar format as well as compression. **Dropping support for Bitnami images** After the recent change in Bitnami’s [LTS support policy](https://github.com/bitnami/containers/issues/75671), we are no longer building Bitnami images for TimescaleDB. We recommend using the [official TimescaleDB Docker image](https://hub.docker.com/r/timescale/timescaledb-ha) **Deprecation Notice** We are deprecating the following parameters, functions, procedures and views. They will be removed with the next major release of TimescaleDB. Please find the replacements in the table below: | Deprecated | Replacement | Type | | --- | --- | --- | | decompress_chunk | convert_to_rowstore | Procedure | | compress_chunk | convert_to_columnstore | Procedure | | add_compression_policy | add_columnstore_policy | Function | | remove_compression_policy | remove_columnstore_policy | Function | | hypertable_compression_stats | hypertable_columnstore_stats | Function | | chunk_compression_stats | chunk_columnstore_stats | Function | | hypertable_compression_settings | hypertable_columnstore_settings | View | | chunk_compression_settings | chunk_columnstore_settings | View | | compression_settings | columnstore_settings | View | | timescaledb.compress | timescaledb.enable_columnstore | Parameter | | timescaledb.compress_segmentby | timescaledb.segmentby | Parameter | | timescaledb.compress_orderby | timescaledb.orderby | Parameter | **Features** * #7341: Vectorized aggregation with grouping by one fixed-size by-value compressed column (such as arithmetic types). * #7104: Hypercore table access method. * #6901: Add hypertable support for transition tables. * #7482: Optimize recompression of partially compressed chunks. * #7458: Support vectorized aggregation with aggregate `filter` clauses that are also vectorizable. * #7433: Add support for merging chunks. * #7271: Push down `order by` in real-time continuous aggregate queries. * #7455: Support `drop not null` on compressed hypertables. * #7295: Support `alter table set access method` on hypertable. * #7411: Change parameter name to enable hypercore table access method. * #7436: Add index creation on `order by` columns. * #7443: Add hypercore function and view aliases. * #7521: Add optional `force` argument to `refresh_continuous_aggregate`. * #7528: Transform sorting on `time_bucket` to sorting on time for compressed chunks in some cases. * #7565: Add hint when hypertable creation fails. * #7390: Disable custom `hashagg` planner code. * #7587: Add `include_tiered_data` parameter to `add_continuous_aggregate_policy` API. * #7486: Prevent building against PostgreSQL versions with broken ABI. * #7412: Add [GUC](https://www.postgresql.org/docs/current/acronyms.html#:~:text=GUC) for the `hypercore_use_access_method` default. * #7413: Add GUC for segmentwise recompression. **Bugfixes** * #7378: Remove obsolete job referencing `policy_job_error_retention`. * #7409: Update `bgw_job` table when altering procedure. * #7410: Fix the `aggregated compressed column not found` error on aggregation query. * #7426: Fix `datetime` parsing error in chunk constraint creation. * #7432: Verify that the heap tuple is valid before using. * #7434: Fix the segfault when internally setting the replica identity for a given chunk. * #7488: Emit error for transition table trigger on chunks. * #7514: Fix the error: `invalid child of chunk append`. * #7517: Fix the performance regression on the `cagg_migrate` procedure. * #7527: Restart scheduler on error. * #7557: Fix null handling for in-memory tuple filtering. * #7566: Improve transaction check in CAGG refresh. * #7584: Fix NaN-handling for vectorized aggregation. * #7598: Match the Postgres NaN comparison behavior in WHERE clause over compressed tables. **Thanks** * @bharrisau for reporting the segfault when creating chunks. * @jakehedlund for reporting the incompatible NaN behavior in WHERE clause over compressed tables. * @k-rus for suggesting that we add a hint when hypertable creation fails. * @staticlibs for sending the pull request that improves the transaction check in CAGG refresh. * @uasiddiqi for reporting the `aggregated compressed column not found` error.
320 lines
11 KiB
CMake
320 lines
11 KiB
CMake
set(INSTALL_FILE ${PROJECT_NAME}--${PROJECT_VERSION_MOD}.sql)
|
|
|
|
include(ScriptFiles)
|
|
|
|
# These files represent the modifications that happen in each version, excluding
|
|
# new objects or updates to functions. We use them to build a path (update
|
|
# script) from every historical version to the current version. Note that not
|
|
# all of these files may exist on disk, in case they would have no contents.
|
|
# There still needs to be an entry here to build an update script for that
|
|
# version. Thus, for every new release, an entry should be added here.
|
|
set(MOD_FILES
|
|
updates/2.5.0--2.5.1.sql
|
|
updates/2.5.1--2.5.2.sql
|
|
updates/2.5.2--2.6.0.sql
|
|
updates/2.6.0--2.6.1.sql
|
|
updates/2.6.1--2.7.0.sql
|
|
updates/2.7.0--2.7.1.sql
|
|
updates/2.7.1--2.7.2.sql
|
|
updates/2.7.2--2.8.0.sql
|
|
updates/2.8.0--2.8.1.sql
|
|
updates/2.8.1--2.9.0.sql
|
|
updates/2.9.0--2.9.1.sql
|
|
updates/2.9.1--2.9.2.sql
|
|
updates/2.9.2--2.9.3.sql
|
|
updates/2.9.3--2.10.0.sql
|
|
updates/2.10.0--2.10.1.sql
|
|
updates/2.10.1--2.10.2.sql
|
|
updates/2.10.2--2.10.3.sql
|
|
updates/2.10.3--2.11.0.sql
|
|
updates/2.11.0--2.11.1.sql
|
|
updates/2.11.1--2.11.2.sql
|
|
updates/2.11.2--2.12.0.sql
|
|
updates/2.12.0--2.12.1.sql
|
|
updates/2.12.1--2.12.2.sql
|
|
updates/2.12.2--2.13.0.sql
|
|
updates/2.13.0--2.13.1.sql
|
|
updates/2.13.1--2.14.0.sql
|
|
updates/2.14.0--2.14.1.sql
|
|
updates/2.14.1--2.14.2.sql
|
|
updates/2.14.2--2.15.0.sql
|
|
updates/2.15.0--2.15.1.sql
|
|
updates/2.15.1--2.15.2.sql
|
|
updates/2.15.2--2.15.3.sql
|
|
updates/2.15.3--2.16.0.sql
|
|
updates/2.16.0--2.16.1.sql
|
|
updates/2.16.1--2.17.0.sql
|
|
updates/2.17.0--2.17.1.sql
|
|
updates/2.17.1--2.17.2.sql
|
|
updates/2.17.2--2.18.0.sql)
|
|
|
|
# The downgrade file to generate a downgrade script for the current version, as
|
|
# specified in version.config
|
|
set(CURRENT_REV_FILE reverse-dev.sql)
|
|
# Files for generating old downgrade scripts. This should only include files for
|
|
# downgrade from one version to its previous version since we do not support
|
|
# skipping versions when downgrading.
|
|
set(OLD_REV_FILES
|
|
2.5.1--2.5.0.sql
|
|
2.5.2--2.5.1.sql
|
|
2.6.0--2.5.2.sql
|
|
2.6.1--2.6.0.sql
|
|
2.7.0--2.6.1.sql
|
|
2.7.1--2.7.0.sql
|
|
2.7.2--2.7.1.sql
|
|
2.8.0--2.7.2.sql
|
|
2.8.1--2.8.0.sql
|
|
2.9.0--2.8.1.sql
|
|
2.9.1--2.9.0.sql
|
|
2.9.2--2.9.1.sql
|
|
2.9.3--2.9.2.sql
|
|
2.10.0--2.9.3.sql
|
|
2.10.1--2.10.0.sql
|
|
2.10.2--2.10.1.sql
|
|
2.10.3--2.10.2.sql
|
|
2.11.0--2.10.3.sql
|
|
2.11.1--2.11.0.sql
|
|
2.11.2--2.11.1.sql
|
|
2.12.0--2.11.2.sql
|
|
2.12.1--2.12.0.sql
|
|
2.12.2--2.12.1.sql
|
|
2.13.0--2.12.2.sql
|
|
2.13.1--2.13.0.sql
|
|
2.14.0--2.13.1.sql
|
|
2.14.1--2.14.0.sql
|
|
2.14.2--2.14.1.sql
|
|
2.15.0--2.14.2.sql
|
|
2.15.1--2.15.0.sql
|
|
2.15.2--2.15.1.sql
|
|
2.15.3--2.15.2.sql
|
|
2.16.0--2.15.3.sql
|
|
2.16.1--2.16.0.sql
|
|
2.17.0--2.16.1.sql
|
|
2.17.1--2.17.0.sql
|
|
2.17.2--2.17.1.sql
|
|
2.18.0--2.17.2.sql)
|
|
|
|
set(MODULE_PATHNAME "$libdir/timescaledb-${PROJECT_VERSION_MOD}")
|
|
set(LOADER_PATHNAME "$libdir/timescaledb")
|
|
|
|
set(TS_MODULE_PATHNAME
|
|
${MODULE_PATHNAME}
|
|
PARENT_SCOPE)
|
|
|
|
set(TSL_MODULE_PATHNAME
|
|
"$libdir/timescaledb-tsl-${PROJECT_VERSION_MOD}"
|
|
PARENT_SCOPE)
|
|
|
|
if(NOT GENERATE_DOWNGRADE_SCRIPT)
|
|
message(
|
|
STATUS "Not generating downgrade script: downgrade generation disabled.")
|
|
elseif(NOT GIT_FOUND)
|
|
message(STATUS "Not generating downgrade script: Git not available.")
|
|
else()
|
|
generate_downgrade_script(
|
|
SOURCE_VERSION
|
|
${PROJECT_VERSION_MOD}
|
|
TARGET_VERSION
|
|
${DOWNGRADE_TO_VERSION}
|
|
INPUT_DIRECTORY
|
|
${CMAKE_CURRENT_SOURCE_DIR}/updates
|
|
FILES
|
|
${CURRENT_REV_FILE})
|
|
if(GENERATE_OLD_DOWNGRADE_SCRIPTS)
|
|
foreach(_downgrade_file ${OLD_REV_FILES})
|
|
if(${_downgrade_file} MATCHES
|
|
"([0-9]+\\.[0-9]+\\.*[0-9]*)--([0-9]+\\.[0-9]+\\.*[0-9]*).sql")
|
|
set(_source_version ${CMAKE_MATCH_1})
|
|
set(_target_version ${CMAKE_MATCH_2})
|
|
generate_downgrade_script(
|
|
SOURCE_VERSION
|
|
${_source_version}
|
|
TARGET_VERSION
|
|
${_target_version}
|
|
INPUT_DIRECTORY
|
|
${CMAKE_CURRENT_SOURCE_DIR}/updates
|
|
FILES
|
|
${_downgrade_file})
|
|
else()
|
|
message(FATAL_ERROR "${_downgrade_file} is not a downgrade file")
|
|
endif()
|
|
endforeach()
|
|
endif()
|
|
endif()
|
|
|
|
# Function to replace @MODULE_PATHNAME@ in source files, producing an output
|
|
# file in the build dir
|
|
function(version_files SRC_FILE_LIST OUTPUT_FILE_LIST)
|
|
set(result "")
|
|
foreach(unversioned_file ${SRC_FILE_LIST})
|
|
set(versioned_file ${unversioned_file})
|
|
list(APPEND result ${CMAKE_CURRENT_BINARY_DIR}/${versioned_file})
|
|
if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${unversioned_file})
|
|
configure_file(${unversioned_file} ${versioned_file} @ONLY)
|
|
endif()
|
|
endforeach(unversioned_file)
|
|
set(${OUTPUT_FILE_LIST}
|
|
"${result}"
|
|
PARENT_SCOPE)
|
|
endfunction()
|
|
|
|
# Function to replace @VARIABLE@ in source files, producing an output file in
|
|
# the build dir. Unlike the previous function we need to build a versioned file
|
|
# for every version we create an update script for.
|
|
function(version_check_file SRC_FILE_LIST OUTPUT_FILE_LIST)
|
|
set(result "")
|
|
foreach(unversioned_file ${SRC_FILE_LIST})
|
|
set(versioned_file ${unversioned_file}-${START_VERSION})
|
|
list(APPEND result ${CMAKE_CURRENT_BINARY_DIR}/${versioned_file})
|
|
if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${unversioned_file})
|
|
configure_file(${unversioned_file} ${versioned_file} @ONLY)
|
|
endif()
|
|
endforeach(unversioned_file)
|
|
set(${OUTPUT_FILE_LIST}
|
|
"${result}"
|
|
PARENT_SCOPE)
|
|
endfunction()
|
|
|
|
# Create versioned files (replacing MODULE_PATHNAME) in the build directory of
|
|
# all our source files
|
|
version_files("${PRE_UPDATE_FILES}" PRE_UPDATE_FILES_VERSIONED)
|
|
version_files("${POST_UPDATE_FILES}" POST_UPDATE_FILES_VERSIONED)
|
|
version_files("${PRE_INSTALL_SOURCE_FILES}" PRE_INSTALL_SOURCE_FILES_VERSIONED)
|
|
version_files("${PRE_INSTALL_FUNCTION_FILES}"
|
|
PRE_INSTALL_FUNCTION_FILES_VERSIONED)
|
|
version_files("${SOURCE_FILES}" SOURCE_FILES_VERSIONED)
|
|
version_files("${MOD_FILES}" MOD_FILES_VERSIONED)
|
|
version_files("updates/latest-dev.sql" LASTEST_MOD_VERSIONED)
|
|
version_files("notice.sql" NOTICE_FILE)
|
|
|
|
# Function to concatenate all files in SRC_FILE_LIST into file OUTPUT_FILE
|
|
function(cat_files SRC_FILE_LIST OUTPUT_FILE)
|
|
if(WIN32)
|
|
set("SRC_ARG" "-DSRC_FILE_LIST=${SRC_FILE_LIST}")
|
|
else()
|
|
set("SRC_ARG" "'-DSRC_FILE_LIST=${SRC_FILE_LIST}'")
|
|
endif()
|
|
add_custom_command(
|
|
OUTPUT ${OUTPUT_FILE}
|
|
DEPENDS ${SRC_FILE_LIST}
|
|
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
|
|
COMMAND ${CMAKE_COMMAND} "${SRC_ARG}" "-DOUTPUT_FILE=${OUTPUT_FILE}" -P
|
|
cat.cmake
|
|
COMMENT "Generating ${OUTPUT_FILE}")
|
|
endfunction()
|
|
|
|
# Concatenate all files in SRC_FILE_LIST into file OUTPUT_FILE Additionally
|
|
# strips out `OR REPLACE` from the generated file as this would allow a
|
|
# privilege escalation attack when present in the CREATE script
|
|
function(cat_create_files SRC_FILE_LIST OUTPUT_FILE)
|
|
if(WIN32)
|
|
set("SRC_ARG" "-DSRC_FILE_LIST=${SRC_FILE_LIST}")
|
|
else()
|
|
set("SRC_ARG" "'-DSRC_FILE_LIST=${SRC_FILE_LIST}'")
|
|
endif()
|
|
if(WIN32)
|
|
add_custom_command(
|
|
OUTPUT ${OUTPUT_FILE}
|
|
DEPENDS ${SRC_FILE_LIST}
|
|
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
|
|
COMMAND ${CMAKE_COMMAND} "${SRC_ARG}" "-DOUTPUT_FILE=${OUTPUT_FILE}"
|
|
"-DSTRIP_REPLACE=ON" -P cat.cmake
|
|
COMMENT "Generating ${OUTPUT_FILE}")
|
|
else()
|
|
add_custom_command(
|
|
OUTPUT ${OUTPUT_FILE}
|
|
DEPENDS ${SRC_FILE_LIST}
|
|
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
|
|
COMMAND ${CMAKE_COMMAND} "'-DSRC_FILE_LIST=${SRC_FILE_LIST}'"
|
|
"-DOUTPUT_FILE=${OUTPUT_FILE}" "-DSTRIP_REPLACE=ON" -P cat.cmake
|
|
COMMENT "Generating ${OUTPUT_FILE}")
|
|
endif()
|
|
endfunction()
|
|
|
|
# Generate the extension file used with CREATE EXTENSION
|
|
cat_create_files(
|
|
"${PRE_INSTALL_SOURCE_FILES_VERSIONED};${SOURCE_FILES_VERSIONED};${NOTICE_FILE}"
|
|
${CMAKE_CURRENT_BINARY_DIR}/${INSTALL_FILE})
|
|
add_custom_target(sqlfile ALL
|
|
DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/${INSTALL_FILE})
|
|
|
|
# Generate the update files used with ALTER EXTENSION <name> UPDATE
|
|
set(MOD_FILE_REGEX
|
|
"([0-9]+\\.[0-9]+\\.*[0-9]+[-a-z0-9]*)--([0-9]+\\.[0-9]+\\.*[0-9]+[-a-z0-9]*).sql"
|
|
)
|
|
|
|
# We'd like to process the updates in reverse (descending) order
|
|
if(EXISTS
|
|
"${CMAKE_CURRENT_SOURCE_DIR}/updates/${UPDATE_FROM_VERSION}--${PROJECT_VERSION_MOD}.sql"
|
|
)
|
|
set(MOD_FILES_LIST ${MOD_FILES_VERSIONED})
|
|
else()
|
|
set(MOD_FILES_LIST
|
|
"${MOD_FILES_VERSIONED};updates/${UPDATE_FROM_VERSION}--${PROJECT_VERSION_MOD}.sql"
|
|
)
|
|
endif()
|
|
|
|
list(REVERSE MOD_FILES_LIST)
|
|
|
|
# Variable that will hold the list of update scripts from every previous version
|
|
# to the current version
|
|
set(UPDATE_SCRIPTS "")
|
|
|
|
# A list of current modfiles. We append to this list for every previous version
|
|
# that moves us further away from the current version, thus making the update
|
|
# path longer as we move back in history
|
|
set(CURR_MOD_FILES "${LASTEST_MOD_VERSIONED}")
|
|
|
|
# Now loop through the modfiles and generate the update files
|
|
foreach(transition_mod_file ${MOD_FILES_LIST})
|
|
|
|
if(NOT (${transition_mod_file} MATCHES ${MOD_FILE_REGEX}))
|
|
message(FATAL_ERROR "Cannot parse update file name ${mod_file}")
|
|
endif()
|
|
|
|
set(START_VERSION ${CMAKE_MATCH_1})
|
|
set(END_VERSION ${CMAKE_MATCH_2})
|
|
set(PRE_FILES ${PRE_UPDATE_FILES_VERSIONED})
|
|
set(POST_FILES_PROCESSED ${POST_UPDATE_FILES_VERSIONED}.processed)
|
|
cat_files(
|
|
"${SET_POST_UPDATE_STAGE};${POST_UPDATE_FILES_VERSIONED};${UNSET_UPDATE_STAGE}"
|
|
${POST_FILES_PROCESSED})
|
|
# Check for version-specific update code with fixes
|
|
if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/updates/${START_VERSION}.sql)
|
|
version_files("updates/${START_VERSION}.sql" ORIGIN_MOD_FILE)
|
|
list(APPEND PRE_FILES ${ORIGIN_MOD_FILE})
|
|
endif()
|
|
|
|
version_check_file("updates/version_check.sql" VERSION_CHECK_VERSIONED)
|
|
list(PREPEND PRE_FILES ${VERSION_CHECK_VERSIONED})
|
|
|
|
# There might not have been any changes in the modfile, in which case the
|
|
# modfile need not be present
|
|
if(EXISTS ${transition_mod_file})
|
|
# Prepend the modfile as we are moving through the versions in descending
|
|
# order
|
|
list(INSERT CURR_MOD_FILES 0 ${transition_mod_file})
|
|
endif()
|
|
|
|
set(UPDATE_SCRIPT
|
|
${CMAKE_CURRENT_BINARY_DIR}/timescaledb--${START_VERSION}--${PROJECT_VERSION_MOD}.sql
|
|
)
|
|
list(APPEND UPDATE_SCRIPTS ${UPDATE_SCRIPT})
|
|
if(CURR_MOD_FILES)
|
|
cat_files(
|
|
"${PRE_FILES};${CURR_MOD_FILES};${PRE_INSTALL_FUNCTION_FILES_VERSIONED};${SOURCE_FILES_VERSIONED};${POST_FILES_PROCESSED}"
|
|
${UPDATE_SCRIPT})
|
|
else()
|
|
cat_files(
|
|
"${PRE_FILES};${PRE_INSTALL_FUNCTION_FILES_VERSIONED};${SOURCE_FILES_VERSIONED};${POST_FILES_PROCESSED}"
|
|
${UPDATE_SCRIPT})
|
|
endif()
|
|
|
|
endforeach(transition_mod_file)
|
|
|
|
add_custom_target(sqlupdatescripts ALL DEPENDS ${UPDATE_SCRIPTS})
|
|
|
|
# Install target for the extension file and update scripts
|
|
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/${INSTALL_FILE} ${UPDATE_SCRIPTS}
|
|
DESTINATION "${PG_SHAREDIR}/extension")
|