postgres/timescaledb
1
0
Fork 0
mirror of https://github.com/timescale/timescaledb.git synced 2025-05-17 02:53:51 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix use-after-free in ChunkAppend

Browse Source 
Fixes use-after-free in chunk_append/exec.c. Also, strip out memory
usage from EXPLAIN ANALYZE output of Sort nodes because it is not
stable across platforms
This commit is contained in:
Sven Klemm 2019-06-21 15:37:44 -04:00 committed by Sven Klemm
parent 2801c6a5f5
commit 6f936ea2e3
6 changed files with 28 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/chunk_append/exec.c
View File

@ -187,7 +187,7 @@ chunk_append_begin(CustomScanState *node, EState *estate, int eflags)
/* /*
* make sure all params are initialized for runtime exclusion * make sure all params are initialized for runtime exclusion
*/ */
node->ss.ps.chgParam = state->subplanstates[0]->plan->allParam; node->ss.ps.chgParam = bms_copy(state->subplanstates[0]->plan->allParam);
} }
} }

14
test/expected/append-10.out
View File

@ -265,7 +265,7 @@ psql:include/append_query.sql:58: NOTICE: Stable function now_s() called!
---------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------
Sort (actual rows=1 loops=1) Sort (actual rows=1 loops=1)
Sort Key: (date_trunc('year'::text, append_test."time")) DESC Sort Key: (date_trunc('year'::text, append_test."time")) DESC
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> HashAggregate (actual rows=1 loops=1) -> HashAggregate (actual rows=1 loops=1)
Group Key: (date_trunc('year'::text, append_test."time")) Group Key: (date_trunc('year'::text, append_test."time"))
-> Custom Scan (ChunkAppend) on append_test (actual rows=3 loops=1) -> Custom Scan (ChunkAppend) on append_test (actual rows=3 loops=1)
@ -292,7 +292,7 @@ ORDER BY t DESC;
Group Key: (date_trunc('year'::text, "time")) Group Key: (date_trunc('year'::text, "time"))
-> Sort (actual rows=0 loops=1) -> Sort (actual rows=0 loops=1)
Sort Key: (date_trunc('year'::text, "time")) DESC Sort Key: (date_trunc('year'::text, "time")) DESC
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> Result (actual rows=0 loops=1) -> Result (actual rows=0 loops=1)
One-Time Filter: false One-Time Filter: false
(7 rows) (7 rows)
@ -363,11 +363,11 @@ psql:include/append_query.sql:98: NOTICE: Stable function now_s() called!
-> Function Scan on generate_series "time" (actual rows=6 loops=1) -> Function Scan on generate_series "time" (actual rows=6 loops=1)
-> Sort (actual rows=6 loops=1) -> Sort (actual rows=6 loops=1)
Sort Key: period.btime Sort Key: period.btime
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> CTE Scan on period (actual rows=6 loops=1) -> CTE Scan on period (actual rows=6 loops=1)
-> Sort (actual rows=3 loops=1) -> Sort (actual rows=3 loops=1)
Sort Key: data.btime Sort Key: data.btime
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> CTE Scan on data (actual rows=3 loops=1) -> CTE Scan on data (actual rows=3 loops=1)
(26 rows) (26 rows)
@ -968,7 +968,7 @@ ORDER BY time DESC;
---------------------------------------------------------------------------------- ----------------------------------------------------------------------------------
Sort (actual rows=17 loops=1) Sort (actual rows=17 loops=1)
Sort Key: _hyper_3_11_chunk."time" DESC Sort Key: _hyper_3_11_chunk."time" DESC
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> Append (actual rows=17 loops=1) -> Append (actual rows=17 loops=1)
-> Sample Scan on _hyper_3_11_chunk (actual rows=4 loops=1) -> Sample Scan on _hyper_3_11_chunk (actual rows=4 loops=1)
Sampling: bernoulli ('5'::real) REPEATABLE ('0'::double precision) Sampling: bernoulli ('5'::real) REPEATABLE ('0'::double precision)
@ -991,7 +991,7 @@ ORDER BY time DESC;
---------------------------------------------------------------------------------- ----------------------------------------------------------------------------------
Sort (actual rows=17 loops=1) Sort (actual rows=17 loops=1)
Sort Key: metrics_date."time" DESC Sort Key: metrics_date."time" DESC
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> Custom Scan (ChunkAppend) on metrics_date (actual rows=17 loops=1) -> Custom Scan (ChunkAppend) on metrics_date (actual rows=17 loops=1)
Chunks excluded during startup: 2 Chunks excluded during startup: 2
-> Sample Scan on _hyper_3_11_chunk (actual rows=4 loops=1) -> Sample Scan on _hyper_3_11_chunk (actual rows=4 loops=1)
@ -1089,7 +1089,7 @@ ORDER BY time DESC;
Heap Fetches: 129 Heap Fetches: 129
-> Sort (actual rows=32 loops=1) -> Sort (actual rows=32 loops=1)
Sort Key: g."time" Sort Key: g."time"
Sort Method: quicksort Memory: 26kB Sort Method: quicksort
-> Function Scan on generate_series g (actual rows=32 loops=1) -> Function Scan on generate_series g (actual rows=32 loops=1)
(18 rows) (18 rows)

14
test/expected/append-11.out
View File

@ -265,7 +265,7 @@ psql:include/append_query.sql:58: NOTICE: Stable function now_s() called!
---------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------
Sort (actual rows=1 loops=1) Sort (actual rows=1 loops=1)
Sort Key: (date_trunc('year'::text, append_test."time")) DESC Sort Key: (date_trunc('year'::text, append_test."time")) DESC
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> HashAggregate (actual rows=1 loops=1) -> HashAggregate (actual rows=1 loops=1)
Group Key: date_trunc('year'::text, append_test."time") Group Key: date_trunc('year'::text, append_test."time")
-> Custom Scan (ChunkAppend) on append_test (actual rows=3 loops=1) -> Custom Scan (ChunkAppend) on append_test (actual rows=3 loops=1)
@ -292,7 +292,7 @@ ORDER BY t DESC;
Group Key: (date_trunc('year'::text, "time")) Group Key: (date_trunc('year'::text, "time"))
-> Sort (actual rows=0 loops=1) -> Sort (actual rows=0 loops=1)
Sort Key: (date_trunc('year'::text, "time")) DESC Sort Key: (date_trunc('year'::text, "time")) DESC
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> Result (actual rows=0 loops=1) -> Result (actual rows=0 loops=1)
One-Time Filter: false One-Time Filter: false
(7 rows) (7 rows)
@ -363,11 +363,11 @@ psql:include/append_query.sql:98: NOTICE: Stable function now_s() called!
-> Function Scan on generate_series "time" (actual rows=6 loops=1) -> Function Scan on generate_series "time" (actual rows=6 loops=1)
-> Sort (actual rows=6 loops=1) -> Sort (actual rows=6 loops=1)
Sort Key: period.btime Sort Key: period.btime
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> CTE Scan on period (actual rows=6 loops=1) -> CTE Scan on period (actual rows=6 loops=1)
-> Sort (actual rows=3 loops=1) -> Sort (actual rows=3 loops=1)
Sort Key: data.btime Sort Key: data.btime
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> CTE Scan on data (actual rows=3 loops=1) -> CTE Scan on data (actual rows=3 loops=1)
(26 rows) (26 rows)
@ -968,7 +968,7 @@ ORDER BY time DESC;
---------------------------------------------------------------------------------- ----------------------------------------------------------------------------------
Sort (actual rows=17 loops=1) Sort (actual rows=17 loops=1)
Sort Key: _hyper_3_11_chunk."time" DESC Sort Key: _hyper_3_11_chunk."time" DESC
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> Append (actual rows=17 loops=1) -> Append (actual rows=17 loops=1)
-> Sample Scan on _hyper_3_11_chunk (actual rows=4 loops=1) -> Sample Scan on _hyper_3_11_chunk (actual rows=4 loops=1)
Sampling: bernoulli ('5'::real) REPEATABLE ('0'::double precision) Sampling: bernoulli ('5'::real) REPEATABLE ('0'::double precision)
@ -991,7 +991,7 @@ ORDER BY time DESC;
---------------------------------------------------------------------------------- ----------------------------------------------------------------------------------
Sort (actual rows=17 loops=1) Sort (actual rows=17 loops=1)
Sort Key: metrics_date."time" DESC Sort Key: metrics_date."time" DESC
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> Custom Scan (ChunkAppend) on metrics_date (actual rows=17 loops=1) -> Custom Scan (ChunkAppend) on metrics_date (actual rows=17 loops=1)
Chunks excluded during startup: 2 Chunks excluded during startup: 2
-> Sample Scan on _hyper_3_11_chunk (actual rows=4 loops=1) -> Sample Scan on _hyper_3_11_chunk (actual rows=4 loops=1)
@ -1089,7 +1089,7 @@ ORDER BY time DESC;
Heap Fetches: 129 Heap Fetches: 129
-> Sort (actual rows=32 loops=1) -> Sort (actual rows=32 loops=1)
Sort Key: g."time" Sort Key: g."time"
Sort Method: quicksort Memory: 26kB Sort Method: quicksort
-> Function Scan on generate_series g (actual rows=32 loops=1) -> Function Scan on generate_series g (actual rows=32 loops=1)
(18 rows) (18 rows)

10
test/expected/plan_ordered_append-10.out
View File

@ -432,7 +432,7 @@ ORDER BY device_id, time LIMIT 1;
Limit (actual rows=1 loops=1) Limit (actual rows=1 loops=1)
-> Sort (actual rows=1 loops=1) -> Sort (actual rows=1 loops=1)
Sort Key: _hyper_1_1_chunk.device_id, _hyper_1_1_chunk."time" Sort Key: _hyper_1_1_chunk.device_id, _hyper_1_1_chunk."time"
Sort Method: top-N heapsort Memory: 25kB Sort Method: top-N heapsort
-> Append (actual rows=73443 loops=1) -> Append (actual rows=73443 loops=1)
-> Seq Scan on _hyper_1_1_chunk (actual rows=20160 loops=1) -> Seq Scan on _hyper_1_1_chunk (actual rows=20160 loops=1)
-> Seq Scan on _hyper_1_2_chunk (actual rows=30240 loops=1) -> Seq Scan on _hyper_1_2_chunk (actual rows=30240 loops=1)
@ -932,7 +932,7 @@ ORDER BY time LIMIT 10;
Order: ht_missing_indexes."time" Order: ht_missing_indexes."time"
-> Sort (actual rows=10 loops=1) -> Sort (actual rows=10 loops=1)
Sort Key: _hyper_5_16_chunk."time" Sort Key: _hyper_5_16_chunk."time"
Sort Method: top-N heapsort Memory: 25kB Sort Method: top-N heapsort
-> Seq Scan on _hyper_5_16_chunk (actual rows=24477 loops=1) -> Seq Scan on _hyper_5_16_chunk (actual rows=24477 loops=1)
Filter: ("time" > 'Fri Jan 07 00:00:00 2000 PST'::timestamp with time zone) Filter: ("time" > 'Fri Jan 07 00:00:00 2000 PST'::timestamp with time zone)
Rows Removed by Filter: 5763 Rows Removed by Filter: 5763
@ -1036,7 +1036,7 @@ ORDER BY time, device_id LIMIT 1;
Limit (actual rows=1 loops=1) Limit (actual rows=1 loops=1)
-> Sort (actual rows=1 loops=1) -> Sort (actual rows=1 loops=1)
Sort Key: _hyper_7_24_chunk."time", _hyper_7_24_chunk.device_id Sort Key: _hyper_7_24_chunk."time", _hyper_7_24_chunk.device_id
Sort Method: top-N heapsort Memory: 25kB Sort Method: top-N heapsort
-> Append (actual rows=77766 loops=1) -> Append (actual rows=77766 loops=1)
-> Seq Scan on _hyper_7_24_chunk (actual rows=6720 loops=1) -> Seq Scan on _hyper_7_24_chunk (actual rows=6720 loops=1)
-> Seq Scan on _hyper_7_26_chunk (actual rows=13440 loops=1) -> Seq Scan on _hyper_7_26_chunk (actual rows=13440 loops=1)
@ -1059,7 +1059,7 @@ ORDER BY device_id, time LIMIT 1;
Limit (actual rows=1 loops=1) Limit (actual rows=1 loops=1)
-> Sort (actual rows=1 loops=1) -> Sort (actual rows=1 loops=1)
Sort Key: _hyper_7_23_chunk.device_id, _hyper_7_23_chunk."time" Sort Key: _hyper_7_23_chunk.device_id, _hyper_7_23_chunk."time"
Sort Method: top-N heapsort Memory: 25kB Sort Method: top-N heapsort
-> Append (actual rows=77766 loops=1) -> Append (actual rows=77766 loops=1)
-> Seq Scan on _hyper_7_23_chunk (actual rows=6241 loops=1) -> Seq Scan on _hyper_7_23_chunk (actual rows=6241 loops=1)
-> Seq Scan on _hyper_7_24_chunk (actual rows=6720 loops=1) -> Seq Scan on _hyper_7_24_chunk (actual rows=6720 loops=1)
@ -1353,7 +1353,7 @@ FROM ordered_append o1 INNER JOIN (SELECT max(time) AS max_time FROM ordered_app
-------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------
Sort (actual rows=3 loops=1) Sort (actual rows=3 loops=1)
Sort Key: o1."time" Sort Key: o1."time"
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> Nested Loop (actual rows=3 loops=1) -> Nested Loop (actual rows=3 loops=1)
-> Result (actual rows=1 loops=1) -> Result (actual rows=1 loops=1)
InitPlan 1 (returns $0) InitPlan 1 (returns $0)

10
test/expected/plan_ordered_append-11.out
View File

@ -432,7 +432,7 @@ ORDER BY device_id, time LIMIT 1;
Limit (actual rows=1 loops=1) Limit (actual rows=1 loops=1)
-> Sort (actual rows=1 loops=1) -> Sort (actual rows=1 loops=1)
Sort Key: _hyper_1_1_chunk.device_id, _hyper_1_1_chunk."time" Sort Key: _hyper_1_1_chunk.device_id, _hyper_1_1_chunk."time"
Sort Method: top-N heapsort Memory: 25kB Sort Method: top-N heapsort
-> Append (actual rows=73443 loops=1) -> Append (actual rows=73443 loops=1)
-> Seq Scan on _hyper_1_1_chunk (actual rows=20160 loops=1) -> Seq Scan on _hyper_1_1_chunk (actual rows=20160 loops=1)
-> Seq Scan on _hyper_1_2_chunk (actual rows=30240 loops=1) -> Seq Scan on _hyper_1_2_chunk (actual rows=30240 loops=1)
@ -932,7 +932,7 @@ ORDER BY time LIMIT 10;
Order: ht_missing_indexes."time" Order: ht_missing_indexes."time"
-> Sort (actual rows=10 loops=1) -> Sort (actual rows=10 loops=1)
Sort Key: _hyper_5_16_chunk."time" Sort Key: _hyper_5_16_chunk."time"
Sort Method: top-N heapsort Memory: 25kB Sort Method: top-N heapsort
-> Seq Scan on _hyper_5_16_chunk (actual rows=24477 loops=1) -> Seq Scan on _hyper_5_16_chunk (actual rows=24477 loops=1)
Filter: ("time" > 'Fri Jan 07 00:00:00 2000 PST'::timestamp with time zone) Filter: ("time" > 'Fri Jan 07 00:00:00 2000 PST'::timestamp with time zone)
Rows Removed by Filter: 5763 Rows Removed by Filter: 5763
@ -1036,7 +1036,7 @@ ORDER BY time, device_id LIMIT 1;
Limit (actual rows=1 loops=1) Limit (actual rows=1 loops=1)
-> Sort (actual rows=1 loops=1) -> Sort (actual rows=1 loops=1)
Sort Key: _hyper_7_24_chunk."time", _hyper_7_24_chunk.device_id Sort Key: _hyper_7_24_chunk."time", _hyper_7_24_chunk.device_id
Sort Method: top-N heapsort Memory: 25kB Sort Method: top-N heapsort
-> Append (actual rows=77766 loops=1) -> Append (actual rows=77766 loops=1)
-> Seq Scan on _hyper_7_24_chunk (actual rows=6720 loops=1) -> Seq Scan on _hyper_7_24_chunk (actual rows=6720 loops=1)
-> Seq Scan on _hyper_7_26_chunk (actual rows=13440 loops=1) -> Seq Scan on _hyper_7_26_chunk (actual rows=13440 loops=1)
@ -1059,7 +1059,7 @@ ORDER BY device_id, time LIMIT 1;
Limit (actual rows=1 loops=1) Limit (actual rows=1 loops=1)
-> Sort (actual rows=1 loops=1) -> Sort (actual rows=1 loops=1)
Sort Key: _hyper_7_23_chunk.device_id, _hyper_7_23_chunk."time" Sort Key: _hyper_7_23_chunk.device_id, _hyper_7_23_chunk."time"
Sort Method: top-N heapsort Memory: 25kB Sort Method: top-N heapsort
-> Append (actual rows=77766 loops=1) -> Append (actual rows=77766 loops=1)
-> Seq Scan on _hyper_7_23_chunk (actual rows=6241 loops=1) -> Seq Scan on _hyper_7_23_chunk (actual rows=6241 loops=1)
-> Seq Scan on _hyper_7_24_chunk (actual rows=6720 loops=1) -> Seq Scan on _hyper_7_24_chunk (actual rows=6720 loops=1)
@ -1353,7 +1353,7 @@ FROM ordered_append o1 INNER JOIN (SELECT max(time) AS max_time FROM ordered_app
-------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------
Sort (actual rows=3 loops=1) Sort (actual rows=3 loops=1)
Sort Key: o1."time" Sort Key: o1."time"
Sort Method: quicksort Memory: 25kB Sort Method: quicksort
-> Nested Loop (actual rows=3 loops=1) -> Nested Loop (actual rows=3 loops=1)
-> Result (actual rows=1 loops=1) -> Result (actual rows=1 loops=1)
InitPlan 1 (returns $0) InitPlan 1 (returns $0)

4
test/runner.sh
View File

@ -58,6 +58,8 @@ ${PSQL} $@ -U $TEST_ROLE_SUPERUSER -d ${TEST_DBNAME} -v ECHO=none -v MODULE_PATH
export TEST_DBNAME export TEST_DBNAME
# we strip out any output between <exclude_from_test></exclude_from_test>
# and the part about memory usage in EXPLAIN ANALYZE output of Sort nodes
${PSQL} -U ${TEST_PGUSER} \ ${PSQL} -U ${TEST_PGUSER} \
-v ON_ERROR_STOP=1 \ -v ON_ERROR_STOP=1 \
-v VERBOSITY=terse \ -v VERBOSITY=terse \
@ -74,4 +76,4 @@ ${PSQL} -U ${TEST_PGUSER} \
-v ROLE_DEFAULT_PERM_USER_2=${TEST_ROLE_DEFAULT_PERM_USER_2} \ -v ROLE_DEFAULT_PERM_USER_2=${TEST_ROLE_DEFAULT_PERM_USER_2} \
-v MODULE_PATHNAME="'timescaledb-${EXT_VERSION}'" \ -v MODULE_PATHNAME="'timescaledb-${EXT_VERSION}'" \
-v TSL_MODULE_PATHNAME="'timescaledb-tsl-${EXT_VERSION}'" \ -v TSL_MODULE_PATHNAME="'timescaledb-tsl-${EXT_VERSION}'" \
$@ -d ${TEST_DBNAME} 2>&1 | sed '/<exclude_from_test>/,/<\/exclude_from_test>/d' $@ -d ${TEST_DBNAME} 2>&1 | sed -e '/<exclude_from_test>/,/<\/exclude_from_test>/d' -e 's! Memory: [0-9]\+kB!!'