postgres/timescaledb
1
0
Fork 0
mirror of https://github.com/timescale/timescaledb.git synced 2025-05-19 04:03:06 +08:00
Code Issues Packages Projects Releases Wiki Activity

Enable indexscan on uncompressed part of partially compressed chunks

Browse Source 
This was previously disabled as no data resided on the
uncompressed chunk once it was compressed, but this is not
the case anymore with partially compressed chunks, so we
enable indexscan for the uncompressed chunk again.

Fixes #5432

Co-authored-by: Ante Kresic <ante.kresic@gmail.com>
This commit is contained in:
Konstantina Skovola 2023-04-11 12:33:11 +03:00 committed by Konstantina Skovola
parent ca472ebb0d
commit 5633960f8b
4 changed files with 64 additions and 82 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG.md
View File

@ -40,6 +40,7 @@ accidentally triggering the load of a previous DB version.**
* #5543 Copy scheduled_jobs list before sorting it * #5543 Copy scheduled_jobs list before sorting it
* #5570 Improve interpolate error message on datatype mismatch * #5570 Improve interpolate error message on datatype mismatch
* #5558 Use regrole for job owner * #5558 Use regrole for job owner
* #5542 Enable indexscan on uncompressed part of partially compressed chunks
**Thanks** **Thanks**
* @nikolaps for reporting an issue with the COPY fetcher * @nikolaps for reporting an issue with the COPY fetcher

7
src/planner/planner.c
View File

@ -1345,12 +1345,15 @@ timescaledb_get_relation_info_hook(PlannerInfo *root, Oid relation_objectid, boo
ts_get_private_reloptinfo(rel)->compressed = true; ts_get_private_reloptinfo(rel)->compressed = true;
/* Planning indexes are expensive, and if this is a compressed chunk, we /* Planning indexes is expensive, and if this is a fully compressed chunk, we
* know we'll never need to use indexes on the uncompressed version, since * know we'll never need to use indexes on the uncompressed version, since
* all the data is in the compressed chunk anyway. Therefore, it is much * all the data is in the compressed chunk anyway. Therefore, it is much
* faster if we simply trash the indexlist here and never plan any useless * faster if we simply trash the indexlist here and never plan any useless
* IndexPaths at all * IndexPaths at all.
* If the chunk is partially compressed, then we should enable indexScan
* on the uncompressed part.
*/ */
if (!ts_chunk_is_partial(chunk))
rel->indexlist = NIL; rel->indexlist = NIL;
table_close(uncompressed_chunk, NoLock); table_close(uncompressed_chunk, NoLock);
} }

102
tsl/test/expected/compression_ddl.out
View File

@ -1442,7 +1442,7 @@ AND time <= '2000-01-05 23:55:00+0';
-- force index scans to check index mapping -- force index scans to check index mapping
-- this verifies that we are actually using compressed chunk index scans -- this verifies that we are actually using compressed chunk index scans
-- currently we cannot use indexes on uncompressed chunks due to a bug: -- previously we could not use indexes on uncompressed chunks due to a bug:
-- https://github.com/timescale/timescaledb/issues/5432 -- https://github.com/timescale/timescaledb/issues/5432
-- --
-- this check basically makes sure that the indexes are built properly -- this check basically makes sure that the indexes are built properly
@ -1452,27 +1452,25 @@ EXPLAIN (costs off) SELECT device_id, count(*)
FROM compression_insert FROM compression_insert
GROUP BY device_id; GROUP BY device_id;
QUERY PLAN QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------------------------------------
GroupAggregate HashAggregate
Group Key: _hyper_31_107_chunk.device_id Group Key: _hyper_31_107_chunk.device_id
-> Sort
Sort Key: _hyper_31_107_chunk.device_id
-> Append -> Append
-> Custom Scan (DecompressChunk) on _hyper_31_107_chunk -> Custom Scan (DecompressChunk) on _hyper_31_107_chunk
-> Index Scan using compress_hyper_32_108_chunk__compressed_hypertable_32_device_id on compress_hyper_32_108_chunk -> Index Scan using compress_hyper_32_108_chunk__compressed_hypertable_32_device_id on compress_hyper_32_108_chunk
-> Seq Scan on _hyper_31_107_chunk -> Index Only Scan using _hyper_31_107_chunk_compression_insert_device_id_time_idx on _hyper_31_107_chunk
(8 rows) (6 rows)
SELECT device_id, count(*) SELECT device_id, count(*)
FROM compression_insert FROM compression_insert
GROUP BY device_id; GROUP BY device_id;
device_id | count device_id | count
-----------+------- -----------+-------
1 | 3596
2 | 3596
3 | 3596 3 | 3596
4 | 3596
5 | 3596 5 | 3596
4 | 3596
2 | 3596
1 | 3596
(5 rows) (5 rows)
CALL recompress_chunk(:'CHUNK_NAME'::regclass); CALL recompress_chunk(:'CHUNK_NAME'::regclass);
@ -1527,32 +1525,27 @@ EXPLAIN (costs off) SELECT device_id, count(*)
FROM compression_insert FROM compression_insert
GROUP BY device_id; GROUP BY device_id;
QUERY PLAN QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------------------------------------
GroupAggregate HashAggregate
Group Key: _hyper_31_107_chunk.device_id Group Key: _hyper_31_107_chunk.device_id
-> Merge Append -> Append
Sort Key: _hyper_31_107_chunk.device_id
-> Custom Scan (DecompressChunk) on _hyper_31_107_chunk -> Custom Scan (DecompressChunk) on _hyper_31_107_chunk
-> Index Scan using compress_hyper_32_108_chunk__compressed_hypertable_32_device_id on compress_hyper_32_108_chunk -> Index Scan using compress_hyper_32_108_chunk__compressed_hypertable_32_device_id on compress_hyper_32_108_chunk
-> Sort
Sort Key: _hyper_31_109_chunk.device_id
-> Custom Scan (DecompressChunk) on _hyper_31_109_chunk -> Custom Scan (DecompressChunk) on _hyper_31_109_chunk
-> Index Scan using compress_hyper_32_110_chunk__compressed_hypertable_32_device_id on compress_hyper_32_110_chunk -> Index Scan using compress_hyper_32_110_chunk__compressed_hypertable_32_device_id on compress_hyper_32_110_chunk
-> Sort -> Index Only Scan using _hyper_31_109_chunk_compression_insert_device_id_time_idx on _hyper_31_109_chunk
Sort Key: _hyper_31_109_chunk.device_id (8 rows)
-> Seq Scan on _hyper_31_109_chunk
(13 rows)
SELECT device_id, count(*) SELECT device_id, count(*)
FROM compression_insert FROM compression_insert
GROUP BY device_id; GROUP BY device_id;
device_id | count device_id | count
-----------+------- -----------+-------
1 | 7192
2 | 7192
3 | 7192 3 | 7192
4 | 7192
5 | 7192 5 | 7192
4 | 7192
2 | 7192
1 | 7192
(5 rows) (5 rows)
CALL recompress_chunk(:'CHUNK_NAME'::regclass); CALL recompress_chunk(:'CHUNK_NAME'::regclass);
@ -1607,34 +1600,29 @@ EXPLAIN (costs off) SELECT device_id, count(*)
FROM compression_insert FROM compression_insert
GROUP BY device_id; GROUP BY device_id;
QUERY PLAN QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------------------------------------
GroupAggregate HashAggregate
Group Key: _hyper_31_107_chunk.device_id Group Key: _hyper_31_107_chunk.device_id
-> Merge Append -> Append
Sort Key: _hyper_31_107_chunk.device_id
-> Custom Scan (DecompressChunk) on _hyper_31_107_chunk -> Custom Scan (DecompressChunk) on _hyper_31_107_chunk
-> Index Scan using compress_hyper_32_108_chunk__compressed_hypertable_32_device_id on compress_hyper_32_108_chunk -> Index Scan using compress_hyper_32_108_chunk__compressed_hypertable_32_device_id on compress_hyper_32_108_chunk
-> Custom Scan (DecompressChunk) on _hyper_31_109_chunk -> Custom Scan (DecompressChunk) on _hyper_31_109_chunk
-> Index Scan using compress_hyper_32_110_chunk__compressed_hypertable_32_device_id on compress_hyper_32_110_chunk -> Index Scan using compress_hyper_32_110_chunk__compressed_hypertable_32_device_id on compress_hyper_32_110_chunk
-> Sort
Sort Key: _hyper_31_111_chunk.device_id
-> Custom Scan (DecompressChunk) on _hyper_31_111_chunk -> Custom Scan (DecompressChunk) on _hyper_31_111_chunk
-> Index Scan using compress_hyper_32_112_chunk__compressed_hypertable_32_device_id on compress_hyper_32_112_chunk -> Index Scan using compress_hyper_32_112_chunk__compressed_hypertable_32_device_id on compress_hyper_32_112_chunk
-> Sort -> Index Only Scan using _hyper_31_111_chunk_compression_insert_device_id_time_idx on _hyper_31_111_chunk
Sort Key: _hyper_31_111_chunk.device_id (10 rows)
-> Seq Scan on _hyper_31_111_chunk
(15 rows)
SELECT device_id, count(*) SELECT device_id, count(*)
FROM compression_insert FROM compression_insert
GROUP BY device_id; GROUP BY device_id;
device_id | count device_id | count
-----------+------- -----------+-------
1 | 10788
2 | 10788
3 | 10788 3 | 10788
4 | 10788
5 | 10788 5 | 10788
4 | 10788
2 | 10788
1 | 10788
(5 rows) (5 rows)
CALL recompress_chunk(:'CHUNK_NAME'::regclass); CALL recompress_chunk(:'CHUNK_NAME'::regclass);
@ -1689,36 +1677,31 @@ EXPLAIN (costs off) SELECT device_id, count(*)
FROM compression_insert FROM compression_insert
GROUP BY device_id; GROUP BY device_id;
QUERY PLAN QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------------------------------------
GroupAggregate HashAggregate
Group Key: _hyper_31_107_chunk.device_id Group Key: _hyper_31_107_chunk.device_id
-> Merge Append -> Append
Sort Key: _hyper_31_107_chunk.device_id
-> Custom Scan (DecompressChunk) on _hyper_31_107_chunk -> Custom Scan (DecompressChunk) on _hyper_31_107_chunk
-> Index Scan using compress_hyper_32_108_chunk__compressed_hypertable_32_device_id on compress_hyper_32_108_chunk -> Index Scan using compress_hyper_32_108_chunk__compressed_hypertable_32_device_id on compress_hyper_32_108_chunk
-> Custom Scan (DecompressChunk) on _hyper_31_109_chunk -> Custom Scan (DecompressChunk) on _hyper_31_109_chunk
-> Index Scan using compress_hyper_32_110_chunk__compressed_hypertable_32_device_id on compress_hyper_32_110_chunk -> Index Scan using compress_hyper_32_110_chunk__compressed_hypertable_32_device_id on compress_hyper_32_110_chunk
-> Custom Scan (DecompressChunk) on _hyper_31_111_chunk -> Custom Scan (DecompressChunk) on _hyper_31_111_chunk
-> Index Scan using compress_hyper_32_112_chunk__compressed_hypertable_32_device_id on compress_hyper_32_112_chunk -> Index Scan using compress_hyper_32_112_chunk__compressed_hypertable_32_device_id on compress_hyper_32_112_chunk
-> Sort
Sort Key: _hyper_31_113_chunk.device_id
-> Custom Scan (DecompressChunk) on _hyper_31_113_chunk -> Custom Scan (DecompressChunk) on _hyper_31_113_chunk
-> Index Scan using compress_hyper_32_114_chunk__compressed_hypertable_32_device_id on compress_hyper_32_114_chunk -> Index Scan using compress_hyper_32_114_chunk__compressed_hypertable_32_device_id on compress_hyper_32_114_chunk
-> Sort -> Index Only Scan using _hyper_31_113_chunk_compression_insert_device_id_time_idx on _hyper_31_113_chunk
Sort Key: _hyper_31_113_chunk.device_id (12 rows)
-> Seq Scan on _hyper_31_113_chunk
(17 rows)
SELECT device_id, count(*) SELECT device_id, count(*)
FROM compression_insert FROM compression_insert
GROUP BY device_id; GROUP BY device_id;
device_id | count device_id | count
-----------+------- -----------+-------
1 | 14384
2 | 14384
3 | 14384 3 | 14384
4 | 14384
5 | 14384 5 | 14384
4 | 14384
2 | 14384
1 | 14384
(5 rows) (5 rows)
CALL recompress_chunk(:'CHUNK_NAME'::regclass); CALL recompress_chunk(:'CHUNK_NAME'::regclass);
@ -1773,11 +1756,10 @@ EXPLAIN (costs off) SELECT device_id, count(*)
FROM compression_insert FROM compression_insert
GROUP BY device_id; GROUP BY device_id;
QUERY PLAN QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------------------------------------
GroupAggregate HashAggregate
Group Key: _hyper_31_107_chunk.device_id Group Key: _hyper_31_107_chunk.device_id
-> Merge Append -> Append
Sort Key: _hyper_31_107_chunk.device_id
-> Custom Scan (DecompressChunk) on _hyper_31_107_chunk -> Custom Scan (DecompressChunk) on _hyper_31_107_chunk
-> Index Scan using compress_hyper_32_108_chunk__compressed_hypertable_32_device_id on compress_hyper_32_108_chunk -> Index Scan using compress_hyper_32_108_chunk__compressed_hypertable_32_device_id on compress_hyper_32_108_chunk
-> Custom Scan (DecompressChunk) on _hyper_31_109_chunk -> Custom Scan (DecompressChunk) on _hyper_31_109_chunk
@ -1786,25 +1768,21 @@ GROUP BY device_id;
-> Index Scan using compress_hyper_32_112_chunk__compressed_hypertable_32_device_id on compress_hyper_32_112_chunk -> Index Scan using compress_hyper_32_112_chunk__compressed_hypertable_32_device_id on compress_hyper_32_112_chunk
-> Custom Scan (DecompressChunk) on _hyper_31_113_chunk -> Custom Scan (DecompressChunk) on _hyper_31_113_chunk
-> Index Scan using compress_hyper_32_114_chunk__compressed_hypertable_32_device_id on compress_hyper_32_114_chunk -> Index Scan using compress_hyper_32_114_chunk__compressed_hypertable_32_device_id on compress_hyper_32_114_chunk
-> Sort
Sort Key: _hyper_31_115_chunk.device_id
-> Custom Scan (DecompressChunk) on _hyper_31_115_chunk -> Custom Scan (DecompressChunk) on _hyper_31_115_chunk
-> Index Scan using compress_hyper_32_116_chunk__compressed_hypertable_32_device_id on compress_hyper_32_116_chunk -> Index Scan using compress_hyper_32_116_chunk__compressed_hypertable_32_device_id on compress_hyper_32_116_chunk
-> Sort -> Index Only Scan using _hyper_31_115_chunk_compression_insert_device_id_time_idx on _hyper_31_115_chunk
Sort Key: _hyper_31_115_chunk.device_id (14 rows)
-> Seq Scan on _hyper_31_115_chunk
(19 rows)
SELECT device_id, count(*) SELECT device_id, count(*)
FROM compression_insert FROM compression_insert
GROUP BY device_id; GROUP BY device_id;
device_id | count device_id | count
-----------+------- -----------+-------
1 | 17980
2 | 17980
3 | 17980 3 | 17980
4 | 17980
5 | 17980 5 | 17980
4 | 17980
2 | 17980
1 | 17980
(5 rows) (5 rows)
CALL recompress_chunk(:'CHUNK_NAME'::regclass); CALL recompress_chunk(:'CHUNK_NAME'::regclass);

2
tsl/test/sql/compression_ddl.sql
View File

@ -642,7 +642,7 @@ AND time <= '2000-01-05 23:55:00+0';
-- force index scans to check index mapping -- force index scans to check index mapping
-- this verifies that we are actually using compressed chunk index scans -- this verifies that we are actually using compressed chunk index scans
-- currently we cannot use indexes on uncompressed chunks due to a bug: -- previously we could not use indexes on uncompressed chunks due to a bug:
-- https://github.com/timescale/timescaledb/issues/5432 -- https://github.com/timescale/timescaledb/issues/5432
-- --
-- this check basically makes sure that the indexes are built properly -- this check basically makes sure that the indexes are built properly