pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

URL: http://github.com/ClickHouse/ClickHouse/pull/112331

ubassets.com/assets/pull-requests-be6017ec12798e73.css" /> Fix out-of-bounds read in SerializationReplicated on the native protocol by Avogar · Pull Request #112331 · ClickHouse/ClickHouse · GitHub
Skip to content

Fix out-of-bounds read in SerializationReplicated on the native protocol - #112331

Open
Avogar wants to merge 1 commit into
masterfrom
fix-replicated-serialization-oob-index
Open

Fix out-of-bounds read in SerializationReplicated on the native protocol#112331
Avogar wants to merge 1 commit into
masterfrom
fix-replicated-serialization-oob-index

Conversation

@Avogar

@Avogar Avogar commented Jul 28, 2026

Copy link
Copy Markdown
Member

Changelog category (leave one):

  • Critical Bug Fix (crash, data loss, RBAC)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Fix out-of-bounds read in SerializationReplicated on the native protocol on corrupted indexes data.

`SerializationReplicated::deserializeBinaryBulkWithMultipleStreams` read a
client-supplied replicated-index buffer from the native TCP protocol and
validated only the index row count, not the index values. Every
`ColumnReplicated` accessor then dereferences `nested_column[index]` without
a bound, so a crafted `REPLICATED`-serialized column whose indexes point past
the nested ("elements") column produces an out-of-bounds read when the column
is materialized.

Validate every deserialized index against the number of nested elements
(throwing `INCORRECT_DATA`) before attaching the indexes, mirroring the
per-discriminator check already done in `SerializationVariant`.

Reachable only over the native TCP protocol (negotiated revision >=
`DBMS_MIN_REVISION_WITH_REPLICATED_SERIALIZATION`); the format layer builds
`NativeReader` with `server_revision = 0`, so `REPLICATED` is never read there.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@clickhouse-gh

clickhouse-gh Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Workflow [PR], commit [aca7466]


AI Review

Summary

This PR hardens SerializationReplicated against one malformed REPLICATED native-TCP payload shape by validating each replicated index before attaching it, and the new raw-protocol regression covers that specific out-of-range-index case. I found one remaining blocker: the deserializer still trusts the advertised nested element count and never verifies that the nested subcolumn actually read that many rows, so a truncated payload can still leave ColumnReplicated with valid-looking indexes that point past the real nested data.

Findings
❌ Blockers
  • [src/DataTypes/Serializations/SerializationReplicated.cpp:292] SerializationReplicated now validates indexes against the declared num_elements, but it still never checks that nested->deserializeBinaryBulkWithMultipleStreams produced num_elements rows. Primitive bulk readers such as SerializationNumber::deserializeBinaryBulk short-read on EOF instead of throwing, and NativeReader::readData only validates the top-level column->size(); for ColumnReplicated that is just the index count. A client can therefore send in-range indexes, claim num_elements = N, truncate the elements payload, and still hand an invalid ColumnReplicated to the insert pipeline, preserving the out-of-bounds read on later materialization. Suggested fix: after nested deserialization, verify column_replicated.getNestedColumn()->size() == num_elements and throw INCORRECT_DATA on mismatch before returning.
Tests
  • ⚠️ Add a regression for the truncated-elements case: advertise num_elements > actual_nested_values, keep all indexes < num_elements, and end the payload after the short nested data. The current new test only exercises out-of-range indexes, not the still-accepted short-read path.
Final Verdict

Changes requested.

@clickhouse-gh clickhouse-gh Bot added pr-critical-bugfix pr-must-backport Pull request should be backported intentionally. Use this label with great care! labels Jul 28, 2026
checkDeserializedIndexes(*indexes, size_of_indexes_type, num_elements);
column_replicated.getIndexes().attachIndexes(std::move(indexes));

nested->deserializeBinaryBulkWithMultipleStreams(column_replicated.getNestedColumn(), 0, num_elements, settings, state, cache);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checkDeserializedIndexes only proves the replicated indexes are < num_elements; we still never prove that nested->deserializeBinaryBulkWithMultipleStreams actually produced num_elements rows. That matters because primitive bulk readers such as SerializationNumber::deserializeBinaryBulk short-read on EOF instead of throwing, and NativeReader::readData only checks the top-level column->size(). For ColumnReplicated that size is the index count, not the nested column size, so a client can send in-range indexes, advertise num_elements = N, truncate the elements payload, and still hand an invalid ColumnReplicated to executor.push, preserving the out-of-bounds read when the block is materialized.

Please mirror the post-read consistency check that SerializationArray and SerializationVariant do here: after the nested deserialization, verify column_replicated.getNestedColumn()->size() == num_elements and throw INCORRECT_DATA on mismatch. A regression with valid indexes plus a truncated elements stream would cover the remaining case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-critical-bugfix pr-must-backport Pull request should be backported intentionally. Use this label with great care!

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

pFad - Phonifier reborn

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.





Check this box to remove all script contents from the fetched content.



Check this box to remove all images from the fetched content.


Check this box to remove all CSS styles from the fetched content.


Check this box to keep images inefficiently compressed and original size.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy