pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

URL: http://github.com/ClickHouse/ClickHouse/pull/112333

quests-be6017ec12798e73.css" /> Backport #109768 to 25.8: Mask session_token and Google ADC secrets in the explicit-url S3 form by robot-ch-test-poll2 · Pull Request #112333 · ClickHouse/ClickHouse · GitHub
Skip to content

Backport #109768 to 25.8: Mask session_token and Google ADC secrets in the explicit-url S3 form - #112333

Merged
Algunenano merged 7 commits into
25.8from
backport/25.8/109768
Aug 3, 2026
Merged

Backport #109768 to 25.8: Mask session_token and Google ADC secrets in the explicit-url S3 form#112333
Algunenano merged 7 commits into
25.8from
backport/25.8/109768

Conversation

@robot-ch-test-poll2

@robot-ch-test-poll2 robot-ch-test-poll2 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Original pull-request #109768
Cherry-pick pull-request #112289

This pull-request is a last step of an automated backporting.
Treat it as a standard pull-request: look at the checks and resolve conflicts.
Merge it only if you intend to backport changes to the target branch, otherwise just close it.

The PR source

The PR is created in the CI job

Version info

  • Merged into: 25.8.29.47

@robot-ch-test-poll2 robot-ch-test-poll2 added pr-backport Changes, backported to release branch. Do not use manually - automated use only! pr-bugfix Pull request with bugfix, not backported by default labels Jul 28, 2026
@clickhouse-gh

clickhouse-gh Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Workflow [BackportPR], commit [c11e745]

QueryAnalyzer.cpp lost its transitive include of
<boost/algorithm/string/predicate.hpp> when the include of the old,
inline-bodied Parsers/FunctionSecretArgumentsFinder.h was swapped for
the new declarations-only Analyzer/FunctionSecretArgumentsFinderTreeNode.h,
breaking the pre-existing, unrelated boost::iequals ORDER BY ALL check.
Added the include explicitly, matching master's QueryAnalyzer.cpp.

The new 04510_s3_explicit_url_named_secret_mask test also exercised
use_environment_credentials on the S3 database engine, which this
branch's DatabaseS3 does not support; dropped that assertion.

#112333
…tion

A duplicated key-value argument such as
s3('http://...', format = 'CSV', format = 'TSV') is user-controlled input,
but parseKeyValueArguments threw LOGICAL_ERROR on the duplicate-key emplace.
LOGICAL_ERROR aborts debug builds and is treated as a critical failure by
stress and fuzzer checks, so a user typo must not produce it. On this branch
the helper still lives in Storages/ObjectStorage/S3/Configuration.cpp
(pre-dates the shared ObjectStorage/Utils.cpp refactor).

Backport of 4939661 (already applied to the
26.4 private backport), needed here too since the new secret-masking tests
from #109768 exercise this exact duplicate-key path.

Closes: #104267
findS3FunctionSecretArguments only special-cased the named-collection
form for non-cluster functions (isNamedCollectionName(0)), so
s3Cluster('cluster', named_collection, ...) fell through to the
positional url/secret classification meant for the literal-url form,
leaving the named collection's secret_access_key (and session_token,
Google ADC secrets) unmasked. Ported master's findS3NamedCollectionSecretArguments
helper and its use in findS3FunctionSecretArguments, findS3TableEngineSecretArguments
and findS3DatabaseSecretArguments, matching the fully merged state of #109768.

This is what test_mask_sensitive_info/test.py::test_table_functions_object_storage_cluster
caught on this backport (amd_asan and amd_tsan integration runs).

#112333


BACKUP TABLE ... TO S3('https://user:pass@localhost:11111/x?...', 'ak', 'sak')
relied on the one-character bucket name being rejected by S3 URI
validation before any network access, matching master's comment for
this case. On this branch BACKUP's S3 destination locator parses the
url without going through S3::URI: the userinfo confuses its ad-hoc
host:port parsing into a bogus endpoint ("user:0"), and the resulting
request is misclassified as retryable, so BackupsWorker retries it
(s3_retry_attempts = 1000) instead of failing fast. The client hung
for the full 600s test timeout, and the background backup task then
kept the server from shutting down cleanly.

The masking behavior itself already works (the query is logged with
the userinfo and signature correctly hidden before the hang); only
the "fails before any network access" assumption doesn't hold here.

#112333
findTableEngineSecretArguments's S3-family dispatch (S3, COSN, OSS,
DeltaLake, Hudi, Iceberg, IcebergS3, S3Queue) was missing GCS and
DeltaLakeS3, even though both exist as table engines on this branch
(StorageObjectStorageDefinitions.h) and are included in master's
merged state of #109768. Engines that fell through this dispatch got
no masking at all: SHOW CREATE TABLE and the query log showed their
secret_access_key in plaintext, as caught by
04510_s3_explicit_url_named_secret_mask's ENGINE = GCS(...) case.

#112333
The `Backup` database engine (`ENGINE = Backup(database_name, S3(...))`)
reconstructs its nested `S3(...)` destination for logging, since that
destination isn't itself recognized as an S3 engine when the formatter
recurses into an argument rather than `function`. This whole function,
its dispatch entry, the `quote_replacement` field it needs on `Result`
(to emit the reconstructed call as raw SQL text instead of a quoted
string), and the matching ASTFunction.cpp formatting change all exist
in #109768's own commits (42bda2e, ced7e82) and in master's
merged state, but were dropped entirely in this cherry-pick: on this
branch, `CREATE DATABASE ... ENGINE = Backup('db', S3('url','ak','sk'))`
showed the secret access key in plaintext in SHOW CREATE and the query
log, and nothing in the trimmed 25.8 test file caught it.

Ported the function, wiring, and the corresponding SQL/reference test
cases from 04510_s3_explicit_url_named_secret_mask (skipping the two
cases with a userinfo-bearing localhost:11111 URL, which hit the same
BACKUP-destination-parsing hang already worked around for the plain
BACKUP TO S3 case), plus the integration test case in
test_mask_sensitive_info::test_create_database.

#112333
@Algunenano
Algunenano merged commit 0b2adda into 25.8 Aug 3, 2026
30 of 35 checks passed
@Algunenano
Algunenano deleted the backport/25.8/109768 branch August 3, 2026 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-backport Changes, backported to release branch. Do not use manually - automated use only! pr-bugfix Pull request with bugfix, not backported by default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

pFad - Phonifier reborn

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.





Check this box to remove all script contents from the fetched content.



Check this box to remove all images from the fetched content.


Check this box to remove all CSS styles from the fetched content.


Check this box to keep images inefficiently compressed and original size.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy