Make restore support multi-project #131661
Conversation
|
Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination) |
|
I will be updating tests like |
| params -> Iterators.concat(Iterators.single((builder, ignored) -> builder.field("id", projectMetadata.id()))) | ||
| params -> Iterators.concat( | ||
| Iterators.single((ToXContent) (builder, p) -> builder.field("id", projectMetadata.id())), | ||
| projectMetadata.toXContentChunked(params) | ||
| ) |
There was a problem hiding this comment.
Fixing a silly bug from #130000 ...
| if (metadata.persistentSettings() != null) { | ||
| assert deserializeProjectMetadata == false || metadata.persistentSettings().isEmpty() |
There was a problem hiding this comment.
should this be (deserializeProjectMetadata == false && projectId.equals(DEFAULT))?
There was a problem hiding this comment.
also I assume we're not restoring settings from the metadata for the project since they're not in metadata?
There was a problem hiding this comment.
The metadata from repository is constructed by repository.getSnapshotGlobalMetadata which reads only the ProjectMetadata in MP mode. The Metadata in this case is simply an emtpy wrapper around it so that it does not break existing methods and can work in both modes. Therefore, if the metadata has non-null settings, it is either (1) non-MP or (2) it's empty due to in MP. Both are asserted here. Btw, I think metadata.persistentSettings() is actually never null. But that's the existing code and I'd like to leave it as is.
we're not restoring settings from the metadata for the project since they're not in metadata
Yes. Project level settings are still being defined. As is right now, all project settings are file-based settings. They live in ProjectStateRegistry and we don't restore file-based stuff.
There was a problem hiding this comment.
eeehhhm, that's all good yeah, but not sure I understand why the suggested more refined assertion wouldn't work here? deserializeProjectMetadata == false means non-MP, right?
There was a problem hiding this comment.
Oh OK I see what you mean. Your suggestion is to tighten the first part of the overall assertion not replace both parts. Yes that makes sense. Pushed 6667aa4
Yeah. That's what it does mostly. And you have already spotted the part that requires a bit more thinking which is restoring ProjectMetadata based on the request and ensure we restore only the project and do not affect the cluster. |
…king * upstream/main: (90 commits) Register a blob cache long counter metric for total evicted regions (elastic#131862) Move plan attribute resolution to its own component (elastic#131830) Make restore support multi-project (elastic#131661) Use logically more correct expression (elastic#131869) [ES|QL] Change equals and hashcode for ConstantNullBlock (elastic#131817) Update `TransportVersion` to support a new model (elastic#131488) Correct slow log user for RCS 2.0 (elastic#130140) Revert "Remove 8.17 from dev branches" Mute org.elasticsearch.compute.aggregation.ValuesBytesRefGroupingAggregatorFunctionTests testSomeFiltered elastic#131878 Remove 8.17 from dev branches Revert "CompressorFactory.compressor (elastic#131655)" (elastic#131866) Add fast path for single value in VALUES aggregator (elastic#130510) Resolve inference release tests failing due to missing feature flag (elastic#131841) [Docs] Replace placeholder URLs (elastic#131309) CompressorFactory.compressor (elastic#131655) add availability info for speed loading setting (elastic#131714) [Logstash] Move `elastic_integration` plugin usage to ES logstash-bridge. (elastic#131486) Migrate x-pack-enrich legacy rest tests to new test framework (elastic#131743) Fix plugin example test failures due to deprecation warning (elastic#131819) Remove deprecated function isNotNullAndFoldable (elastic#130944) ...
…-tracking * upstream/main: (106 commits) Pipelines: Add `created_date` and `modified_date` (elastic#130847) add thread pool change availability (elastic#131734) Add failure store availability info / and port over privileges (elastic#131729) add availability information for ssl handshake timeout settings (elastic#131786) add availability information for rescore_vector (elastic#131710) add availability to oversample value of 0 (elastic#131707) clarify hnsw filter heuristic setting availability (elastic#131715) add availability info for default heap dump path change (elastic#131713) clarify default algorithms per stack version (elastic#131728) Refine error messages in `Fork` for correctness and clarity. (elastic#131701) [ES|QL] Replace RoundTo linear search evaluator with manual evaluators (elastic#131733) ESQL: Fix buildParams in tests with --configuration-cache (elastic#131826) Unmute `CrossClusterEsqlRCS2EnrichUnavailableRemotesIT#testEsqlEnrichWithSkipUnavailable` (elastic#131916) Allow templates for `.chat-*` index template (elastic#131914) ESQL: Fix NPE on empty to_lower/to_upper call (elastic#131917) Fix score computation in ES91Int4VectorsScorer (elastic#131905) Register a blob cache long counter metric for total evicted regions (elastic#131862) Move plan attribute resolution to its own component (elastic#131830) Make restore support multi-project (elastic#131661) Use logically more correct expression (elastic#131869) ...
This PR makes the restore process project aware and unmute relevant tests. The later requires TransportRecoveryAction to be project aware which is done in this PR as well.
Relates: #130000
Resolves: ES-10228