Esql dimension aware attributes #131463
Esql dimension aware attributes #131463
Conversation
Conflicts: docs/reference/query-languages/esql/kibana/docs/operators/is_null.md
Conflicts: x-pack/plugin/esql-core/src/main/java/org/elasticsearch/xpack/esql/core/expression/Attribute.java
| @@ -83,13 +123,8 @@ public static Reader<? extends EsField> getReader(String name) { | |||
| private final Map<String, EsField> properties; | |||
| private final String name; | |||
| private final boolean isAlias; | |||
| // Because the subclasses all reimplement serialization, this needs to be writeable from subclass constructors | |||
There was a problem hiding this comment.
(Comment was outdated - I ended up not reimplementing the serialization in the sub classes, but forgot to delete this in the previous PR)
| timeSeriesFieldType = timeSeriesFieldType.merge(EsField.TimeSeriesFieldType.fromIndexFieldCapabilities(fc)); | ||
| } catch (IllegalArgumentException e) { | ||
| return new InvalidMappedField(name, e.getMessage()); | ||
| } |
There was a problem hiding this comment.
This is the core thing this PR does. As we are loading the fields, we collect their time series metadata, and merge it across indices. This is then made available to the field attribute.
| @@ -165,7 +165,6 @@ public void testFallbackIndicesOptions() throws Exception { | |||
| assertThat(cloned.clusterAlias(), equalTo(request.clusterAlias())); | |||
| assertThat(cloned.sessionId(), equalTo(request.sessionId())); | |||
| RemoteClusterPlan plan = cloned.remoteClusterPlan(); | |||
| assertThat(plan.plan(), equalTo(request.remoteClusterPlan().plan())); | |||
There was a problem hiding this comment.
Test was failing on this line because plans serialized with older transport versions are not equal to plans serialized with the new transport version. This is expected, since when translating from the old version, we default the ts metadata to unknown, while the current version serializes and deserializes an actual value.
I discussed this failure with @dnhatn , and we concluded this assert wasn't adding any value in this test.
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
| if (other != METRIC) { | ||
| return DIMENSION; | ||
| } | ||
| throw new IllegalStateException("Time Series Metadata conflict. Cannot merge [" + other + "] with [DIMENSION]."); |
There was a problem hiding this comment.
To confirm, we do not allow mixing dimensions and metrics, but we do allow NONE with dimensions. I think the GAUGE type is optional - should we treat it similarly to NONE?
Relates to #128621
This PR adds more building blocks for working with dimensions in ESQL. Specifically, it makes the TSDB metadata available on attributes.
I did not add new tests for this yet. That's on purpose, as we have more components we need before we can really wire this up the way we want to.