1 Release published by 1 person

• apache-iceberg-1.9.1 Apache Iceberg 1.9.1

  published May 28, 2025

26 Pull requests merged by 16 people

• Spark: clean up obsoleted/unused writer classes

  #13193 merged May 31, 2025

• Spark 3.5: Update to Spark 3.5.6

  #13142 merged May 30, 2025

• Make PartitionSpec.Builder::identity public

  #12975 merged May 29, 2025

• AWS: Configure Default s3Async credentials the same as s3

  #13132 merged May 29, 2025

• Add Variant toString for time/nano-timestamps type

  #13151 merged May 29, 2025

• Build: Bump kafka from 3.9.0 to 3.9.1

  #13145 merged May 29, 2025

• Spec: Mark version 3 as completed

  #13175 merged May 29, 2025

• Core: Avoid table corruption from 409 on self conflicts after 5xx retries by throwing CommitStateUnknown

  #12818 merged May 29, 2025

• Core: Fix incremental compute of partition stats for various edge cases

  #13163 merged May 29, 2025

• Site: Add Versioned Docs for 1.9.1

  #13172 merged May 28, 2025

• Docs: Adds 1.9.1 Versioned JavaDocs

  #13173 merged May 28, 2025

• Site: Updates for 1.9.1 Release

  #13176 merged May 28, 2025

• Infra: Set 1.9.1 to Latest Release

  #13177 merged May 28, 2025

• Spark 3.4: streaming-skip-overwrite-snapshots fix

  #13168 merged May 28, 2025

• Docs, Flink: Fix equality fields requirement in upsert mode

  #13127 merged May 28, 2025

• Flink 1.19, 1.20: Remove the MiniClusterWithClientResource dependency

  #13165 merged May 28, 2025

• REST Spec: Add row lineage fields

  #13010 merged May 27, 2025

• Core, AWS, Spark: use loopback address explicitly in tests to work in more restrictive firewall env on dev machines

  #13101 merged May 27, 2025

• Build: Bump com.google.cloud:libraries-bom from 26.60.0 to 26.61.0

  #13146 merged May 27, 2025

• Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.4.4 to 5.5

  #13147 merged May 27, 2025

• Build: Bump software.amazon.awssdk:bom from 2.31.45 to 2.31.50

  #13148 merged May 27, 2025

• Build: Bump com.palantir.gradle.gitversion:gradle-git-version from 3.2.0 to 3.3.0

  #13144 merged May 27, 2025

• Build: Upgrade to Gradle 8.14.1

  #13149 merged May 27, 2025

• Docs: add Tinybird to the list of vendors and blog posts

  #13128 merged May 26, 2025

• AWS: Close the S3SeekableInputStreamFactory before removing from cache

  #12891 merged May 26, 2025

• Flink: Backport fix npe in TaskResultAggregator when job recovery to Flink 1.19 and 1.20

  #13140 merged May 26, 2025

27 Pull requests opened by 17 people

• OpenAPI: Add missing schema field for TableMetadata

  #13152 opened May 26, 2025

• Core: Don't copy stats of delete files in ManifestGroup

  #13161 opened May 27, 2025

• spark 4.0 : SPJ : add hour to day reducer

  #13166 opened May 28, 2025

• spark 4.0: SPJ: add bucket reducer using gcd

  #13167 opened May 28, 2025

• Fix(catalog): Only remove metadata files if TableOp created a new one

  #13169 opened May 28, 2025

• Spec: Update common v2,v3 table headers as v2+

  #13181 opened May 29, 2025

• chore: Update status.md

  #13182 opened May 29, 2025

• AWS: Refactor S3FileIOProperties to use common builder interface

  #13183 opened May 29, 2025

• API, Core: Rename RowDelta deleteFile to removeRows

  #13184 opened May 29, 2025

• Azure: KeyManagementClient implementation for Azure Key Vault

  #13186 opened May 30, 2025

• Spark: Iceberg_Spark Add delegation token for HiveCatalog

  #13187 opened May 30, 2025

• Spec: Add DV information in overview

  #13189 opened May 30, 2025

• [REST] Add option to configure TLS settings in REST client

  #13190 opened May 30, 2025

• [CORE][REST]: Add context aware response parsing

  #13191 opened May 30, 2025

• Fixed OAuth2Util

  #13192 opened May 30, 2025

• Docs: Add docs for Spark SQL Iceberg transform functions (#13156)

  #13194 opened May 31, 2025

• Support for TIME, TIMESTAMPNTZ_NANO, UUID types in Inclusive Metrics Evaluator #13157

  #13195 opened May 31, 2025

• Core: Remove deprecated left code in ORC and MetricsContext

  #13197 opened May 31, 2025

• Build: Bump junit-platform from 1.12.2 to 1.13.0

  #13198 opened Jun 1, 2025

• Build: Bump testcontainers from 1.21.0 to 1.21.1

  #13199 opened Jun 1, 2025

• Build: Bump net.snowflake:snowflake-jdbc from 3.24.0 to 3.24.2

  #13200 opened Jun 1, 2025

• Build: Bump com.azure:azure-sdk-bom from 1.2.31 to 1.2.35

  #13201 opened Jun 1, 2025

• Build: Bump io.delta:delta-standalone_2.12 from 3.3.1 to 3.3.2

  #13202 opened Jun 1, 2025

• Build: Bump calcite from 1.39.0 to 1.40.0

  #13203 opened Jun 1, 2025

• Build: Bump junit from 5.12.2 to 5.13.0

  #13204 opened Jun 1, 2025

• Build: Bump io.delta:delta-spark_2.12 from 3.3.1 to 3.3.2

  #13205 opened Jun 1, 2025

• Build: Bump software.amazon.awssdk:bom from 2.31.50 to 2.31.54

  #13206 opened Jun 1, 2025

14 Issues closed by 7 people

• AWS: DefaultAwsClientFactory s3 / s3Async credentials are not equivalent

  #13131 closed May 29, 2025

• Incrementally computing partition stats can miss deleted files

  #13155 closed May 29, 2025

• feature to add timestamp in data and metadata file names in iceberg

  #12935 closed May 29, 2025

• ERROR AppendDataExec: Data source write support IcebergBatchWrite(xxx) aborting

  #11562 closed May 29, 2025

• When `distribution-mode` = RANGE, Flink Stop-with-savepoint operation will definitely fail

  #11556 closed May 29, 2025

• [Spark Integration Tests] TestCreateTable::testCreateTableCommitProperties won't work on RESTCatalog

  #11554 closed May 29, 2025

• Storage Partitioned Join (SPJ) fails when >2 tables are joined

  #10450 closed May 28, 2025

• Flink Merge On Read Behavior? Equality & Positional Deletes

  #11535 closed May 28, 2025

• Direct memory leaks when reading parquet files containing interleaving plain/dictionary pages

  #11533 closed May 28, 2025

• Equality delete lost after compact data files

  #10312 closed May 28, 2025

• Modify REST Tests to Bind Loopback instead of Localhost

  #13097 closed May 27, 2025

• Spark 4.0

  #13162 closed May 27, 2025

• How to run streaming upserts and maintenance simultaneously?

  #11530 closed May 27, 2025

• ADLSFileIO cache DefaultAzureCredentials?

  #11523 closed May 27, 2025

17 Issues opened by 15 people

• Move metadata into the catalog, like DuckLake

  #13196 opened May 31, 2025

• Spark Streaming connector read initial snapshot of iceberg table

  #13188 opened May 30, 2025

• Support DV for partition stats

  #13180 opened May 29, 2025

• Append-only option for Spark incremental append read that throws

  #13179 opened May 28, 2025

• Build: Fix errorprone warnings

  #13178 opened May 28, 2025

• S3 client for storage path not available

  #13174 opened May 28, 2025

• ERROR: Procedure system.refresh_table not found

  #13171 opened May 28, 2025

• ClassCastException in RowDataWrapper.java

  #13170 opened May 28, 2025

• Nested field IDs in user-defined schema are reassigned during table creation, causing query failures in external engines

  #13164 opened May 27, 2025

• Literals.LongLiteral conversion issue to TimestampNanoLiteral

  #13160 opened May 27, 2025

• Flink: add IcebergHiveConnectorDelegationTokenProvider for HiveCatalog

  #13159 opened May 27, 2025

• Add IcebergHiveConnectorDelegationTokenProvider for Iceberg

  #13158 opened May 27, 2025

• Support for Variant PhysicalType.TIME, TIMESTAMPNTZ_NANO, TIMESTAMPTZ_NANO & UUID types in InclusiveMetricsEvaluator

  #13157 opened May 26, 2025

• Add docs of Spark SQL functions for Iceberg transforms

  #13156 opened May 26, 2025

• Add iam-policy-builder into iceberg-aws-bundle

  #13154 opened May 26, 2025

• Column Stats Improvements

  #13153 opened May 26, 2025

• JDBCMetricReporter to support governance and compliance use cases

  #13150 opened May 25, 2025

69 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Flink: Support compact in iceberg sink v2

  #12979 commented on May 30, 2025 • 52 new comments

• SPARK: Remove dependency on hadoop's filesystem class from remove orphan files

  #12254 commented on May 30, 2025 • 16 new comments

• Proposal: IRC Events endpoint

  #12584 commented on May 31, 2025 • 11 new comments

• Core: Add basic classes for writing table format-version 4

  #13123 commented on May 29, 2025 • 8 new comments

• Core: Fix numeric overflow of timestamp nano literal

  #11775 commented on May 30, 2025 • 6 new comments

• Hive: Throw exception for when listing a non-existing namespace

  #13130 commented on May 30, 2025 • 6 new comments

• API: Follow up on adding Variant data type to implement sanitizing for Variant #11479

  #13137 commented on May 31, 2025 • 5 new comments

• Spark 3.5, Arrow: Support for Row lineage when using the Parquet Vectorized reader

  #12928 commented on May 26, 2025 • 5 new comments

• Docs: add column descriptions for entries metadata table

  #13104 commented on May 28, 2025 • 5 new comments

• Core: Make pageToken query parameter optional

  #13129 commented on May 31, 2025 • 4 new comments

• Flink: Dynamic Iceberg Sink: Add table update code for schema comparison and evolution

  #13032 commented on May 30, 2025 • 4 new comments

• Flink: Migrate Flink TableSchema for IcebergSource

  #13072 commented on May 31, 2025 • 4 new comments

• Flink: Dynamic Iceberg Sink Contribution

  #12424 commented on May 28, 2025 • 3 new comments

• AWS: Fix DynamoDB and Glue integration test failures

  #12718 commented on May 30, 2025 • 3 new comments

• Spark: Fix row lineage inheritance for distributed planning

  #13061 commented on May 29, 2025 • 2 new comments

• Flink: port range distribution to v2 iceberg sink

  #12071 commented on May 29, 2025 • 2 new comments

• AWS: update test cases to verify credentials for the prefixed S3 client

  #13118 commented on May 26, 2025 • 2 new comments

• Reduce code duplication in VectorizedParquetDefinitionLevelReader

  #11661 commented on May 31, 2025 • 2 new comments

• Spark: Add 'skip_file_list' option to RewriteTablePathProcedure for optional file-list generation

  #12844 commented on May 28, 2025 • 1 new comment

• fix(Catalog): Handle NotFound exception for missing metadata file

  #13143 commented on May 28, 2025 • 1 new comment

• AWS: Refactor DynamoDB and Glue properties into separated properties classes

  #12722 commented on May 31, 2025 • 0 new comments

• Core: Fix a cast that is too narrow

  #12743 commented on May 28, 2025 • 0 new comments

• Core, Data: File Format API interfaces

  #12774 commented on May 28, 2025 • 0 new comments

• Docs: Remove obsolete version attribute to avoid confusion

  #13139 commented on May 27, 2025 • 0 new comments

• pluggable routers

  #12859 commented on May 29, 2025 • 0 new comments

• Spark: Avoid closing deserialized copies of shared resources like FileIO

  #12868 commented on May 31, 2025 • 0 new comments

• AVRO: Support UUID logical type on string fields in Avro schema

  #12877 commented on May 26, 2025 • 0 new comments

• AWS: KeyManagementClient implementation that works with AWS KMS

  #13136 commented on May 28, 2025 • 0 new comments

• Arrow, AWS, Azure, Core, GCP, Hive, Kafka, Snowflake: Rename test classes to use Test as prefix instead of suffix

  #12879 commented on May 26, 2025 • 0 new comments

• Core: Implement source-ids to deal with multi arguments transforms

  #12897 commented on May 30, 2025 • 0 new comments

• Error handling with DLQ support

  #13135 commented on May 30, 2025 • 0 new comments

• Website: Add PyIceberg, IcebergRust, and IcebergGo to top nav bar

  #12950 commented on May 26, 2025 • 0 new comments

• Fix Issue #13064

  #13113 commented on May 31, 2025 • 0 new comments

• API: Compute truncate decimal result precision based on lowest value bound

  #12969 commented on May 29, 2025 • 0 new comments

• Introduce MetricsMaxInferredColumnDefaultsStrategy

  #13039 commented on May 30, 2025 • 0 new comments

• Build: Run SPARK CI on include path patterns

  #13033 commented on May 26, 2025 • 0 new comments

• Part 1: Support Scan Planning in Rest Client

  #13004 commented on May 30, 2025 • 0 new comments

• Spark: Make maxRecordPerMicrobatch a soft limit

  #12988 commented on May 28, 2025 • 0 new comments

• OpenAPI spec missing `schema` field for `TableMetadata`

  #13103 commented on May 26, 2025 • 0 new comments

• Relative Path Support In Table Spec

  #13141 commented on May 26, 2025 • 0 new comments

• java.io.IOException: can not read class org.apache.iceberg.shaded.org.apache.parquet.format.PageHeader: Required field 'num_values' was not found in serialized data

  #11614 commented on May 27, 2025 • 0 new comments

• Iceberg Kafka Connector experiences a constant hanging lag for low-volume topics

  #11818 commented on May 27, 2025 • 0 new comments

• Iceberg + MinIO S3 - Invalid signature after 3 hours

  #13045 commented on May 28, 2025 • 0 new comments

• Running MERGE INTO with more than one WHEN condition fails if the number of columns in the target table is > 321

  #10294 commented on May 28, 2025 • 0 new comments

• INT96 timestamp is read as OffsetDateTime, not LocalDateTime

  #12266 commented on May 28, 2025 • 0 new comments

• Support build full-text and vector index for iceberg

  #12636 commented on May 29, 2025 • 0 new comments

• TableOperations.locationProvider is not respected by Spark

  #11527 commented on May 30, 2025 • 0 new comments

• [Arrow]Incorrect argument passed to setInitialCapacity in allocateVectorBasedOnTypeName method (should be count of values, not bytes)

  #11672 commented on May 30, 2025 • 0 new comments

• Kafka Connect: Add dead letter queue support

  #10840 commented on May 30, 2025 • 0 new comments

• Implement error handling mechanism for DataException

  #12992 commented on May 30, 2025 • 0 new comments

• Spark, Avro and Iceberg Timestamp, Timestamp NTZ definition need more clarifications

  #12751 commented on May 30, 2025 • 0 new comments

• Spark: add IcebergHiveConnectorDelegationTokenProvider for HiveCatalog

  #13116 commented on May 30, 2025 • 0 new comments

• Kafka Connect Sporadic Commit Delay

  #11796 commented on May 31, 2025 • 0 new comments

• Handle leftover deprecations

  #13054 commented on May 31, 2025 • 0 new comments

• Spark: Support rewrite file with z-order for nested Struct type

  #9818 commented on May 26, 2025 • 0 new comments

• Core: HadoopFileIO to support bulk delete through the Hadoop Filesystem APIs

  #10233 commented on May 30, 2025 • 0 new comments

• Kafka Connect: Add mechanisms for routing records by topic name

  #11623 commented on Jun 1, 2025 • 0 new comments

• Core: Interface based DataFile reader and writer API

  #12298 commented on May 28, 2025 • 0 new comments

• Support In and notIn operators in ParquetFilters.ConvertFilterToParquet

  #12449 commented on May 30, 2025 • 0 new comments

• Spark-3.5: Add spark action to compute partition stats

  #12450 commented on May 29, 2025 • 0 new comments

• Core/REST: generify AuthSessionCache

  #12562 commented on May 31, 2025 • 0 new comments

• Core: introduce shared authentication refresh executor

  #12563 commented on May 31, 2025 • 0 new comments

• Parquet: Fix column pruning for deeply nested fields

  #12634 commented on May 29, 2025 • 0 new comments

• Spark: when doing rewrite_data_files, check for partitioning schema compatibility

  #12651 commented on May 30, 2025 • 0 new comments

• API, Core: Geospatial bounds and spatial predicates

  #12667 commented on May 29, 2025 • 0 new comments

• Build: Bump org.apache.hadoop.thirdparty:hadoop-shaded-guava from 1.3.0 to 1.4.0

  #12684 commented on Jun 1, 2025 • 0 new comments

• AWS: Support StaticCredentialsProvider in DefaultAwsClientFactory

  #12695 commented on May 31, 2025 • 0 new comments

• Spark 3.5: Support case sensitive in replace where statement

  #12706 commented on May 30, 2025 • 0 new comments

• Build and test hive-metastore with Hive 2, 3 and 4 with a single source set

  #12721 commented on May 28, 2025 • 0 new comments