49 Pull requests merged by 18 people

• Core: Remove deprecated Util.blockLocations method and StructCopy class

  #12320 merged Feb 20, 2025

• Spark: Remove Spark 3.3 support

  #12279 merged Feb 20, 2025

• Bump versions in {LICENSE,NOTICE}

  #12337 merged Feb 20, 2025

• Parquet: Remove deprecated VectorizedReader.setRowGroupInfo and ParquetValueReader.setPageSource

  #12321 merged Feb 20, 2025

• [1.8.x] Build: Revert AWS SDK from 2.30.11 to 2.29.52

  #12339 merged Feb 20, 2025

• Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure

  #12319 merged Feb 19, 2025

• Docs: Add documentation for Rate limiting in Spark Structured Streaming

  #12217 merged Feb 19, 2025

• Docs: Fix link of catalog in terms.md

  #12326 merged Feb 19, 2025

• [1.8.x] Kafka: Pin Kafka-Connect version to fix integration tests

  #12341 merged Feb 19, 2025

• Kafka: Pin Kafka-Connect version to fix integration tests

  #12340 merged Feb 19, 2025

• [1.8.x] Parquet: Fix performance regression in reader init (#12305)

  #12338 merged Feb 19, 2025

• Checkstyle: Apply the same generic type naming rules to interfaces and classes

  #12333 merged Feb 19, 2025

• [1.8.x] Core: Adjust Jackson settings to handle large metadata json (#12224)

  #12330 merged Feb 19, 2025

• [1.8.x] Parquet: Fix performance regression in reader init (#12305)

  #12329 merged Feb 19, 2025

• Revert "Core: Serialize null when there is no current snapshot"

  #12312 merged Feb 19, 2025

• Fix: fix apache amoro ams doc pic ref

  #12332 merged Feb 19, 2025

• [1.8.x] Core: Fallback to GET requests for namespace/table/view exists checks

  #12328 merged Feb 19, 2025

• Core: Fallback to GET requests for namespace/table/view exists checks

  #12314 merged Feb 19, 2025

• [1.8.x] Revert "Core: Serialize null when there is no current snapshot"

  #12313 merged Feb 19, 2025

• Parquet: Fix performance regression in reader init

  #12305 merged Feb 19, 2025

• Docs: add apache amoro(incubating) with iceberg (#11965)

  #11966 merged Feb 19, 2025

• Parquet: Fix errorprone warning

  #12324 merged Feb 19, 2025

• Docs: Add rewrite-table-path in spark procedure

  #12115 merged Feb 19, 2025

• Parquet: Implement Variant readers

  #12139 merged Feb 18, 2025

• Docs: Refactor site navigation bar

  #12289 merged Feb 18, 2025

• Fix CI: Update tests with UnknownType from Required to Optional

  #12316 merged Feb 18, 2025

• Core: add variant type support

  #11831 merged Feb 18, 2025

• API: Fix TestInclusiveMetricsEvaluator notStartsWith tests

  #12303 merged Feb 18, 2025

• API: Reject unknown type for required fields and validate defaults

  #12302 merged Feb 18, 2025

• Core: Fix non-setting row-lineage from table properties on initial table creation

  #12307 merged Feb 18, 2025

• Build: Bump mkdocs-material from 9.6.3 to 9.6.4

  #12284 merged Feb 17, 2025

• OpenAPI: Add overwrite option when registering an iceberg table

  #12239 merged Feb 17, 2025

• Build: Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.21

  #12286 merged Feb 17, 2025

• Build: Bump io.netty:netty-buffer from 4.1.117.Final to 4.1.118.Final

  #12287 merged Feb 17, 2025

• Spark 3.5: Fix job description of RewriteTablePathSparkAction

  #12282 merged Feb 17, 2025

• Build: Bump datamodel-code-generator from 0.27.2 to 0.28.1

  #12290 merged Feb 16, 2025

• Minor: update Learn More to point to spark quickstart

  #12272 merged Feb 14, 2025

• OpenAPI: Add RemoveSchemas REST update type

  #12022 merged Feb 14, 2025

• Docker: Pin QEMU version temporarily

  #12262 merged Feb 14, 2025

• API: Deprecate NestedType.of in favor of builder

  #12227 merged Feb 14, 2025

• Spark: Fix assertion checks

  #12255 merged Feb 14, 2025

• Spark: Remove unused PruneColumnsWithReordering class

  #12258 merged Feb 14, 2025

• Core: Fix divide by zero when adjust split size

  #12201 merged Feb 13, 2025

• update site to include iceberg summit link

  #12256 merged Feb 13, 2025

• API, Core: Support default values in UpdateSchema

  #12211 merged Feb 13, 2025

• Core: Add InternalData read and write builders

  #12060 merged Feb 13, 2025

• Build: Clean up dependencies

  #12252 merged Feb 13, 2025

• Build: Bump Hive to 2.3.10

  #12253 merged Feb 13, 2025

• Core: Adjust Jackson settings to handle large metadata json

  #12224 merged Feb 13, 2025

31 Pull requests opened by 22 people

• SPARK: Remove dependency on hadoop's filesystem class from remove orphan files

  #12254 opened Feb 13, 2025

• Spark: support rewrite on specified target branch

  #12257 opened Feb 13, 2025

• Spark: Structured Streaming read limit support follow-up

  #12260 opened Feb 13, 2025

• Core: use ReachableFileCleanup when table has discontinuous snapshots

  #12261 opened Feb 13, 2025

• S3: Disable strong integrity checksums

  #12264 opened Feb 14, 2025

• Spark: Detect dangling DVs properly

  #12270 opened Feb 14, 2025

• AWS, AZURE: Move docker-based tests to integration test source

  #12274 opened Feb 14, 2025

• Spark-3.5: Add unit tests for ColumnarBatchUtil

  #12275 opened Feb 14, 2025

• List data and metadata directories instead of table root

  #12278 opened Feb 14, 2025

• Docs: Add warning about `snapshot_ids` arg in `expired_snapshots` procedure

  #12291 opened Feb 16, 2025

• Replace usages of Aws4Signer with AwsV4HttpSigner in REST SigV4

  #12295 opened Feb 17, 2025

• Add properties support for HadoopTables.load() (#12251)

  #12296 opened Feb 17, 2025

• WIP: Interface based FileFormat API

  #12298 opened Feb 17, 2025

• AWS: Integrate S3 analytics accelerator library

  #12299 opened Feb 17, 2025

• Fix IndexOutOfBounds exception in FileFormat#fromFileName

  #12301 opened Feb 17, 2025

• API: Move variant to API and add extract expression

  #12304 opened Feb 17, 2025

• Core: Interface changes for separating rewrite planner and runner

  #12306 opened Feb 18, 2025

• Path parameters should encode spaces as '%20' instead of '+'

  #12309 opened Feb 18, 2025

• API, Core: Update inclusive metrics evaluator for extract and transforms

  #12311 opened Feb 18, 2025

• Throw on `{write.folder-storage.path,write.object-storage.path}` properties

  #12315 opened Feb 18, 2025

• Wrap variant in PrimitiveHoder so serialization can result same instance

  #12317 opened Feb 18, 2025

• Core: Print un-pretty metadata files without whitespace

  #12318 opened Feb 18, 2025

• Use delimited column names in `CreateChangelogViewProcedure`

  #12322 opened Feb 18, 2025

• Parquet: Implement Variant writers

  #12323 opened Feb 18, 2025

• Spark: Infer partition spec in ADD_FILES procedure for FileTables than taking latest table spec

  #12327 opened Feb 19, 2025

• Spec: Add implementation note on `current-snapshot-id`

  #12334 opened Feb 19, 2025

• Core: Write `null` for `current-snapshot-id` for V3+

  #12335 opened Feb 19, 2025

• Core, Spark: Remove deprecated code for 1.9.0

  #12336 opened Feb 19, 2025

• Docs: Add Stackable to the Vendors page

  #12344 opened Feb 19, 2025

• API, Core: Add geometry and geography types support

  #12346 opened Feb 20, 2025

• WIP Parquet: Support reading/writing geometry and geography columns

  #12347 opened Feb 20, 2025

14 Issues closed by 5 people

• Do not override finalize

  #10901 closed Feb 20, 2025

• Partition spec mismatch when 'compatibility.snapshot-id-inheritance.enabled' is true

  #12273 closed Feb 19, 2025

• Nested column filter expression

  #12331 closed Feb 19, 2025

• Rest catalog: write.metadata.delete-after-commit set true not deleting expired metadata files

  #10894 closed Feb 19, 2025

• Some schema updates do not support dots inside a field name

  #10875 closed Feb 19, 2025

• Apache Flink not committing new snapshots to Iceberg Table

  #9089 closed Feb 18, 2025

• Incorrect Results and SIGSEGV on Read with Iceberg + PySpark + Nessie

  #12178 closed Feb 17, 2025

• Java doc link is not working

  #12166 closed Feb 17, 2025

• Field comments are not written for timestamp field

  #4212 closed Feb 17, 2025

• Define behavior of gc.enabled and location ownership

  #4159 closed Feb 17, 2025

• MERGE INTO TABLE is not supported temporarily.

  #10882 closed Feb 16, 2025

• Athena Iceberg does not delete orphan files

  #10878 closed Feb 16, 2025

• throw exception : InvalidOperationException(message:The following columns have types incompatible with the existing columns in their respective positions : idd1) when add column

  #3747 closed Feb 16, 2025

• Link LEARN MORE vom https://iceberg.apache.org/about/ runs into Not Found

  #12265 closed Feb 14, 2025

18 Issues opened by 14 people

• does iceberg support spark connect ?

  #12345 opened Feb 20, 2025

• Limit the delete file/records

  #12343 opened Feb 19, 2025

• Add possibility of configuration for Coordinator and Worker prefix

  #12342 opened Feb 19, 2025

• Add option to provide partition spec in spark ADD_FILES procedure

  #12325 opened Feb 19, 2025

• Serialize `null` for `current-snapshot-id` when there is no current snapshot for ≥V3

  #12310 opened Feb 18, 2025

• Spaces in path parameters are encoded as '+' instead of '%20'

  #12308 opened Feb 18, 2025

• IndexOutOfBounds in FileFormat#fromFileName

  #12300 opened Feb 17, 2025

• Support for Identity Columns in Apache Iceberg

  #12297 opened Feb 17, 2025

• API to find out the number of datafiles deleted

  #12288 opened Feb 16, 2025

• REST API responses with Spark return status code 200 instead of 204

  #12283 opened Feb 16, 2025

• Print un-pretty metadata JSON files without whitespace

  #12281 opened Feb 15, 2025

• previous eq deletes handling on new write

  #12280 opened Feb 15, 2025

• Problems using rewriteTablePath action on local filesystem tables

  #12277 opened Feb 14, 2025

• Cherrypick the data rows [deleted or old values] from a past snapshot

  #12271 opened Feb 14, 2025

• Allow to configure the tables' namespace when using dynamic routing with Kafka Connect

  #12269 opened Feb 14, 2025

• After WRITE ORDERED BY is set in the Iceberg table, Spark will generate an additional layer of Job tasks and no shuffle is generated. Is this a bug?

  #12268 opened Feb 14, 2025

• INT96 timestamp is read as OffsetDateTime, not LocalDateTime

  #12266 opened Feb 14, 2025

• Support for Shallow Clone / Zero Copy Cloning in Apache Iceberg

  #12263 opened Feb 14, 2025

69 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Add Variant custom logical type for Avro

  #12238 commented on Feb 20, 2025 • 16 new comments

• Core: Handle partition evolution case in PartitionStatsUtil#computeStats

  #12137 commented on Feb 20, 2025 • 13 new comments

• Data: Add partition stats writer and reader

  #11216 commented on Feb 20, 2025 • 13 new comments

• Core,Api: Add overwrite option when register external table to catalog

  #12228 commented on Feb 17, 2025 • 11 new comments

• support create table like in flink catalog

  #12199 commented on Feb 20, 2025 • 8 new comments

• Materialized View Spec

  #11041 commented on Feb 18, 2025 • 7 new comments

• Spark: Rewrite V2 deletes to V3 DVs

  #12250 commented on Feb 19, 2025 • 6 new comments

• Core,REST: extend httpClient builder to support tls factory

  #11979 commented on Feb 18, 2025 • 4 new comments

• Core: Bulk deletion in RemoveSnapshots

  #11837 commented on Feb 14, 2025 • 4 new comments

• [WIP] Ignore UnknownType in General Parquet Writer

  #12177 commented on Feb 14, 2025 • 3 new comments

• Retry on NoSuchNamespaceException not found in rename table for rest catalog

  #12159 commented on Feb 18, 2025 • 3 new comments

• Data: Handle case where partition location is missing for `TableMigrationUtil`

  #12212 commented on Feb 19, 2025 • 2 new comments

• Spec additions for encryption

  #12162 commented on Feb 17, 2025 • 2 new comments

• Core: Fix failure when reading files table with branch

  #11719 commented on Feb 18, 2025 • 2 new comments

• Azure: Support vended credentials refresh in ADLSFileIO.

  #11577 commented on Feb 17, 2025 • 2 new comments

• Core: add variant builder implementation

  #11857 commented on Feb 14, 2025 • 1 new comment

• Arrow, Parquet, Spark 3.5, Flink 1.20: Avoid deprecated method

  #11874 commented on Feb 20, 2025 • 1 new comment

• Core: Extended header support for RESTClient implementations

  #12194 commented on Feb 19, 2025 • 1 new comment

• Remove deprecated `OBJECT_STORE_PATH ` and `WRITE_FOLDER_STORAGE_LOCATION `

  #12174 commented on Feb 18, 2025 • 1 new comment

• Core: HadoopFileIO to support bulk delete through the Hadoop Filesystem APIs

  #10233 commented on Feb 17, 2025 • 1 new comment

• Iceberg parallel writes failing - "An error occurred while calling o213.append"

  #11426 commented on Feb 17, 2025 • 0 new comments

• Flaky test `TestFlinkTableSink > testInsertFromSourceTable`

  #11833 commented on Feb 17, 2025 • 0 new comments

• [Core] Support Truncate(0) for metrics

  #11905 commented on Feb 16, 2025 • 0 new comments

• Kafka Connect: Add SMTs for Debezium and AWS DMS

  #11936 commented on Feb 18, 2025 • 0 new comments

• Spark 3.5: Support RewriteManifestsProcedure with a target size parameter

  #11959 commented on Feb 15, 2025 • 0 new comments

• Core: Properly detect metadata tables

  #11963 commented on Feb 17, 2025 • 0 new comments

• Add aliyun-bundle jar

  #10970 commented on Feb 17, 2025 • 0 new comments

• Implementation of version metadata table for view

  #12014 commented on Feb 14, 2025 • 0 new comments

• Spec: Update partition stats for V3

  #12098 commented on Feb 14, 2025 • 0 new comments

• Spark: Support singular form of years, months, days, and hours functions

  #12117 commented on Feb 18, 2025 • 0 new comments

• Spark: Remove closing of IO in SerializableTable*

  #12129 commented on Feb 14, 2025 • 0 new comments

• Does iceberg has plan to support Json Type?

  #6467 commented on Feb 17, 2025 • 0 new comments

• Specify in lower/upper bounds in data_file struct are exact

  #10930 commented on Feb 16, 2025 • 0 new comments

• FlinkSchemaUtil.toSchema should return Schema or ResolvedSchema instead of deprecated TableSchema

  #10950 commented on Feb 15, 2025 • 0 new comments

• Spark streaming (merge into) iceberg table concurrent write with compaction job

  #12187 commented on Feb 14, 2025 • 0 new comments

• Auth Manager API part 6: API enablement

  #12197 commented on Feb 19, 2025 • 0 new comments

• Docs: Add clear indicators for required fields in Spark syntax on CREATE TABLE.

  #9545 commented on Feb 14, 2025 • 0 new comments

• Support bucket transform on multiple data columns

  #5626 commented on Feb 14, 2025 • 0 new comments

• Core: Remove duplicate definitions of MAX_FILE_GROUP_SIZE_BYTES

  #12222 commented on Feb 19, 2025 • 0 new comments

• Iceberg SDK failed to clean up files when table has multiple references with different retention time

  #12200 commented on Feb 13, 2025 • 0 new comments

• Spec: Allow Equality Deletes with Row Lineage and Define Behavior

  #12230 commented on Feb 20, 2025 • 0 new comments

• set tblproperties, spark action expireSnapshots is not work.

  #12078 commented on Feb 13, 2025 • 0 new comments

• Add Zordering to table specification

  #12198 commented on Feb 13, 2025 • 0 new comments

• Spark aggreation by partition could use metadata files

  #11394 commented on Feb 17, 2025 • 0 new comments

• Long-running Spark rewrite Files Action may lead to OutOfMemoryError

  #11277 commented on Feb 17, 2025 • 0 new comments

• It's not possible to readStream from an Iceberg table as source when its snapshots expire

  #9504 commented on Feb 18, 2025 • 0 new comments

• Move docker-specific tests to integrationTest configuration

  #12236 commented on Feb 18, 2025 • 0 new comments

• UpdateSchema.add_column doesn't support adding parent and child in the same transaction

  #12223 commented on Feb 18, 2025 • 0 new comments

• Proxy Settings for catalog REST API client

  #12059 commented on Feb 18, 2025 • 0 new comments

• deadlock when spark call delete row postition

  #10987 commented on Feb 19, 2025 • 0 new comments

• Add properties support for HadoopTables.load()

  #12251 commented on Feb 19, 2025 • 0 new comments

• JdbcCatalog fails to initialize with MS SQL Server

  #10068 commented on Feb 19, 2025 • 0 new comments

• TestS3FileIO fails locally

  #12237 commented on Feb 19, 2025 • 0 new comments

• MERGE INTO requires sorting in already sorted iceberg tables

  #10891 commented on Feb 19, 2025 • 0 new comments

• RewriteDataFiles maintenance action never converges

  #6669 commented on Feb 20, 2025 • 0 new comments

• Manifest list encryption

  #7770 commented on Feb 13, 2025 • 0 new comments

• Does iceberg support "Predicate Pushdown" when spark read data from it?

  #11617 commented on Feb 17, 2025 • 0 new comments

• Questions of using flink to update iceberg table partial columns and delete

  #11720 commented on Feb 17, 2025 • 0 new comments

• Flink: Maintenance - RewriteDataFiles

  #11497 commented on Feb 14, 2025 • 0 new comments

• Core, Rest: Enable useSystemProperties on RESTClient

  #11548 commented on Feb 19, 2025 • 0 new comments

• Subfolder with no name under /data folder

  #12065 commented on Feb 17, 2025 • 0 new comments

• Core: Set missing table-default property in RESTSessionCatalog

  #11646 commented on Feb 17, 2025 • 0 new comments

• Core: Expose `added_rows_count`, `existing_rows_count` and `deleted_rows_count` fields in all_manifests and manifests tables

  #11679 commented on Feb 17, 2025 • 0 new comments

• Purge RCK test entries in `afterEach` instead of `beforeEach`

  #11699 commented on Feb 18, 2025 • 0 new comments

• java.lang.IllegalStateException: Connection pool shut down when close FileAppender.

  #12114 commented on Feb 17, 2025 • 0 new comments

• Spark 3.5: Add query runner in test module

  #11758 commented on Feb 17, 2025 • 0 new comments

• Kafka Connect: Add the configuration option to provide a transactional id prefix to use

  #11780 commented on Feb 19, 2025 • 0 new comments

• Fix comment on `WRITE_OBJECT_STORE_PARTITIONED_PATHS` table property

  #11798 commented on Feb 18, 2025 • 0 new comments

• backport #11301(rowconverter) to Flink 1.19 and 1.18

  #11826 commented on Feb 15, 2025 • 0 new comments