When using the "PARQUET:filename.parquet" syntax, this should already be implemented to the best extent we can using the ScannerBuilder::Filter API (https://arrow.apache.org/docs/cpp/api/dataset.html#_CPPv4N5arrow7dataset14ScannerBuilder6FilterERKN7compute10ExpressionE)

With regular "filename.parquet" syntax, we do optimizations on OGR side, that may be inferior to the prior approach. The reason we have those 2 modes is that when the driver was developped, the ArrowDataset library which makes it possible to use Scannerbuilder::Filter wasn't declared to be stable

Thanks for the quick answer. Just to understand you correctly: The parquet reader already does filtering and projection pushdown? (sorry my ignorance about the implementation where I don't yet see what the typical dependency to Arrow as a row format means).

E.g. like with following test query from https://docs.overturemaps.org/getting-data/duckdb/ not yet adapted to OGR, where the attributes id, name and geometry are "projected" and the WHERE statement "filters" optimized - including hive_partitioning and zone maps):

SELECT
  id,
  names.primary as name,
  geometry                            -- the GEOMETRY type to be implemted 
FROM read_parquet('s3://overturemaps-us-west-2/release/2024-10-23.0/theme=places/type=place/*', filename=true, hive_partitioning=1)
WHERE 
  bbox.xmin BETWEEN -75 AND -73       -- Only use the bbox miny/miny values 
  AND bbox.ymin BETWEEN 40 AND 41     -- because they are point geometries.
  AND categories.primary = 'pizza_restaurant':

The parquet reader already does filtering and projection pushdown?

yes, for partitionned datasets like OvertureMaps, the "PARQUET:" dataset mode (https://gdal.org/en/stable/drivers/vector/parquet.html#dataset-partitioning-read-support) is necessary used and attribute and spatial filter pushdown is implemented using https://arrow.apache.org/docs/cpp/dataset.html#filtering-data . I believe the reason for DuckDB being much faster than Arrow is a smarter implementation. I don't think there's anything that can be done on GDAL side (I may have missed something of course)

#11459 implements attribute and spatial filter pushdown for Parquet through the new to 3.11dev ADBC driver using DuckDB Parquet capabilities

@rouault - I may be missing something, but whenever I've tried to create a geoparquet file using GDAL, they are missing Column and Offset indexes - is this expected? I don't see how efficient filter pushdown would be possible without these. I, of course, could be mistaken here.

they are missing Column and Offset indexes

what do you mean exactly by Column and Offset indexes ?

The page indexes for each column chunk: https://parquet.apache.org/docs/file-format/pageindex/

The page indexes for each column chunk: https://parquet.apache.org/docs/file-format/pageindex/

thanks. The capability was added in libarrow 12 per apache/arrow#34054 , but required to be explicitly enabled. Will be done per #11565

Fantastic! Thanks for the speedy response!