Bug Fixes

• Fix a regression in the evaluations UI where incremental results are not displayed until all generated outputs are ready.
• Fix a regression in the evaluations UI where users could not open the detail page for completed datapoints in partial (e.g. failed) runs.

& multiple under-the-hood and UI improvements (thanks @adithya-adee @Garvity!)

Bug Fixes

• Fix an edge case in the Python client affecting list_datapoints and get_datapoint for datapoints without output.

Bug Fixes

• Handle a regression in the Fireworks AI SFT API

New Features

• Add endpoints (and client methods) for listing and querying datapoints programmatically

& multiple under-the-hood and UI improvements

Bug Fixes

• Allow multiple text content blocks in individual input messages.

Bug Fixes

• Optimized the performance of additional ClickHouse queries in the UI that previously consumed excessive time and memory at scale.

& multiple under-the-hood and UI improvements (thanks @bhatt-priyadutt!)

Bug Fixes

• Comply with the OpenAI API format when using dynamic output schemas in the OpenAI-compatible inference API

Bug Fixes

• Optimized the performance of ClickHouse queries in the UI that previously consumed excessive time and memory at scale

New Features

• Add support for managing datasets and datapoints programmatically
• Enable tool use with SGLang (thanks @subygan!)
• Improve error messages across the stack

& multiple under-the-hood and UI improvements (thanks @subham73 @Daksh14!)

Bug Fixes

• Fix an issue in the UI where multimodal inferences fail to parse when no object storage region is specified.
• Handle an edge case with Google AI Studio's streaming API where some response fields are missing in certain chunks.

New Features

• Support tensorzero::extra_body and tensorzero::extra_headers in the OpenAI-compatible inference endpoint.
• Allow users to specify inference caching behavior in the evaluations UI.
• Improve the performance of some database queries in the UI.

& multiple under-the-hood and UI improvements

Bug Fixes

• Fix an issue affecting a small subset of users caused by a recent change to the OpenAI API
• Support Windows-specific signal handling to enable non-WSL Windows users to run the gateway natively

New Features

• Support exporting OpenTelemetry traces (OTLP) from the TensorZero Gateway
• Add batch inference for GCP Vertex AI Gemini models (gcp_vertex_gemini provider)
• Enable users to send extra headers to model providers at inference time (thanks @oliverbarnes!)

& multiple under-the-hood and UI improvements (thanks @nyurik @rushatgabhane!)

Bug Fixes

• Fix an edge case that could result in duplicate inference results in the database for batch inference jobs.
• Improve the performance of a database query in the inference detail page in the UI.
• Fix an edge case that prevented the UI from parsing tool choices correctly in some scenarios.
• Avoid unnecessarily parsing tool results sent to GCP Vertex AI models.

New Features

• Add examples integrating TensorZero with Cursor and OpenAI Codex.
• Make the OpenAI-compatible inference endpoint respect include_usage.

& multiple under-the-hood and UI improvements