pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

URL: http://github.com/RustPython/RustPython/pull/8403

quests-5f61a10d0c5fd0c3.css" /> Add new_payload_exception helper for constructing built-in payload exceptions by kangdora · Pull Request #8403 · RustPython/RustPython · GitHub
Skip to content

Add new_payload_exception helper for constructing built-in payload exceptions - #8403

Open
kangdora wants to merge 7 commits into
RustPython:mainfrom
kangdora:generic-payload-exception
Open

Add new_payload_exception helper for constructing built-in payload exceptions#8403
kangdora wants to merge 7 commits into
RustPython:mainfrom
kangdora:generic-payload-exception

Conversation

@kangdora

@kangdora kangdora commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #8348.

Summary

Adds new_payload_exception, a generic helper for constructing built-in payload exceptions (a #[repr(C)] struct with extra fields, e.g. SystemExit.code, StopIteration.value, the OSError fields) whose type is known at compile time. It builds them directly with py_new + slot_init instead of going through PyType::call, and replaces the ad hoc construction in the internal builders.

Background

Payload exceptions were introduced in #8282 (SystemExit.code) and #8301 (StopIteration.value), where the value moved from the instance dict into a real struct field. Constructing them currently happens either through PyType::call (via invoke_exception) or through a bespoke per-type builder like OSErrorBuilder. This adds a single direct path for the internal builders that know their exact type at compile time:

pub fn new_payload_exception<T>(&self, cls: PyTypeRef, args: FuncArgs) -> PyResult<PyRef<T>>
where
    T: Constructor<Args = FuncArgs> + Initializer,
{
    let payload = T::py_new(&cls, args.clone(), self)?;
    // cls not matching T structurally is a caller bug, not a runtime error
    let exc = payload
        .into_ref_with_type_lazy_dict(self, cls)
        .expect("new_payload_exception: cls is not a matching subtype of T");
    T::slot_init(exc.as_object().to_owned(), args, self)?;
    Ok(exc)
}

new_exception stays as the thin, restricted fast path for payload-free types. invoke_exception stays for construction where the concrete type is only known at runtime and a compile-time T cannot express it: raise <expr>, the C-API entry, and ctypes COMError. BaseExceptionGroup is left as is too, since it is repr(transparent) and carries no extra payload.

Call sites

This routes the internal builders that know their type at compile time through the helper:

  • new_stop_iteration and OSErrorBuilder, the simplest and most complex payload exceptions respectively
  • SystemExit raise sites: builtins exit, sys.exit, _thread.exit, and the finalization check in check_signals
  • BlockingIOError raise sites in _io
  • the errno-based OSError subtype dispatch in PyOSError::slot_new. The target subtype is chosen at runtime, but every result of errno_to_exc_type is a repr(transparent) OSError subtype, so PyOSError is a valid payload type for all of them

What this buys

The main effect is on invoke_exception. Once these sites move off it, invoke_exception is left holding only construction that is genuinely runtime-typed (raise <expr>, C-API, COMError), so its role is explicit rather than a catch-all. Going through the helper also skips PyType::call, which for each construction avoids two slot lookups, the __new__ and __init__ indirect calls, an args clone, and a subclass check. For the hot sites (StopIteration on iterator exhaustion, and the errno OSError path) that saving is real per call. For the cold raise sites (SystemExit, BlockingIOError) it is not measurable.

Notes

Two things I would welcome feedback on:

  • For the cold sites the change is consistency-driven, not a perf win. It makes every internal payload construction go through one helper and narrows invoke_exception to the runtime-typed path, but those raise sites are rare. Happy to land only the higher-value sites if the consistency-only ones aren't worth the churn.
  • The owned upcast has a cost of its own. Call sites that return PyBaseExceptionRef upcast the helper's PyRef<T>, and .upcast() goes through a runtime downcast (the zero-cost upcast_ref/to_base only yield &Py<_>). So the migration trades PyType::call's slot dispatch for a smaller runtime downcast: a net reduction in steps where it is hot, and a wash where it is cold. If there is a zero-cost owned upcast I missed, I would rather use that.

Assisted-by: claude-fable-5

Summary by CodeRabbit

  • Bug Fixes
    • Improved construction and handling of built-in exceptions, including OSError, BlockingIOError, SystemExit, and StopIteration.
    • Preserved error details such as messages, errno values, exit arguments, and bytes written.
    • Improved exception behavior during thread shutdown and I/O operations.
  • Compatibility
    • No public API changes; existing exception behavior remains consistent.

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: f29fad18-2851-4821-a1e7-144a235d6b91

📥 Commits

Reviewing files that changed from the base of the PR and between 6e28029 and abad718.

📒 Files selected for processing (5)
  • crates/vm/src/stdlib/_thread.rs
  • crates/vm/src/stdlib/builtins.rs
  • crates/vm/src/stdlib/sys.rs
  • crates/vm/src/vm/mod.rs
  • crates/vm/src/vm/vm_new.rs
🚧 Files skipped from review as they are similar to previous changes (5)
  • crates/vm/src/vm/mod.rs
  • crates/vm/src/stdlib/builtins.rs
  • crates/vm/src/stdlib/sys.rs
  • crates/vm/src/vm/vm_new.rs
  • crates/vm/src/stdlib/_thread.rs

📝 Walkthrough

Walkthrough

The VM now provides typed payload exception construction. OSError, BlockingIOError, SystemExit, and StopIteration paths use this helper instead of generic exception invocation.

Changes

Payload exception construction

Layer / File(s) Summary
Typed payload constructor
crates/vm/src/vm/vm_new.rs
Adds VirtualMachine::new_payload_exception and uses it for StopIteration.
OSError construction migration
crates/vm/src/exceptions.rs, crates/vm/src/stdlib/_io.rs
Updates OSError and nonblocking I/O error paths to construct typed payload exceptions while preserving errno, messages, and character counts.
SystemExit construction migration
crates/vm/src/stdlib/_thread.rs, crates/vm/src/stdlib/builtins.rs, crates/vm/src/stdlib/sys.rs, crates/vm/src/vm/mod.rs
Updates exit handling and non-main-thread finalization to construct and upcast PySystemExit payloads.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ExceptionPath
  participant VirtualMachine
  participant PyException
  ExceptionPath->>VirtualMachine: new_payload_exception(exception type, arguments)
  VirtualMachine->>PyException: py_new(arguments)
  VirtualMachine->>PyException: slot_init(arguments)
  PyException-->>VirtualMachine: typed exception
  VirtualMachine-->>ExceptionPath: upcast exception
Loading

Possibly related PRs

Suggested reviewers: youknowone

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the primary change: adding the new_payload_exception helper for built-in payload exceptions.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@youknowone youknowone added the z-ca-2026 Tag to track Contribution Academy 2026 label Jul 28, 2026
@youknowone

Copy link
Copy Markdown
Member

@kangdora do you have any concern to work more on this patch?

@kangdora

kangdora commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Yeah, one concern. Since invoke_exception is the shared construction path for every exception, this change touches a lot of call sites at once, so I want to be careful about scope. Sorry this has been sitting quiet, I haven't pushed more on it yet.

Honestly this is a little beyond my current level, so instead of changing everything at once I've been going through the call sites to figure out which ones are actually safe to touch first. Where I've landed so far:

  • Safe to migrate (payload type known at compile time): the SystemExit and BlockingIOError sites. These aren't hot paths, so the value is consistency rather than perf.
  • Has to stay on invoke_exception (type only known at runtime): the ExceptionCtor path, reduce, the C-API entry, and ctypes' COMError. These need to respect user __new__/__init__ overrides, which requires the PyType::call slot dispatch that new_payload_exception deliberately skips.
    There are also a few edge cases that might be migratable too, but I haven't fully convinced myself yet, so I'm leaving them for now.

So rather than limiting it upfront, I'd like to keep working through these gradually as I understand the area better. If I've misjudged any of these boundaries, I'd really appreciate a correction.

@kangdora
kangdora marked this pull request as ready for review August 4, 2026 04:45

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
crates/vm/src/vm/vm_new.rs (1)

357-369: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a debug-mode type/class consistency guard.

new_payload_exception builds a PyRef<T> from an arbitrary cls: PyTypeRef argument. The function relies only on the doc comment to state that T must match the exact built-in type. new_exception in this same file guards this exact class of misuse with a debug_assert_eq! on basicsize. Add a similar guard here.

Do this before merging the payload into into_ref_with_type_lazy_dict. Compare cls.slots.basicsize against core::mem::size_of::<T>() in a debug_assert_eq!, with a message that points callers to the doc comment.

♻️ Proposed defensive guard
     pub fn new_payload_exception<T>(&self, cls: PyTypeRef, args: FuncArgs) -> PyResult<PyRef<T>>
     where
         T: Constructor<Args = FuncArgs> + Initializer,
     {
+        debug_assert_eq!(
+            cls.slots.basicsize,
+            core::mem::size_of::<T>(),
+            "vm.new_payload_exception::<{}>() called with mismatched type '{}'",
+            core::any::type_name::<T>(),
+            cls.name()
+        );
         let payload = T::py_new(&cls, args.clone(), self)?;
         let exc = payload.into_ref_with_type_lazy_dict(self, cls)?;
         T::slot_init(exc.as_object().to_owned(), args, self)?;
         Ok(exc)
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/vm/src/vm/vm_new.rs` around lines 357 - 369, Add a debug_assert_eq! in
new_payload_exception immediately before into_ref_with_type_lazy_dict, comparing
cls.slots.basicsize with core::mem::size_of::<T>() and including a message
directing callers to the doc comment.
crates/vm/src/stdlib/sys.rs (1)

779-783: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Duplicate SystemExit construction across four files. Each site builds SystemExit through the identical sequence new_payload_exception::<PySystemExit>(vm.ctx.exceptions.system_exit.to_owned(), <args>)?.upcast(), differing only in the args value. Extract one shared helper instead of repeating the construction logic.

  • crates/vm/src/stdlib/sys.rs#L779-L783: replace the inline construction with a call to a new vm.new_system_exit(args.into()) helper.
  • crates/vm/src/stdlib/_thread.rs#L638-L643: replace the inline construction with vm.new_system_exit(vec![].into()).
  • crates/vm/src/stdlib/builtins.rs#L1044-L1049: replace the inline construction with vm.new_system_exit(vec![code].into()).
  • crates/vm/src/vm/mod.rs#L2172-L2177: replace the inline construction with self.new_system_exit(vec![].into()).

Add the helper to crates/vm/src/vm/vm_new.rs, next to new_stop_iteration:

pub fn new_system_exit(&self, args: FuncArgs) -> PyBaseExceptionRef {
    self.new_payload_exception::<PySystemExit>(
        self.ctx.exceptions.system_exit.to_owned(),
        args,
    )
    .expect("SystemExit is a BaseException Subclass.")
    .upcast()
}

As per coding guidelines, "When branches differ only in a value but share logic, extract the differing value and invoke the common logic once."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/vm/src/stdlib/sys.rs` around lines 779 - 783, Extract the repeated
SystemExit construction into new_system_exit in crates/vm/src/vm/vm_new.rs
beside new_stop_iteration, accepting FuncArgs and returning PyBaseExceptionRef
with the existing payload-exception behavior. Replace the inline construction at
crates/vm/src/stdlib/sys.rs:779-783 with vm.new_system_exit(args.into()), at
crates/vm/src/stdlib/_thread.rs:638-643 with vm.new_system_exit(vec![].into()),
at crates/vm/src/stdlib/builtins.rs:1044-1049 with
vm.new_system_exit(vec![code].into()), and at crates/vm/src/vm/mod.rs:2172-2177
with self.new_system_exit(vec![].into()).

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/vm/src/stdlib/sys.rs`:
- Around line 779-783: Extract the repeated SystemExit construction into
new_system_exit in crates/vm/src/vm/vm_new.rs beside new_stop_iteration,
accepting FuncArgs and returning PyBaseExceptionRef with the existing
payload-exception behavior. Replace the inline construction at
crates/vm/src/stdlib/sys.rs:779-783 with vm.new_system_exit(args.into()), at
crates/vm/src/stdlib/_thread.rs:638-643 with vm.new_system_exit(vec![].into()),
at crates/vm/src/stdlib/builtins.rs:1044-1049 with
vm.new_system_exit(vec![code].into()), and at crates/vm/src/vm/mod.rs:2172-2177
with self.new_system_exit(vec![].into()).

In `@crates/vm/src/vm/vm_new.rs`:
- Around line 357-369: Add a debug_assert_eq! in new_payload_exception
immediately before into_ref_with_type_lazy_dict, comparing cls.slots.basicsize
with core::mem::size_of::<T>() and including a message directing callers to the
doc comment.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5f03cc7c-268b-4a89-aa69-a4a2bf0b2e9d

📥 Commits

Reviewing files that changed from the base of the PR and between 3aaec06 and 6e28029.

📒 Files selected for processing (7)
  • crates/vm/src/exceptions.rs
  • crates/vm/src/stdlib/_io.rs
  • crates/vm/src/stdlib/_thread.rs
  • crates/vm/src/stdlib/builtins.rs
  • crates/vm/src/stdlib/sys.rs
  • crates/vm/src/vm/mod.rs
  • crates/vm/src/vm/vm_new.rs

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! The changes look good in general, but I have a few suggestions. Please check them

self.ctx.exceptions.stop_iteration.to_owned(),
args,
)
.expect("StopIteration is a BaseException Subclass.")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new_payload_exception now can raise exceptions by multiple reasons, so this except message doesn't work. I am even not sure expect is good idea anymore.

Comment on lines +639 to +643
.new_payload_exception::<PySystemExit>(
vm.ctx.exceptions.system_exit.to_owned(),
vec![].into(),
)
.expect("SystemExit is a BaseException Subclass.")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this pattern repeats this many times, I prefer to add vm.new_system_exit() rather than manual building for multiple times.

cls.name()
);
let payload = T::py_new(&cls, args.clone(), self)?;
let exc = payload.into_ref_with_type_lazy_dict(self, cls)?;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calling into_ref_with_type_lazy_dict came from OSErrorBuilder. I don't know this part well, so I'd like to ask if this is safe for every exception types or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

z-ca-2026 Tag to track Contribution Academy 2026

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

pFad - Phonifier reborn

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.





Check this box to remove all script contents from the fetched content.



Check this box to remove all images from the fetched content.


Check this box to remove all CSS styles from the fetched content.


Check this box to keep images inefficiently compressed and original size.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy