Skip to content

Integrate async transport with SDK #4615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 107 commits into
base: srothh/async-transport
Choose a base branch
from

Conversation

srothh
Copy link
Member

@srothh srothh commented Jul 23, 2025

Integrate the async transport with the rest of the SDK. Provide a new experimental option transport_async that enables the async transport if an event loop is running. Otherwise, fall back to the sync transport.

Furthermore, adapt the client to work with the async transport. To this end, flush and close were changed to be non blocking and awaitable in an async context to avoid deadlocks, however close enforces a completed flush before shutdown. As there are to my knowledge no background threads running flush/close, these methods are currently not thread-safe/loop-aware for async, which can be changed if necessary.

Atexit issue: The atexit integration used by the SDK runs after the event loop has already closed if asyncio.run() is used. This makes it impossible for the async flush to happen, as atexit calls client.close(), but a loop is no longer present. I attempted to apply this fix by patching the loop close in the asyncio integration, but I am unsure if I did it correctly/put it in the correct spot, or if this is a good idea. From my SDK test however, it seems to fix the flush issue. Note also that this will apparently be patched in Python 3.14, as per the discussion in the linked thread.

As a final note, I added event loop checking. Whenever the event loop is used, the transport/client catch RuntimeErrors, which would be thrown in case the event loop was already shut down. I am not sure if this is a case we need to consider, but I added it for now because I did not want the transport to potentially throw RuntimeError if the event loop is shutdown during a program. If this should be left out currently for simplicity, I can remove it again.

I added the httpcore[asyncio] dependency to requirements-testing, as it is needed for the async httpcore functionality.

GH-4601

srothh added 26 commits July 21, 2025 11:44
…ted a sync transport HTTP subclass

Moved shared sync/async logic into a new superclass (HttpTransportCore), and moved sync transport specific code into a new subclass(BaseSyncHttpTransport), from which the current transport implementations inherit

Fixes GH-4568
Removed an unnecessary TODO message and reverted a class name change for BaseHTTPTransport.

GH-4568
Adds test coverage for the error handling path when HTTP requests return
error status codes.

GH-4568
Restore comments accidentally removed during a previous commit.
Refactored class names such that BaseHttpTransport now has the same functionality as before the hierarchy refactor

GH-4568
Add a new flush_async method in the Transport ABC. This is needed for the async transport, as calling it from the client
while preserving execution order in close will require flush to be a coroutine, not a function.

GH-4568
Move flush_async down to the specific async transport subclass. This makes more sense anyway, as
this will only be required by the async transport. If more async transports are expected,
another shared superclass can be created.

GH-4568
Add necessary type annotations to the core HttpTransport to accomodate for async transport.

GH-4568
Add an abstract bass class for implementation of the background worker. This was done to provide a shared interface for the current
implementation of a threaded worker in the sync context as well as the upcoming async task-based worker implementation.

GH-4578
Add a new factory method instead of direct instatiation of the threaded background worker.
This allows for easy extension to other types of workers, such as the upcoming task-based async worker.

GH-4578
Add a new flush_async method to worker ABC. This is necessary because the async transport cannot use a
synchronous blocking flush.

GH-4578
Move the flush_async down to the concrete subclass to not break existing testing. This makes sense,
as this will only really be needed by the async worker anyway and therefore is not shared logic.

GH-4578
Coroutines have a return value, however the current function signature for the worker methods does not
accomodate for this. Therefore, this signature was changed.

GH-4578
Add a new implementation of the worker interface, implementing the worker as an async task. This is
to be used by the upcoming async transport.

GH-4581
Refactor the flush method in the async worker to use the async_flush coroutine.

GH-4581
…unctions

Add a check to see wheter callbacks are awaitable coroutines or functions, as coroutines need to be awaited.

GH-4581
…coroutines

Coroutines do not return None, therefore it is necessary to consider this in the callback parameter of the worker. Previously,
only callbacks with return Type None were accepted.

GH-4581
Enable concurrent callbacks on async task worker by firing them as a task rather than awaiting them. A done callback handles the necessary queue and exception logic.

GH-4581
Changed kill to also use the _TERMINATOR sentinel, so the queue is still drained to this point on kill instead of cancelled immediately. This should also fix potential race conditions with flush_async.

GH-4581
Add proper type annotation to worker task list to fix linting problems

GH-4581
Add an implementation of Transport to work with the async background worker and HTTPCore async.

GH-4582
Async Transport now properly checks for the presence of the event loop in capture_envelop, and drops items
in case the event loop is no longer running for some reason.

GH-4582
Implement a kill method that properly shuts down the async transport. The httpcore async connection pool needs
to be explicitly shutdown at the end of its usage.

GH-4582
Fix type errors resulting from async override and missing type definition in the async transport.

GH-4582
Copy link

codecov bot commented Jul 23, 2025

Codecov Report

❌ Patch coverage is 70.65217% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.69%. Comparing base (6cb72ad) to head (9f226cf).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
sentry_sdk/client.py 71.42% 10 Missing and 8 partials ⚠️
sentry_sdk/integrations/asyncio.py 67.85% 6 Missing and 3 partials ⚠️
Additional details and impacted files
@@                    Coverage Diff                     @@
##           srothh/async-transport    #4615      +/-   ##
==========================================================
+ Coverage                   83.42%   84.69%   +1.26%     
==========================================================
  Files                         158      158              
  Lines                       15654    15729      +75     
  Branches                     2472     2484      +12     
==========================================================
+ Hits                        13060    13321     +261     
+ Misses                       1842     1628     -214     
- Partials                      752      780      +28     
Files with missing lines Coverage Δ
sentry_sdk/consts.py 99.67% <ø> (ø)
sentry_sdk/transport.py 84.41% <100.00%> (+17.22%) ⬆️
sentry_sdk/integrations/asyncio.py 81.60% <67.85%> (-6.53%) ⬇️
sentry_sdk/client.py 83.49% <71.42%> (-1.39%) ⬇️

... and 6 files with indirect coverage changes

Add a try/catch to ensure silent fail on kill in case the event loop shuts down.

GH-4582
@srothh srothh force-pushed the srothh/async-transport-integration branch from e5ea8e9 to 57ea658 Compare July 24, 2025 07:47
@srothh srothh marked this pull request as ready for review July 24, 2025 07:58
@srothh srothh force-pushed the srothh/async-transport branch from fca8740 to 328d8ad Compare July 30, 2025 14:08
cursor[bot]

This comment was marked as outdated.

Copy link
Member

@antonpirker antonpirker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some questsions

cursor[bot]

This comment was marked as outdated.

@srothh
Copy link
Member Author

srothh commented Jul 31, 2025

I am not sure why CI Lint is failing. The issue is in a completely unrelated file that the PR did not change

cursor[bot]

This comment was marked as outdated.

@srothh srothh force-pushed the srothh/async-transport branch from 295a0e9 to 6cb72ad Compare July 31, 2025 12:11
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Async Context Manager Exit Issue

The Client.__exit__ method calls self.close() but does not await the asyncio.Task returned by close() when AsyncHttpTransport is used. Since __exit__ is synchronous, this prevents async flush and cleanup operations from completing, leading to potential data loss or resource leaks when the client is used as a context manager.

sentry_sdk/client.py#L1043-L1045

def __exit__(self, exc_type: Any, exc_value: Any, tb: Any) -> None:
self.close()

Fix in Cursor Fix in Web

Copy link
Member

@antonpirker antonpirker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great now.

ignore the linting problem, we need to fix this in master and potel-base branches.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy