Skip to content

Improve performance with nested PacketFields #4727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 32 commits into
base: master
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
9db46b8
Add 'test/perf_packet_fields.uts' test (#4705)
alxroyer-thales Mar 21, 2025
4bbaab2
Rename classes in 'test/perf_packet_fields.uts' (#4705)
alxroyer-thales Mar 31, 2025
9bf0c54
Add 'test/auto_clear_cache.uts' test (#4706)
alxroyer-thales Mar 31, 2025
772459f
Improve 'perf_packet_fields.uts' (#4705)
alxroyer-thales Apr 6, 2025
cc2572d
Improve 'auto_clear_cache.uts' (#4706, #4707)
alxroyer-thales Apr 6, 2025
2d3c005
Improve 'perf_packet_fields.uts' with operation counts (#4705)
alxroyer-thales Apr 23, 2025
6e72bc3
Fix 'perf_packet_fields.uts' when `NUMBER_OF_F_PER_I>255` (#4705)
alxroyer-thales Apr 23, 2025
af50227
Report scapy@2.4.5 optimizations in scapy@2.6.1 (#4705, #4706, #4707)
alxroyer-thales Apr 3, 2025
debbbde
Avoid `add_payload()` clearing the cache when dissecting (#4706)
alxroyer-thales Apr 7, 2025
35877da
Restore default `clear_cache()` behaviour (#4707)
alxroyer-thales Apr 7, 2025
d6e0cde
Revert `_PacketField.m2i()` changes (#4705)
alxroyer-thales Apr 18, 2025
ead2e51
Revert `clear_cache()` for payload modifications (#4706)
alxroyer-thales Apr 23, 2025
e7465c8
Ensure cache set for good at the end of `Packet._fast_copy()` (#4705)
alxroyer-thales Apr 23, 2025
33aeae4
Fix parent not set when building default packet fields (#4706, #4707)
alxroyer-thales Apr 23, 2025
0976a60
Fix Thales copyrights (#4705, #4706, #4707)
alxroyer-thales Jun 27, 2025
7fed967
Fix `post_build()` not being called anymore (#4705)
alxroyer-thales Jul 16, 2025
625154c
Fix `NoPayload.clear_cache()` signature (#4707)
alxroyer-thales Jul 16, 2025
4583af8
Fix `setfieldval()` when the field is not set yet (#4705)
alxroyer-thales Jul 16, 2025
7f0f095
Add `clear_cache()` calls in 'test/fields.uts' (#4705)
alxroyer-thales Jul 18, 2025
d4d6655
Fix cache issues with packet list fields (#4705)
alxroyer-thales Jul 18, 2025
bf0fa0e
Revert "Add `clear_cache()` calls in 'test/fields.uts' (#4705)"
alxroyer-thales Jul 18, 2025
e32dab1
Avoid caching for packets returned by `fuzz()` (#4705)
alxroyer-thales Jul 18, 2025
23b1628
Adjust `MAX_TIME_PER_OP` in 'perf_packet_fields.uts' (#4705)
alxroyer-thales Jul 18, 2025
5a1f68f
Avoid `fuzz()` messing up `default_fields` (#4705)
alxroyer-thales Jul 21, 2025
471bf40
Make `FlagValue` bound to a `Packet` (#4705)
alxroyer-thales Jul 21, 2025
9415f58
Make `Packet._ensure_bound_field_value()` recursive on lists (#4705)
alxroyer-thales Jul 21, 2025
2fec703
Rework `list_field` as `ListValue` in 'fields.py' (#4705)
alxroyer-thales Jul 21, 2025
9c26a32
Fix `Packet.setfieldval()` when `attr` not already in `self.fields` (…
alxroyer-thales Jul 22, 2025
000e16b
Copy the `no_cache` flag in `Packet._fast_copy()` (#4705)
alxroyer-thales Jul 23, 2025
46f12db
Fix unit test (#4705)
alxroyer-thales Jul 23, 2025
d64e574
Avoid caching with `BTLE` layer (#4705)
alxroyer-thales Jul 23, 2025
d877990
Hide `ValueError`s in `Packet.setfieldval()` optimization (#4705)
alxroyer-thales Jul 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert clear_cache() for payload modifications (#4706)
In `Packet.add_payload()` and `remove_payload()`.
  • Loading branch information
alxroyer-thales committed Apr 23, 2025
commit ead2e516212c2f29fac6547ed88eea07a61d75f0
18 changes: 6 additions & 12 deletions scapy/packet.py
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ def __init__(self,
# Clarify the meaning for this `explicit` flag.
# Now that cache is cleared as soon as the packet is modified, possibly we could get rid of it?
self.explicit = 0
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As noted above, the meaning of the explicit attribute should probably be explained.
To be discussed.

#: Caches bytes after `dissect()` or `build()` for the current layer only (not the payload, even if set).
self.raw_packet_cache = None # type: Optional[bytes]
self.wirelen = None # type: Optional[int]
self.direction = None # type: Optional[int]
Expand Down Expand Up @@ -379,12 +380,12 @@ def get_field(self, fld):
"""DEV: returns the field instance from the name of the field"""
return self.fieldtype[fld]

def add_payload(self, payload, clear_cache=True):
# type: (Union[Packet, bytes], bool) -> None
def add_payload(self, payload):
# type: (Union[Packet, bytes]) -> None
if payload is None:
return
elif not isinstance(self.payload, NoPayload):
self.payload.add_payload(payload, clear_cache=clear_cache)
self.payload.add_payload(payload)
else:
if isinstance(payload, Packet):
self.payload = payload
Expand All @@ -398,19 +399,12 @@ def add_payload(self, payload, clear_cache=True):
else:
raise TypeError("payload must be 'Packet', 'bytes', 'str', 'bytearray', or 'memoryview', not [%s]" % repr(payload)) # noqa: E501

# Invalidate cache when the packet has changed.
if clear_cache:
self.clear_cache(upwards=True, downwards=False)

def remove_payload(self):
# type: () -> None
self.payload.remove_underlayer(self)
self.payload = NoPayload()
self.overloaded_fields = {}

# Invalidate cache when the packet has changed.
self.clear_cache(upwards=True, downwards=False)

def add_underlayer(self, underlayer):
# type: (Packet) -> None
self.underlayer = underlayer
Expand Down Expand Up @@ -1203,7 +1197,7 @@ def do_dissect_payload(self, s):
):
# stop dissection here
p = conf.raw_layer(s, _internal=1, _underlayer=self)
self.add_payload(p, clear_cache=False)
self.add_payload(p)
return
cls = self.guess_payload_class(s)
try:
Expand All @@ -1226,7 +1220,7 @@ def do_dissect_payload(self, s):
if cls is not None:
raise
p = conf.raw_layer(s, _internal=1, _underlayer=self)
self.add_payload(p, clear_cache=False)
self.add_payload(p)

def dissect(self, s):
# type: (bytes) -> None
Expand Down
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy