k/generator: do not use standard vectors for offset commit protocol #26280
Conversation
ca99c81 to
dc94b03
Compare
Retry command for Build#66564please wait until all jobs are finished before running the slash command |
CI test resultstest results on build#66564
test results on build#66692
|
dc94b03 to
50cf97b
Compare
| return fut.then(func).handle_exception( | ||
| [&eptr](std::exception_ptr ex) mutable { | ||
| return fut.then([&func] { return func(); }) | ||
| .handle_exception([&eptr](std::exception_ptr ex) mutable { |
There was a problem hiding this comment.
this looks ok, but i'm having trouble determining if this is safe w.r.t. to guarnateeing that the retry function will always execute with a fresh version of it's original captures. i think that is true if func is the user provided function, but i'm less sure that is true for gated_retry_with_mitigation_impl which wraps the user function and how the const-ness of operator()() propogates to the captured function.
maybe @BenPope knows. i've seen this exact function have similar proposals made recently and I'm not sure where things stand with those changes and if there are footguns here.
--
more broadly, though, and not necessarily blocking this PR: do we want to optimize for large captures? it seems like the caller could always arrange for a large object to be captured as a reference, shared pointer, etc...?
There was a problem hiding this comment.
the static assert pass, i.e. it should not be possible to do std::move() in the body of operator()() of the func
There was a problem hiding this comment.
fwiw the static assert only tests that it has a const operator, right? it doesn't test that there isn't a non-const operator.
There was a problem hiding this comment.
overall yeh i think it seems safe
| @@ -146,4 +147,18 @@ ss::future<shared_broker_t> find_coordinator_with_retry_and_mitigation( | |||
| }); | |||
| } | |||
|
|
|||
| inline chunked_vector<kafka::offset_commit_request_topic> | |||
| make_copy(const chunked_vector<kafka::offset_commit_request_topic>& topics) { | |||
There was a problem hiding this comment.
I can see new fields being left out of this. I wonder if we should change the code generator to generate a copy() for types containing noncopyable containers?
A quick mitigation would be to static_assert the size of offset_commit_request_topic
There was a problem hiding this comment.
it is good idea to generate a copy() method but that would be more involved, i will add a static_assert here to make sure it is noticed when changed, i will add a test case as well
Implementation of `then()` uses a `std::forward<Func>` when creating continuation. Passing in an lvalue ref to `fut.then(...)` forces the copy. Replaced it with lambda capturing the reference to `func` not to have to copy the whole `func` Signed-off-by: Michał Maślanka <michal@redpanda.com>
In transactional and standard offset commit request and replies topic partition offsets are stored as a vectors. The number of partitions/topics in a single request may grow large, therefore we should use a fragmented vector when handling those requests. Changed the Kafka protocol generator overrides for offset commit requests and responses to use default `chunked_vector` Signed-off-by: Michał Maślanka <michal@redpanda.com>
Signed-off-by: Michał Maślanka <michal@redpanda.com>
50cf97b to
f57ecd6
Compare
In transactional and standard offset commit request and replies topic partition offsets are stored as a vectors. The number of partitions/topics in a single request may grow large, therefore we should use a fragmented vector when handling those requests.
Changed the Kafka protocol generator overrides for offset commit requests and responses to use default
chunked_vectorBackports Required
Release Notes
Improvements