Skip to content

Fee rate formatting #4414

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

Kixunil
Copy link
Collaborator

@Kixunil Kixunil commented Apr 29, 2025

This makes fixes to FeeRate. It's still missing tests and I think something else as well, I just don't remember what it was :D But should be fine to review if you want to take a look. I just wanted to get it out earlier.

Closes #4339

Kixunil added 8 commits April 28, 2025 17:19
`FormatOptions` was a workaround to support arbitrary `fmt::Write` in
legacy API. That API is now removed so `FormatOptions` is no longer
needed because we can just accept `Formatter` and pull the options from
itself.

This commit removes the struct and the argument from `fmt_satoshi_in`,
changing `f` from `fmt::Write` to `fmt:: Formatter`.
We'd like to improve formatting of other types to support decimals. To
avoid code duplication, this commit does part of the change: simple move
to a separate file with appropriate plumbing.

This is still not fully general but it's simpler to review.
Before a decimal number is displayed it is normalized to not end with 0
unless it's 0. This was performed in amount formatting but it applies to
all numbers. The change was intentionally split up to make review
easier.

This moves the computation from a single branch and makes it
unconditional. This is valid because other branches keep
`num_after_decimal_point` as 0 and the condition in the moved code does
0 check, returning 0 which is the valid value for the other branches.
When displaying a decimal number rounding may be required to satisfy
`precision` field on formatter. This implies rounding is also a general
concept, not tied to `Amount` and needs to be moved to decimal as well.

This commit moves rounding but because decimal accepts numbers before
and after decimal point it needs to be modified a bit to account for the
possibility that the number before decimal point increases because of
rounding. E.g. 0.999 Rounded to two decimal places is 1.00.
So far `FeeRate` had very sketchy implementation that relied on
`alternate` to select the unit, which was surprising and limiting.
Also the sat/vB implementation computed ceiling but also added `.00`
which looked like the value was exact. Further, the unit was always
shown in case of sat/vB and never shown in case of sat/kwu.

To fix it, this commit adds a new `Display` type similar to the one in
`amount` which can be used to tweak how it's displayed. Because
`FeeRate` so far supports multiple rounding modes the support of these
modes was added to the `Decimal` type and the `Decimal` type was used to
implement `Display`. The unit is displayed by default to avoid confusion
and can be hidden with a method call.

This doesn't modify the `Display` impl yet because it's more complicated
and easier to review separately.
As mentioned in previous commit, the `Display` impl was inconsistent and
problematic in multiple ways. The previous commit introduced an
alternative API that can tweak the formatting but didn't touch the
`Display` impl. This one does but to do that `FromStr` had to be changed
as well.

This changes the `Display` impl to always show the unit and also
delegates existing `sat/vB` code to the newly-added API, so that decimal
formatting is more sensible. This change is very obvious in output and
breaks parsing which is intended. So to ensure our roundtrip tests keep
working this also modifies `FromStr` to require the unit similarly to
how `Amount` requires denomination.

For simplicity, the code currently doesn't support sat/vB parsing but
does detect the attempt and reports the error properly. This can be
added later.
The comment claimed that, as opposed to floats in `core`, the display
implementation doesn't support lower precision. This was true in the
past but it was later changed to support it making the comment stale.
When a number has no decimals, setting the value and rounding doesn't
make sense since they will not be used. Conversely, if the number after
decimal point is 0 then setting number of decimals makes no sense since
it would be normalized to 0. So far the code dealing with these cases
was setting useless values to those fields.

This change refactors the code to use a single filed which is an
`Option` to represent the number after decimal point being present or
not. So either all values need to be provided or none of them. To
improve clarity, `NonZero` integers are used to enforce that the values
are not zero.

Since the code internally transforms the value another, similar type is
used. But this time it's an enum to represent the possibility that
there's no actual number and it's just trailing zeros requested by
formatter.

Using `NonZero` also uncovered an API edge case: setting inconsistent
values to those fields can lead to broken behavior. I don't see a
straightforward solution, and the API is internal, so keeping it as
panic seems reasonable.

During the refactor I also noticed overflow when consumer entered
absurdly large precision was not handled. Since it only affects padding
and the resulting number would be larger than any possible `width`
provided by the formatter the correct fix is to use `saturating_add`.
That way the ("incorrectly") computed number width will be larger or
equal to `width` leading to zero padding and if it was possible to
compute larger number width it'd still be larger than `width` leading to
zero padding.
@github-actions github-actions bot added C-units PRs modifying the units crate test labels Apr 29, 2025
@coveralls
Copy link

Pull Request Test Coverage Report for Build 14741157675

Details

  • 196 of 248 (79.03%) changed or added relevant lines in 3 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage decreased (-0.07%) to 84.148%

Changes Missing Coverage Covered Lines Changed/Added Lines %
units/src/amount/mod.rs 25 29 86.21%
units/src/fee_rate/mod.rs 32 80 40.0%
Files with Coverage Reduction New Missed Lines %
units/src/amount/mod.rs 1 88.64%
Totals Coverage Status
Change from base Build 14717791466: -0.07%
Covered Lines: 22938
Relevant Lines: 27259

💛 - Coveralls

Copy link
Member

@tcharding tcharding left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caveat Emptor: In general I don't have a strong opinion on the fee_rate module and what changes are needed.

I commented a few things that are basically just typos or cut'n'paste mistakes.

In general, in my opinion:

  • This PR needs almost exhaustive tests before its possible to ack it.
  • The changes to fee_rate/mod.rs look good and correct.
  • decimal is private anyway so I did not review it super closely.

pub(crate) num_before_decimal_point: u64,
/// `num_before_decimal_point` is multiplied by `10 ^ exp`.
pub(crate) exp: usize,
/// None means the number is round.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does 'round' mean here? Is one of these correct?

  • 'None means the number is an integer'
  • 'None means number is rounded to the correct number of decimal places already'

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it to mean it needs no rounding. It's odd wording though. Divides without remainder maybe?

Copy link
Collaborator Author

@Kixunil Kixunil Apr 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is an integer (or, even if the type it came from represented decimals somehow it's zero)

/// By default this has precision of 3 decimal places (but the trailing zeros are not
/// displayed). In that case the value is precise and no rounding is applied.
///
/// However, using smaller formatter precision will require rounding of the numer. This method
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// However, using smaller formatter precision will require rounding of the numer. This method
/// However, using smaller formatter precision will require rounding of the nubmer. This method

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And in a bunch of other places too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

your correction is misspelled heh: s/nubmer/number

}
}

/// Displays the fee rate using sat/kwu unit, rounding if needed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect unit.

}
}

/// Displays the fee rate using sat/kwu unit, rounding down if needed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect unit.

}
}

/// Displays the fee rate using sat/kwu unit, rounding up if needed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect unit.

@Kixunil
Copy link
Collaborator Author

Kixunil commented Apr 29, 2025

  • This PR needs almost exhaustive tests before its possible to ack it.

Yes, just note that we already have really, really good tests in amount. They caught some of my edge case mistakes.

I'm not sure if some of them should be moved to decimal and if so, I'd do it in a separate commit to prove that I didn't break stuff in previous commits. I certainly want to add test suite for FeeRate but I don't think it needs to be as crazy as Amount given the code is shared.

@tcharding
Copy link
Member

Fair, I didn't think of that.

@tcharding
Copy link
Member

Reviewed 0905ce3

impl core::str::FromStr for FeeRate {
type Err = ParseError;

fn from_str(s: &str) -> Result<Self, Self::Err> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I was wanting FromStr for FeeRate elsewhere. Nice that this PR includes it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It already existed via a macro, this just implements it manually because of formatting changes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah cool I didn't know

@yancyribbens
Copy link
Contributor

Thanks for doing this. There are parts of the decimal parsing I haven't reviewed super close because I'm not already very familiar with how it was being parsed before. This looks much better though and some tests would be great.

@yancyribbens
Copy link
Contributor

This PR needs almost exhaustive tests before its possible to ack it.

Yes, just note that we already have really, really good tests in amount. They caught some of my edge case mistakes.

There is still an open issue to add proptests still to amount which would help.

@Kixunil
Copy link
Collaborator Author

Kixunil commented Apr 30, 2025

There is still an open issue to add proptests still to amount which would help.

Yeah, I'm pretty sure those should test the newly-added Decimal.

.map(Self::from_sat_per_kwu)
.map_err(|error| ParseError(ParseErrorInner::InvalidSatPerKwu(error)))
},
"sat/vb" => Err(ParseError(ParseErrorInner::UnsupportedSatPerVB)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit this should be sat/vB since it's virtual bytes not bits.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, yes, thanks!

@apoelstra
Copy link
Member

CI failure is real.

pub fn display_sat_per_vb_round(self) -> Display {
Display {
fee_rate: self,
format: Format::SatPerVB { rounding: Rounding::Round },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I forgot to say yesterday, all these identifiers that use VB would be better as Vb, justified by the fact that display_sat_per_vb_round has 'vb'.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. VB is not an abbreviation comparable to things like HTTP, where Http makes sense. The B has to be capital to denote "byte", b means "bit"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few data points:

  • The vernacular is 'per vb'
  • The function names use 'vb' implying that 'vb' is a single word
  • It is surprising to see two capitals in a Rust identifier

Subjectively, no one is going to confuse bit and byte when discussing weight units or fee rates.

Copy link
Member

@tcharding tcharding May 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In documentation it is not that important (to me) but as an identifier, for those who type everything without relying on completion, the double capital is surprising.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vb is a single word

Nah, that's not a word but a standard unit with "v" prefix. v_b would be stupid.

If there didn't exist b unit I wouldn't care but I really, really don't like mixing these. I almost want to write the function name as vB but that is the one case where I'm willing to let it go.

If someone writes it without relying on completion I guarantee you Rustc will tell it has different capitalization.

(Side note: IMO even things like HTTP should've been just upper case, I have no idea why someone thought it'd be a good idea to do mixed case there.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The vernacular is 'per vb'

The vernacular is per vB

I am very ambivalent on this ... I tend to agree with @tcharding`s position that there won't be any confusion and it doesn't matter, but OTOH if you search "sat per vb" on the web you'll find almost every result uses "sat per vB" so clearly people out there are careful about this.

Bitcoin core seems very particular about vB. You'll find vB used through out the codebase. I think if we agree that vB is correct, then snake case SatPerVB is correct.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

funny, I found this instance of VBDeploymentInfo where the b is for bits. I think this would be wrong since b for bits should be lowercased.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's uppercase because it's an acronym for "version bits". If the b were lowercase it would suggest that "vb" were a word (but not suggest anything about its canonical capitalization).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's uppercase because it's an acronym for "version bits".

Right, good point. virtual bytes is also an acronym, so would not SatPerVB be correct then.

BTW I meant camel case above not snake case.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yancyribbens note that C++ style of Bitcoin Core has different capitalization than the standard Rust style. But it's indeed funny.

@tcharding
Copy link
Member

FeeRate is top priority at the moment @Kixunil. Its the last thing to get done before we release units-1.0.0-alpha.0.

@tcharding
Copy link
Member

Perhaps we just merge #4512 and then this PR can be done at our leisure?

@apoelstra
Copy link
Member

To be explicit: #4512 just drops the Display/FromStr impls. This PR puts them back. So concept ACK doing the removal before release and then the re-add later.

@Kixunil
Copy link
Collaborator Author

Kixunil commented May 27, 2025

I'm not happy with the conflicts caused by removal but whatever, it is done. IMO alpha->final could have the impl changed anyway since nobody should use alpha in production and the output is entirely clear.

@yancyribbens yancyribbens mentioned this pull request May 29, 2025
@yancyribbens
Copy link
Contributor

@Kixunil friendly ping to see what's up with this. Is there something that is holding this up?

@apoelstra
Copy link
Member

It needs a rebase and CI is failing, for two.

@Kixunil
Copy link
Collaborator Author

Kixunil commented Jul 2, 2025

Yeah, @tcharding made a huge break of my PR by removing the code and rebasing is a big mess, so I didn't have time to do it yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-units PRs modifying the units crate test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

FeeRate uses different units for Display and Display alternate
5 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy