Fix alpha compositing in ft2font's draw_bitmap. #30043
Conversation
src/ft2font.cpp
Outdated
| for (auto a = 0; a < 0x100; ++a) { | ||
| for (auto b = 0; b < 0x100; ++b) { |
There was a problem hiding this comment.
Can we rename a/b to dst/src so it's easier to correspond with the usage below?
There was a problem hiding this comment.
Sure (although note that the table is actually symmetric in the two arguments).
src/ft2font.cpp
Outdated
| auto table = std::array<uint8_t, 0x10000>{}; | ||
| for (auto a = 0; a < 0x100; ++a) { | ||
| for (auto b = 0; b < 0x100; ++b) { | ||
| table[(a << 8) + b] = a + b - (a * b + 0x7f) / 0xff; |
There was a problem hiding this comment.
In the FreeType docs for FT_Render_Glyph, it suggests that applying the bitmap should use the OVER operator:
dst = alpha * src + (1 - alpha) * dst ,
I'm not following if this is the same operation in 8-bit, or a different one.
It also suggests applying gamma correction, though I think that would not happen here, but in the application of the final bitmap to its colour on the Agg side.
There was a problem hiding this comment.
Actually, here we don't really want to composite the glyph over a pre-existing bitmap, but rather (because the rendering occurs to a temporary buffer where all the glyphs of the string will be drawn) compute the "total" alpha coverage map that will result from all the glyphs. (See the explanation on alpha coverage maps in the link you provided.) (Also, I am indeed ignoring gamma correction here.)
If I draw two glyphs with coverage a1 and a2 in 0-1 scale (and color src) at a given pixel, in that order, over a bitmap (dst), then the bitmap color becomes (by applying the OVER formula twice)
dst = a2 * src + (1 - a2) * (a1 * src + (1 - a1) * dst)
= (1 - (1 - a1) * (1 - a2)) * src + (1 - a1) * (1 - a2) * dst
i.e. it's as if I drew a single glyph with coverage atotal = 1 - (1 - a1) * (1 - a2) = a1 + a2 - a1 * a2 at that position (again, note that this is symmetric in a1 and a2).
Multuplying to 0-255 scale (A1 = 255*a1) yields Atotal = A1 + A2 - A1 * A2 / 255, which is the formula I implemented (where the +0x7f term leads to rounding in the division term rather than truncation).
There was a problem hiding this comment.
Can we add a comment saying it's alpha coverage merging then? The alpha_compositing name suggests something different, I think.
The old formula (`*dst |= *src`) works fine when either dst or src is
full transparent or fully opaque, but not for compositing intermediate
values. Fix that (while keeping a fast-path for the common case of
writing on an empty buffer).
Example (note the more uniform gray zone between the two letters):
```
from matplotlib import pyplot as plt, ft2font as f, cbook
import numpy as np
font = f.FT2Font(str(cbook._get_data_path("fonts/ttf/DejaVuSans.ttf")))
font.set_size(24, 72)
im = f.FT2Image(30, 30)
ga = font.load_char(ord("A"))
gv = font.load_char(ord("V"))
font.draw_glyph_to_bitmap(im, 2, 2, ga)
font.draw_glyph_to_bitmap(im, 12, 2, gv)
(plt.figure(layout="constrained", figsize=(3, 3))
.add_subplot(xticks=[], yticks=[])
.imshow(np.asarray(im), cmap="gray"))
plt.show()
```
|
Actually, #30059 (direct rendering into the Agg buffer) would be even more general than this, as we'd just rely on Agg's compositing. |
The old formula (
*dst |= *src) works fine when either dst or src is full transparent or fully opaque, but not for compositing intermediate values. Fix that (while keeping a fast-path for the common case of writing on an empty buffer).Example (note the more uniform gray zone between the two letters):
old/new:
Somewhat unsurprisingly, this also breaks a bunch of baseline images; maybe this could be squashed into the FreeType update (#29816)?
PR summary
PR checklist