[css-compositing-2] Add plus-lighter to mix-blend-mode and background-blend-mode

[jakearchibald]

Fixes https://github.com/w3c/csswg-drafts/issues/6821.
Tests web-platform-tests/wpt#31757.

This allows any two elements to be cross-faded together. More details: https://jakearchibald.com/2021/dom-cross-fade/.

I'm not really sure about the difference between "blend" and "composite", particularly since globalCompositeOperation in canvas includes both.

We might eventually get to the stage where mix-blend-mode and globalCompositeOperation have the same set of operations, which is kinda weird since they have different names.

[jakearchibald]

I'm going to try and write some tests for this, but I haven't written CSS tests before. Wish me luck!

[jakearchibald]

@tabatkins are the tests in the right format for this? The other tests look like ref tests, but maybe there's a more modern way?

[jakearchibald]

#446 - wondering if plus-lighter is meaningfully different to lighter. If it turns out 'no', I'll switch to lighter in this PR.

[jakearchibald]

The more I think about it, moving lighter from a compositing mode to a blend mode is cheating. I'll think of a better way to do this.

@cabanier maybe you have thoughts here? Right now globalCompositingOperation has both blend modes and composite modes. Maybe the right thing to do is make mix-blend-mode the same. However, Chrome is only interested in implementing lighter right now.

Should the spec change be limited to what Chrome's currently prepared to implement, or converge with globalCompositingOperation hoping that browsers get to implementing it eventually?

[cabanier]

The more I think about it, moving lighter from a compositing mode to a blend mode is cheating. I'll think of a better way to do this.

Yes, don't make it into a blend mode. It doesn't solve the issue with clamping.
Are you looking into making it apply to CSS or only canvas?

@cabanier maybe you have thoughts here? Right now globalCompositingOperation has both blend modes and composite modes. Maybe the right thing to do is make mix-blend-mode the same. However, Chrome is only interested in implementing lighter right now.

The plan (from 8 years ago?) was to make another css property for compositing but that never happened.
I was never sure about the compositing part because it feels too low level.

Should the spec change be limited to what Chrome's currently prepared to implement, or converge with globalCompositingOperation hoping that browsers get to implementing it eventually?

Is Chrome looking into implementing cross-fade in css?

[jakearchibald]

Are you looking into making it apply to CSS or only canvas?

The purpose of this PR is to allow developers to make an element composite using lighter. Among other things, it allows developers to cross-fade elements https://jakearchibald.com/2021/dom-cross-fade/

lighter is already supported in canvas, so the aim here is to add it to CSS.

The plan (from 8 years ago?) was to make another css property for compositing but that never happened.
I was never sure about the compositing part because it feels too low level.

Yeah, some combinations of blend & composite don't seem to make much sense.

Is Chrome looking into implementing cross-fade in css?

Not cross-fade() (we already support that as -webkit-cross-fade()), but the ability to cross-fade two DOM elements.

I've updated the PR to use 'lighter' rather than 'plus-lighter'.

[cabanier]

Is Chrome looking into implementing cross-fade in css?

Not cross-fade() (we already support that as -webkit-cross-fade()), but the ability to cross-fade two DOM elements.

I've updated the PR to use 'lighter' rather than 'plus-lighter'.

If the intent is for cross-fading, making lighter a compositing mode won't work because it won't behave as expected when there's foreground or background alpha.
You either need to create a new css property that just does the cross fading or introduce compositing.

[jakearchibald]

Are you sure? I think isolation does the trick, no? Fwiw it behaves as expected in canvas. See the demos in https://jakearchibald.com/2021/dom-cross-fade/

[cabanier]

Are you sure? I think isolation does the trick, no? Fwiw it behaves as expected in canvas. See the demos in https://jakearchibald.com/2021/dom-cross-fade/

Maybe it does the right thing. The alpha of the backdrop determines how much of the blend operation is applied. It's possible that this ends up doing what is expected.

[jakearchibald]

I think it does. It's what Chrome uses for -webkit-cross-fade() https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/graphics/crossfade_generated_image.cc;l=73

[cabanier]

I think it does. It's what Chrome uses for -webkit-cross-fade() https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/graphics/crossfade_generated_image.cc;l=73

Maybe... That code is setting a "blend mode" which is really a compositing mode.
The easiest way to check this is to do write a test page that blends in javascript on a canvas.

[khushalsagar]

@cabanier I'm a bit confused about how blend-mode vs compositing-mode are supposed to be different.

There are existing blend modes which define how the source and backdrop color values should be combined (the blending step) and then how this combined result is modulated by the backdrop alpha (the compositing step). For example, the lighten blend-mode states something similar to what we want to do with lighter, the backdrop is replaced with the result of blending the 2 color values instead of any further compositing operation. Is that not a valid way to specify a mix-blend-mode?

[cabanier]

A blend mode changes the color of the foreground color based on the blending formula and the alpha of the backdrop.
This changed color is then fed into the compositing mode (which is always source-over).

[khushalsagar]

This changed color is then fed into the compositing mode (which is always source-over)

That's the part I'm unsure about. The text for lighten says, "The backdrop is replaced with the source where the source is lighter". Are we doing source over compositing as defined below :

co = αs x Cs + αb x Cb x (1 – αs), where

Cs = max(Cs, Cb) via the blending operation
Cb = same Cb as the blending step

[cabanier]

The spec has a step by step explanation how blending is done: https://drafts.fxtf.org/compositing-1/#blending

[khushalsagar]

Thank you. This makes it much clearer now. You're right that what we want here is "lighter" composite operator with "normal" blend mode. And there is currently no way to change the composite operator in CSS.

Would it be reasonable to include composite operators in mix-blend-mode and follow a pattern similar to globalCompositeOperation. The latter lets you specify a blend mode (in which case we default to source-over compositing) or composite operator (in which case we default to normal blending) : https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/graphics/graphics_types.cc;l=56;drc=d7044ac79851029fa1698731d9ce5c251a5939be

We can have mix-blend-mode provide the same capability as globalCompositeOperation in CSS. It's unclear why a developer shouldn't be able to specify any combination of the 2 but the syntax could be extended to permit that for both mix-blend-mode and globalCompositeOperation going forward.

[cabanier]

Would it be reasonable to include composite operators in mix-blend-mode and follow a pattern similar to globalCompositeOperation.

I'm unsure. At the time, it seemed to me that these compositing modes were too primitive to expose directly which is why we punted them to later.
I think that if you want to create a certain effect with compositing, you should create that effect instead.

[khushalsagar]

I think that if you want to create a certain effect with compositing, you should create that effect instead.

Hmmm, I didn't follow the alternate being proposed. Could you clarify how developers should do this instead of being able to configure the compositing of 2 elements?

A demo showing the exact use-case is here. It's using canvas to mimic the capability needed in CSS. We have 2 elements with partially identical pixels that draw on top of each other and we need to ensure that blending the identical pixels is a no-op.

[jakearchibald]

This PR is ready to land. It's unclear if plus-darker should be part of this, or if plus-darker should be removed from the spec altogether. That can be addressed in another PR.

@cabanier what are the next steps?

[cabanier]

@cabanier what are the next steps?

It should be simple enough to create this as an experiment in a version of chromium that you built yourself. If the elements can be blended by regular drawing (and not by the compositor), you just need to add it to skia. You could even just replace a blend mode with plus-darker.

I'm still unsure if the compositing mode will always work in the way you expect. Also, blending creates a stacking context out of the content that is blending but not the content that you are blending with. This could cause some confusion for authors.

[cabanier]

I'm still unsure if the compositing mode will always work in the way you expect. Also, blending creates a stacking context out of the content that is blending but not the content that you are blending with. This could cause some confusion for authors.

Regardless, the CSSWG resolved to add it. Is there something specific about the PR that needs improvement before landing?

OK then. Not sure why you need my input if it's set in stone.

[chrishtr]

Not set in stone, but you're the editor so we need your code review to land the PR. If there is a specific testcase you have in mind that we should check in the prototype, or a WPT example to add, we're happy to do that.

If your concern is a general one about fit for our desired use cases, that's legit but IMO the best way to find that out is by testing with partners later on, not blocking the a PR for an initial spec if we don't know why it is problematic...

[cabanier]

I agree.
Great to hear that you implemented this behind a flag already. @jakearchibald, did you already experiment with the flag and determine that it satisfies your use case?
I don't see any problems adding it to background-blend-mode because that feature doesn't deal with layout and stacking contexts.
For mix-blend-mode, it's more complicated so we need examples and author feedback.

[jakearchibald]

Yep, if I take a nightly from https://download-chromium.appspot.com/ and use the --enable-blink-features=CSSMixBlendModePlusLighter flag, the tests pass, as does the practical example at https://jakearchibald.com/2021/dom-cross-fade/#update-it-already-works-in-safari

[chrishtr]

Furthermore, the layout effects of causing a stacking context are not a problem for the desired use cases.

Rik, why do you think more author feedback is needed for that case? mix-blend-mode already causes a stacking context, and that is necessary in order for it to be well-defined and implementable in general. plus-lighter is no exception.

[cabanier]

Rik, why do you think more author feedback is needed for that case? mix-blend-mode already causes a stacking context, and that is necessary in order for it to be well-defined and implementable in general. plus-lighter is no exception.

The stated use case for this feature was: This allows any two elements to be cross-faded together.
I'm assuming that this means that this feature is supposed to fade between 2 images or text blocks which mix-blend-mode doesn't do. Can you post an example where 2 text blocks are faded that doesn't need a lot more additional CSS to line things up? It's possible that I'm overlooking something...

[chrishtr]

The example here is such a case: https://jakearchibald.com/2021/dom-cross-fade/#update-it-already-works-in-safari.

It has two sibling SVG elements with mix-blend-mode: plus-lighter and a containing element with isolation:isolate (which causes a stacking context).

The CSS to do this is not a lot - just the one containing div and 3 lines of CSS.

[cabanier]

I was thinking of an HTML example. Couldn't you could already do this in SVG with feComposite?

[chrishtr]

What we need is a method that works for all kinds of HTML content, including in particular for replaced elements containing snapshot images from a current or previous document. I don't think SVG is general enough to cover all such cases.

[jakearchibald]

@cabanier I only used SVG in that example because the scaling worked well for the article. I can make a demo using DOM elements if I really need to, but SVG elements are also DOM elements so that demo does the job IMO

[cabanier]

@jakearchibald Yes, please make an example with HTML elements to see how most people are going to use this feature.

[mstange]

Can you post an example where 2 text blocks are faded that doesn't need a lot more additional CSS to line things up?

With CSS Grid you can line up "stacked" elements very easily. In this example, all direct child elements of .wrapper will occupy the same area:

.wrapper { display: grid; }
.wrapper > * { grid-area: 1 / 1 / 2 / 2; }

[jakearchibald]

Yep! That's the pattern I recommended in https://youtu.be/PYSOnC2CrD8 too

[jakearchibald]

@cabanier https://static-misc-3.glitch.me/composite-test/mix-blend-mode-dom.html - here's a demo using only DOM elements.

Of course, the cross-fade is likely to be an animation triggered by a click, but the slider used on this page makes it easier to see the difference between the default compositing method (source-over) and plus-lighter.

Summary of material relating to this:

• Blog post with demos detailing the problem, and how plus-lighter solves it.
• A video covering the issue, with animated examples and explanation.
• More demos showing how various techniques work in canvas, including "lighter" which achieves a good cross-fade
• Demo of mix-blend mode using plus-lighter on two DOM nodes (<svg> in this case), which highlights a couple of WebKit bugs.
• As above but using two <div>s rather than <svg>
• Web platform tests which show the feature behaving as expected.

[cabanier]

Nice! I didn't know you could do it with grid :-)

[jakearchibald]

@cabanier is this good to land then? Anything else I need to do?

[cabanier]

@cabanier is this good to land then? Anything else I need to do?

This looks good!
I do have a question about the implementation. Was it correctly implemented so the timing of the plus-ligher operation is not dependent on color or alpha value?

[jakearchibald]

I'm not sure what you mean by 'timing', can you explain a bit?

[cabanier]

After we shipped this feature with blend modes that weren't susceptible to timing, someone on the skia team refactored the formulas which enabled an attack. See https://arstechnica.com/information-technology/2018/05/chrome-and-firefox-leaks-let-sites-steal-visitors-facebook-names-profile-pics/
You need to make sure that plus-lighter executes the same regardless of its input. Since there's a min(..., ...), there might be an if block in the implementation which would introduce a timing attack.

[jakearchibald]

Gotcha. We'll double check that.

That shouldn't block the spec landing though, right? It's an implementation detail.

[cabanier]

Gotcha. We'll double check that.

That shouldn't block the spec landing though, right? It's an implementation detail.

Yes, the spec already specifies this but it's easily overlooked.

[jakearchibald]

Can the spec and tests land now, then?

[cabanier]

Yes :-)

[jakearchibald]

Thanks for reviewing and landing this!