Subpixel inking tested. Result below. Specular quantization and light-to-dark smoothing enabled so that the only option that changes is the inking type.
Comments? Better or worse than the previous attempt? Update the patch to use this?
Left: vanilla 1.8.1 inking
Right: subpixel inking (with the same settings)
Observe the ponytail holder and the outermost outline. Note also changes in internal outlines between the ponytail bunches (at the right in the image) and the edgemost front bang at the left edge of the image.
Beside adding subpixel sampling, I also made two slight changes to the computation:
- The original inker used only the first two components from the view normal map, while this version uses all three. This was the reason behind some of the missing outlines (note the chin).
- The ink calculation now uses an internal half-pixel shift, using the center point of the pixel as the reference. This visibly improved the results especially for the problematic side bangs in the picture.
It’s obvious that the red channel represents x (and the scaling is 0…255 for -1.0…1.0), but I’m still not sure which way the green and blue channels in the aux texture map to the y (up/down) or the z (in/out) directions in the view space (and what is the possible range for z, since backward pointing faces are usually not rendered). This is something that could use more documentation. (I was trying to find out whether a component, and which one, needs to be weighted twice as much as the others. Currently they all use the same weight.)
As for the side bangs, they’re somewhat of a challenge to ink. The problem is that there simply is not much variation in the view space normal where the two surfaces meet. Maybe a more advanced version could use a separate texture to identify different objects or materials, and ink those outlines too. (This would require some nontrivial changes to the main shader generator.)
One more thing about the subpixel approach. To eliminate noise, the code uses a voting method, where at least any 3 of the 9 subsamples must decide to ink the pixel before any ink is applied. The amount of ink is then linearly interpolated from 3 votes = very translucent to 9 votes = replace pixel with specified cartoon inking color. This improved the output slightly (again, especially in the side bangs) when compared to an initial version which did not use voting.