Cartoon shader improvements

I do think that it’s an improvement–at the least (as you point out) it inks edges that the previous version missed, and it lacks the blurriness of that version. There’s still some jagginess, but there are regions in which it’s softening the lines somewhat.

I hadn’t realised that you’re using the normals for your inking; the outlining that I’m doing at the moment is for a pencil-shader, and does pretty much what you seem to be suggesting: since I don’t care about the colour of the object for the final output, I give each object a unique colour, then render colour and normals. The normals are used for lighting and thus the actual pencil-shading (with a bit of contribution to the outlines), and the colour is used for edge-detection, with changes in sample colour identifying edges.

Here’s a quick mock-up of what I had in mind:



The result isn’t perfect: it lacks an understanding of the intention behind the edge, and the in the final image might be better with a small multiplier applied to the “run-length”, causing the pixel-value to fall off more quickly.

Indeed, this is the problem that I keep hitting, and a significant reason to doubt that my suggestion would likely work. :confused:

If there were some way of keeping state I do think that it could be made to work; moving the processing over to the CPU should enable that, but that seems likely to be very slow…

The only other thing that comes to mind is switching from a pixel-based approach to the inking to a geometry-based approach: before rendering, determine the “outline” edges and place cards (or a Meshdrawer-style trail) along them, drawing an appropriate texture along them.