Papervision’s dirty little secret (rectangle) now cleaned up!

Any of you using Andy’s ViewportLayers (see his post here) to selectively render separate layers will now find a fairly significant improvement in performance. And it’s because we’ve managed to shrink Flash’s dirty rectangle!

The dirty rectangle is the area of the screen that is being redrawn by the Flash Player every frame, and you can see it by turning on “Show Redraw Regions” in the right click menu of the debug player. One of the optimisations in the player is that it only redraws the rectangle that contains everything that has changed between frames, and the smaller the rectangle is, the faster it renders.

So if you have a very complicated background with lots of shapes, but only move a small object across it, it only renders the area around the small object, and you should get a fast frame rate.

By default, Papervision3D re-renders the entire scene every frame, even if you’re not moving the camera. But using Andy’s viewport layers system, you can put objects into different layers and then selectively render them. (Using renderer.renderLayers(…))

So let’s say you have a room, and in that room are several items of furniture. And in the middle there’s a spinning Jedi training orb (that one’s for John Grden 🙂 ).

If the only thing in the room that’s moving is the training sphere, then there’s no point in rendering everything else. So you put the sphere in its own viewport layer and call renderer.renderLayers and pass it through an array with the sphere viewport layer in. And now you’re only rendering the sphere! Or are you…?

I had a system that worked like this, except when I turned on the “Show Redraw Regions” the dirty rectangle was still showing that the whole scene was being redrawn. I went through PV’s render process with a fine tooth comb and categorically confirmed that it was only my single object that was being redrawn (and Andy has done a GREAT job on ViewportLayers by the way). So why was the whole screen dirty and therefore getting re-rendered?

After much investigation I discovered the culprit. During the render process the viewport layers are sorted in the display list by their z depth position. So that objects within layers appear correctly on top of each other. The code goes something like this :

// sort the layers relative to how far they are from the camera
allLayers.sortOn("zDepth");

for(var i:int =0 ; i<allLayers.length; i++)
{
   // and change their position in the display list
   setChildIndex(layer[i],i);
}

And what’s wrong with that? Well nothing. Except that changing a DisplayObject’s position in the display list marks it as being dirty by the Flash Player and requires a re-render, even if we’re setting it to the index that it’s already at!

So possibly the smallest fix I’ve ever made to Papervision (with perhaps the most effort in investigations) : now we only setChildIndex for the layer if its index has changed.

if(getChildIndex(layer)!=i) setChildIndex(layer, i);

I can’t tell you how happy I was when I found this. It took me hours of digging and it was so delightful to see my dirty rectangle shrink. (I should just clarify that this is a weirdness in the Flash Player and NOT a bug in Papervision!)

As it happens there are some other unexpected ways to mark a DisplayObject as requiring rendering by the FlashPlayer, but I’ll do some more investigations and give you a full report at some point in the future.