Palapeli: Multithreading, memory usage and interface

The Blue Wonder Meeting last week was an excellent opportunity to do some work on Palapeli. One thing that I wanted to do for at least a year was to rewrite the multithreading code around the (fairly central) Palapeli::Puzzle class. The new class has an aspect-oriented design, where single components of the puzzle (like the metadata or the contents) are always made available by worker threads, by casting existing components. For example, a puzzle file can be casted into metadata by reading the latter from the puzzle file. The possible casting paths are actually so convoluted that I documented them in an image inside the source code directory:

Yet there are still areas where Palapeli’s concurrency needs to improve. Much of the puzzle loading is done in the GUI thread, which causes the loading animation to freeze frequently. But that’s for another day. Moving back from what I should do to what I actually did, I noticed that Palapeli’s memory usage is awful. During the sprint, I wanted to demo Palapeli’s performance with big piece counts, and created a puzzle with the maximum of 2000 pieces from a 5000×5000 px image. It actually rendered fluently, but only because I upgraded my notebook’s RAM to 2 GB last year. Palapeli used an enormous 1.6 gigabytes of memory.

I figured that’s a bit much, and looked yesterday evening into how this can be optimized. I was testing with a 30 pieces puzzle from a 4000x4000px image. After having started Palapeli and having loaded the puzzle, the application used about 800 MB memory. The first, and simple, optimization was to remove some intermediary images which are only used while loading the image. That saved something like 100 MB of memory. But wait! Shouldn’t a 4000×4000 pixel image need only 61 MB (four bytes per pixel)? Yes, that’s true, but actually we’re not talking about one big image, but about 30 images, one for each piece. While pieces might have an arbitrary shape, their image must internally be rectangular all the time, even if big parts (e.g. around the plugs) are fully transparent. This means that the memory usage for the 30 pieces is bigger than what one might expect from the complete image.

A randomly chosen piece was 855×1098 pixels big, so by a rough approximation, all 30 pieces take 30*855*1098*4 bytes = 107 MB in memory. That still does not explain why the process still occupies 700 MB. But what’s that?

There’s a shadow around the pieces! This shadow is not added by the slicer when the puzzle is created, but by the engine when the puzzle is loaded. And because we need to control how shadows cover other pieces, we cannot render the shadows into the piece image, but must store them as separate images! And to make matters even worse, we need different shadow images for selected and unselected pieces (note the difference between blue and black shadows). The shadow size for the test image is 0.15 * (width + height) = 300 pixels on each side. Putting everything together, the memory usage of piece images + normal shadows + selected shadows is about 700 MB, i.e. everything can be explained by the pure size of the displayed images. The second optimization therefore was to restrict the shadow size to 50 pixels. This reduces the memory consumption to 380 MB, and allows to play even 5000×5000 px (i.e. 25 megapixel) images with below 1 GB of RAM.

More sophisticated optimizations are certainly possible, but let’s stop at this 50% improvement for now. There’s another thing I want to show: Palapeli’s mainwindow has been cleaned up. I really liked the old tab interface. It had its time and justification, but I want to move forward and need a cleaner, more standard interface for that. So now Palapeli has only a menubar and toolbar, just like all the other apps:

This change was possible after I realized that all toolbar actions for the collection also make sense on the puzzle table: Delete and Export work on the current puzzle. Newly created or imported puzzled will be loaded into the puzzle table afterwards. The “Back to collection” button replaces the “My collection” and “Puzzle table” tabs.