iOS Drawing Performance: Why Core Graphics Slows to a Crawl and How to Fix It

Software Drawing

The term drawing usually refers to software drawing in the context of Core Animation (meaning: drawing not assisted by the GPU). In iOS, software drawing is typically done using the Core Graphics framework. However, in some necessary situations, Core Graphics is considerably slower compared to Core Animation and OpenGL.

Software drawing is not only inefficient but also consumes a significant amount of memory. CALayer only needs a small amount of memory related to itself: only its backing image consumes a certain amount of memory space. Even if you directly assign an image to the contents property, it does not require additional photo storage size. If the same image is used as the contents property for multiple layers, they will share the same memory block instead of copying it.

But once you implement the -drawLayer:inContext: method in the CALayerDelegate protocol or the -drawRect: method in UIView (which is essentially a wrapper for the former), the layer creates a drawing context. The memory size required by this context can be calculated from this formula: layer width * layer height * 4 bytes, where width and height are in pixels. For a full-screen layer on a Retina iPad, this memory amount is 204815264 bytes, equivalent to 12MB of memory. Each time the layer redraws, it needs to erase the memory and reallocate it.

Software drawing is expensive, and unless absolutely necessary, you should avoid redrawing your views. The secret to improving drawing performance lies in avoiding drawing as much as possible.

Vector Graphics

A common reason we use Core Graphics for drawing is that it's not easy to draw vector graphics using just images or layer effects. Vector drawing includes the following:

• Arbitrary polygons (not just rectangles)
• Lines or curves
• Text
• Gradients

For example, Listing 13.1 shows a basic line-drawing application. This application converts the user's touch gestures into points on a UIBezierPath and then draws them in the view. We implement all the drawing logic in a UIView subclass DrawingView, and in this case, we don't use a view controller. But if you prefer, you can implement the touch event handling in a view controller. Figure 13.1 is the result of running the code.

Listing 13.1 Implementing a Simple Drawing Application with Core Graphics

#import "DrawingView.h"

@interface DrawingView ()

@property (nonatomic, strong) UIBezierPath *path;

@end

@implementation DrawingView

- (void)awakeFromNib
{
    //create a mutable path
    self.path = [[UIBezierPath alloc] init];
    self.path.lineJoinStyle = kCGLineJoinRound;
    self.path.lineCapStyle = kCGLineCapRound;
    
    self.path.lineWidth = 5;
}

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event
{
    //get the starting point
    CGPoint point = [[touches anyObject] locationInView:self];

    //move the path drawing cursor to the starting point
    [self.path moveToPoint:point];
}

- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event
{
    //get the current point
    CGPoint point = [[touches anyObject] locationInView:self];

    //add a new line segment to our path
    [self.path addLineToPoint:point];

    //redraw the view
    [self setNeedsDisplay];
}

- (void)drawRect:(CGRect)rect
{
    //draw path
    [[UIColor clearColor] setFill];
    [[UIColor redColor] setStroke];
    [self.path stroke];
}
@end

Figure 13.1 Making a simple "sketch" with Core Graphics

The problem with this implementation is that the more we draw, the slower the program becomes. Because the entire Bezier path (UIBezierPath) is redrawn every time a finger moves, as the path becomes more complex, the work for each redraw increases, directly leading to a drop in frame rate. It seems we need a better approach.

Core Animation provides specialized classes for these types of graphic drawing and offers them hardware support (detailed in Chapter 6, "Specialized Layers"). CAShapeLayer can draw polygons, lines, and curves. CATextLayer can draw text. CAGradientLayer is used to draw gradients. These are generally faster than Core Graphics, and they also avoid creating a backing image.

If we slightly modify the previous code and replace Core Graphics with CAShapeLayer, performance will improve (see Listing 13.2). Although drawing performance will still decrease as the path complexity increases, a noticeable frame rate difference will only be felt with very, very complex drawing.

Listing 13.2 Reimplementing the Drawing Application with CAShapeLayer

#import "DrawingView.h"
#import <QuartzCore/QuartzCore.h>

@interface DrawingView ()

@property (nonatomic, strong) UIBezierPath *path;

@end

@implementation DrawingView

+ (Class)layerClass
{
    //this makes our view create a CAShapeLayer
    //instead of a CALayer for its backing layer
    return [CAShapeLayer class];
}

- (void)awakeFromNib
{
    //create a mutable path
    self.path = [[UIBezierPath alloc] init];

    //configure the layer
    CAShapeLayer *shapeLayer = (CAShapeLayer *)self.layer;
    shapeLayer.strokeColor = [UIColor redColor].CGColor;
    shapeLayer.fillColor = [UIColor clearColor].CGColor;
    shapeLayer.lineJoin = kCALineJoinRound;
    shapeLayer.lineCap = kCALineCapRound;
    shapeLayer.lineWidth = 5;
}

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event
{
    //get the starting point
    CGPoint point = [[touches anyObject] locationInView:self];

    //move the path drawing cursor to the starting point
    [self.path moveToPoint:point];
}

- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event
{
    //get the current point
    CGPoint point = [[touches anyObject] locationInView:self];

    //add a new line segment to our path
    [self.path addLineToPoint:point];

    //update the layer with a copy of the path
    ((CAShapeLayer *)self.layer).path = self.path.CGPath;
}
@end

Dirty Rectangles

Sometimes using CAShapeLayer or other vector graphic layers to replace Core Graphics is not so practical. For example, our drawing application: we perfectly completed vector drawing with lines. But imagine if we could further improve the application's performance, making it work like a blackboard and using "chalk" to draw lines. The simplest way to simulate chalk is to use a "line brush" image and paste it where the user's finger touches, but this method cannot be implemented with CAShapeLayer.

We could create an independent layer for each "line brush," but there is a big problem with implementation. The maximum number of layers allowed on the screen simultaneously is about a few hundred, and we would quickly exceed that. In this case, we have no choice but to use Core Graphics (unless you want to do something more complex with OpenGL).

The initial implementation of our "blackboard" application is shown in Listing 13.3. We changed the previous version of DrawingView, replacing UIBezierPath with an array of brush positions. Figure 13.2 is the running result.

Listing 13.3 A Simple Blackboard-like Application

#import "DrawingView.h"
#import <QuartzCore/QuartzCore.h>
#define BRUSH_SIZE 32

@interface DrawingView ()

@property (nonatomic, strong) NSMutableArray *strokes;

@end

@implementation DrawingView

- (void)awakeFromNib
{
    //create array
    self.strokes = [NSMutableArray array];
}

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event
{
    //get the starting point
    CGPoint point = [[touches anyObject] locationInView:self];

    //add brush stroke
    [self addBrushStrokeAtPoint:point];
}

- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event
{
    //get the touch point
    CGPoint point = [[touches anyObject] locationInView:self];

    //add brush stroke
    [self addBrushStrokeAtPoint:point];
}

- (void)addBrushStrokeAtPoint:(CGPoint)point
{
    //add brush stroke to array
    [self.strokes addObject:[NSValue valueWithCGPoint:point]];

    //needs redraw
    [self setNeedsDisplay];
}

- (void)drawRect:(CGRect)rect
{
    //redraw strokes
    for (NSValue *value in self.strokes) {
        //get point
        CGPoint point = [value CGPointValue];

        //get brush rect
        CGRect brushRect = CGRectMake(point.x - BRUSH_SIZE/2, point.y - BRUSH_SIZE/2, BRUSH_SIZE, BRUSH_SIZE);

        //draw brush stroke    
        [[UIImage imageNamed:@"Chalk.png"] drawInRect:brushRect];
    }
}
@end

Figure 13.2 Drawing a simple "sketch" programmatically

This implementation performs quite well on the simulator, but not so well on a real device. The problem is that every time a finger moves, we redraw the previous line brushes, even though most of the scene hasn't changed. The more we draw, the slower it gets. Over time, each redraw takes more time, and the frame rate drops (see Figure 13.3). How can we improve performance?

Figure 13.3 Frame rate and line quality degrade over time.

To reduce unnecessary drawing, Mac OS and iOS devices divide the screen into areas that need redrawing and areas that do not. The parts that need redrawing are called "dirty areas." In practice, given the complexity of clipping and compositing non-rectangular regions, a rectangular position containing the specified view is usually identified, and this position is the "dirty rectangle."

When a view has been modified, it may need to be redrawn. But in many cases, only a part of the view has changed, so redrawing the entire backing image is wasteful. However, Core Animation usually doesn't understand your custom drawing code, and it cannot calculate the location of the dirty area by itself. However, you can indeed provide this information.

When you detect that a specific part of a specified view or layer needs to be redrawn, you directly call -setNeedsDisplayInRect: to mark it, passing the affected rectangle as a parameter. This will call the view's -drawRect: (or the layer delegate's -drawLayer:inContext: method) during a view refresh cycle.

The CGContext parameter passed into -drawLayer:inContext: is automatically clipped to fit the corresponding rectangle. To determine the size of the rectangle, you can use the CGContextGetClipBoundingBox() method to get the size from the context. Calling -drawRect() is simpler because the CGRect is passed directly as a parameter.

You should limit your drawing work to this rectangle. Any drawing outside this area will be automatically ignored, but the CPU time spent calculating and discarding it is wasted, which is really not worth it.

Compared to relying on Core Graphics to redraw for you, clipping your own drawing area might help you avoid unnecessary operations. That said, if your clipping logic is quite complex, let Core Graphics do it for you. Remember: only do it yourself when you can do it efficiently.

Listing 13.4 shows an upgraded version of the -addBrushStrokeAtPoint: method, which only redraws the area near the current line brush. Additionally, it refreshes the area near the previous line brush; we can also use CGRectIntersectsRect() to avoid redrawing any old line brushes that don't cover the already updated area. Doing this significantly improves drawing efficiency (see Figure 13.4).

Listing 13.4 Using -setNeedsDisplayInRect: to Reduce Unnecessary Drawing

- (void)addBrushStrokeAtPoint:(CGPoint)point
{
    //add brush stroke to array
    [self.strokes addObject:[NSValue valueWithCGPoint:point]];

    //set dirty rect
    [self setNeedsDisplayInRect:[self brushRectForPoint:point]];
}

- (CGRect)brushRectForPoint:(CGPoint)point
{
    return CGRectMake(point.x - BRUSH_SIZE/2, point.y - BRUSH_SIZE/2, BRUSH_SIZE, BRUSH_SIZE);
}

- (void)drawRect:(CGRect)rect
{
    //redraw strokes
    for (NSValue *value in self.strokes) {
        //get point
        CGPoint point = [value CGPointValue];

        //get brush rect
        CGRect brushRect = [self brushRectForPoint:point];
        
        //only draw brush stroke if it intersects dirty rect
        if (CGRectIntersectsRect(rect, brushRect)) {
            //draw brush stroke
            [[UIImage imageNamed:@"Chalk.png"] drawInRect:brushRect];
        }
    }
}

Figure 13.4 Better frame rate and smooth lines

Asynchronous Drawing

UIKit's single-threaded nature means that the backing image usually needs to be updated on the main thread, which means drawing can interrupt user interaction and even make the entire app appear unresponsive. We can't do much about this, but it would be much better if we could avoid making the user wait for drawing to complete.

There are some methods to address this problem: in some cases, we can speculatively draw content on another thread in advance and then set the resulting image directly as the layer's content. This might not be very convenient to implement, but it is feasible in specific situations. Core Animation offers some options: CATiledLayer and the drawsAsynchronously property.

CATiledLayer

We briefly explored CATiledLayer in Chapter 6. In addition to subdividing the layer into independently updated small tiles (similar to the concept of automatic dirty rectangle updates), CATiledLayer has another interesting feature: it calls the -drawLayer:inContext: method for each tile simultaneously on multiple threads. This avoids blocking user interaction and can utilize multi-core chips to draw faster. A CATiledLayer with only one tile is a simple way to implement an asynchronously updating image view.

drawsAsynchronously

In iOS 6, Apple introduced this curious property for CALayer. The drawsAsynchronously property modifies the CGContext passed into -drawLayer:inContext:, allowing the CGContext to defer the execution of drawing commands so as not to block user interaction.

It is not the same as the asynchronous drawing used by CATiledLayer. Its own -drawLayer:inContext: method is only called on the main thread, but the CGContext does not wait for each drawing command to finish. Instead, it queues the commands, and when the method returns, the actual drawing is executed one by one on a background thread.

According to Apple, this feature works best on views that need frequent redrawing (like our drawing application, or things like UITableViewCell), and it doesn't help much for layer content that is drawn only once or rarely redrawn.

Summary

In this chapter, we mainly discussed some performance challenges surrounding software drawing with Core Graphics, and then explored some improvement methods: such as improving drawing performance or reducing the amount that needs to be drawn.

Chapter 14, "Image I/O," will discuss image loading performance.