Wednesday, 2 January 2013


I recently finished reading Learning CoreAudio by Chris Adamson and Kevin Avila, and for anyone looking at working with CoreAudio it is invaluable. Chris and Kevin take you through the complexities of the various CoreAudio APIs in a simple to understand way with a lot of very good examples. It is highly recommended.

It covers from the basics of AudioQueues right down to the lowlevel AudioUnit API, and also covers CoreMIDI and OpenAL on both OSX and iOS.

NSScrollView and CoreLayout

Recently I've been trying to convert an application to use CoreLayout and one of the problems I ran into was with getting content to correctly fill an NSScrollView.

I have an NSView subclass which lays out its subviews in a vertical list using constraints and while the height of the layout is obviously dependent on the combined height of the subviews (plus spacing) the children don't have any intrinsic width and so are dependent on the width the parent class gave them. Ultimately, this was the width which the NSScrollView gives, but the problem was that the NSScrollView wanted to give them 0 width.

The solution I came up with is to add a constraint to the NSClipView which sits inbetween the NSScrollView and my layout view like so:

[_layout setTranslatesAutoresizingMaskIntoConstraints:NO];
[_scrollView setDocumentView:_layout];

NSClipView *clipView = [_scrollView contentView];
NSDictionary *viewsDict = @{@"layout":_layout};

// Add our new constraint that will fix the sides of _layout
// to the sides of its parent, which in this case is clipView
[clipView addConstraints:[NSLayoutConstraint constraintsWithVisualFormat:@"|[layout]|" options:0 metrics:nil views:viewsDict]];

It was surprisingly easy once I realised that I could add constraints like this. Hopefully this will help some people if they run into issues like this.

Thursday, 15 September 2011

Custom cells in NSTableView (Part 1 - The NSCell version)

Before OSX 10.7 there was only one way of creating a custom cell: to subclass NSCell and do layout and drawing by hand. This is quite an awkward way as you can no longer use Interface Builder to create the layout and is potentially error prone. Then in Lion, the ability was added to use subclasses of NSViews for the cells, which means that they can be designed and laid out in Interface Builder and more complex bindings can be created.

In this article we'll look at the old NSCell based way and in a future article we'll look at the new 10.7 way and the other things that can be done with it. Our application will eventually look something like this, with each cell holding an image and two lines of text:

We'll start a new Cocoa application project in XCode and create a new class that derives from NSTextFieldCell. In our interface definition we'll add iVars for the second row of text and the image. The first row of text is handled by the NSTextField parent class.

@interface CustomCell : NSTextFieldCell {
    NSImage *image;
    NSString *subtitle;

@property (readwrite, retain) NSImage *image;
@property (readwrite, copy) NSString *subtitle;

Nothing too odd there. In the implementation we synthesize the iVars like normal, and don't need anything in the init method. The reference documentation for NSCell says that when subclassing it we need to implement four initialiser functions, init, initWithCoder:, initTextCell: and initImageCell: but we're subclassing NSTextFieldCell which takes care of that for us. One thing we do need to implement however is copyWithZone: as NSTableView likes to copy cells around.

- (id)copyWithZone:(NSZone *)zone
    CustomCell *cell = [super copyWithZone:zone];
    if (cell == nil) {
        return nil;
    // Clear the image and subtitle as they won't be retained
    cell->image = nil;
    cell->subtitle = nil;
    [cell setImage:[self image]];
    [cell setSubtitle:[self subtitle]];
    return cell;

When the cell is copied, the subtitle NSString and the NSImage will be copied with it however they won't be correctly retained. Therefore we need to set them to nil directly, before setting them because if we just called setImage: or setSubtitle: nothing would happen as we would be setting the same memory addresses. Similarly we need to access the values directly with -> for if we called [cell setImage:nil] the first thing it would do would be to release the old value which didn't have a matching retain and so would lead to a zombie object somewhere down the line.

The other method that we have to implement is drawInteriorWithFrame:inView:. This is where we do our drawing, one piece at a time. The most complicated part of this is working out where each bit goes. We will look at the functions that calculate the appropriate bounds further on below, first lets look at the drawing code.

- (void)drawInteriorWithFrame:(NSRect)cellFrame inView:(NSView *)controlView
    NSRect imageRect = [self imageRectForBounds:cellFrame];
    if (image) {
        [image drawInRect:imageRect 
    } else {
        NSBezierPath *path = [NSBezierPath bezierPathWithRect:imageRect];
        [[NSColor grayColor] set];
        [path fill];
    NSRect titleRect = [self titleRectForBounds:cellFrame];
    NSAttributedString *aTitle = [self attributedStringValue];
    if ([aTitle length] > 0) {
        [aTitle drawInRect:titleRect];
    NSRect subtitleRect = [self subtitleRectForBounds:cellFrame forTitleBounds:titleRect];
    NSAttributedString *aSubtitle = [self attributedSubtitleValue];
    if ([aSubtitle length] > 0) {
        [aSubtitle drawInRect:subtitleRect];

First off we draw the image, if it exists, or a grey square if it doesn't. The method to get the image's bounds is quite simple.

#define BORDER_SIZE 5
#define IMAGE_SIZE 64

- (NSRect)imageRectForBounds:(NSRect)bounds
    NSRect imageRect = bounds;
    imageRect.origin.x += BORDER_SIZE;
    imageRect.origin.y += BORDER_SIZE;
    imageRect.size.width = IMAGE_SIZE;
    imageRect.size.height = IMAGE_SIZE;
    return imageRect;

You'll notice that we don't clip imageRect to the bounds that are passed in. To do so would complicate the drawing of the image, because we want the image to always be drawn at 64x64, so if we clip the image rectangle, we need to also work out what part of the image needs drawn into that rectangle. There is nothing complicated in this method, we simply set the origin of the image to be BORDER_SIZE pixels away from the cell's bounds and set the size of the image to IMAGE_SIZE.

Next in the drawing method we work out where the first line of text is going to go:

- (NSRect)titleRectForBounds:(NSRect)bounds
    NSRect titleRect = bounds;
    titleRect.origin.x += IMAGE_SIZE + (BORDER_SIZE * 2);
    titleRect.origin.y += BORDER_SIZE;
    NSAttributedString *title = [self attributedStringValue];
    if (title) {
        titleRect.size = [title size];
    } else {
        titleRect.size = NSZeroSize;

    CGFloat maxX = NSMaxX(bounds);
    CGFloat maxWidth = maxX - NSMinX(titleRect);
    if (maxWidth < 0) {
        maxWidth = 0;
    titleRect.size.width = MIN(NSWidth(titleRect), maxWidth);
    return titleRect;

This method is slightly more complicated as this time we do need to clip the text bounds to the cell bounds and we need to limit the text width to the amount of space that is left in the cell after the image has been taken into consideration.

Finally we layout the second line of text. It needs the same clipping as the first line, but it also needs to know the bounds of the first line so that it can calculate it's origin correctly.

- (NSRect)subtitleRectForBounds:(NSRect)bounds forTitleBounds:(NSRect)titleBounds
    NSRect subtitleRect = bounds;
    if (!subtitle) {
        return NSZeroRect;
    subtitleRect.origin.x = NSMinX(titleBounds);
    subtitleRect.origin.y = NSMaxY(titleBounds) + BORDER_SIZE;
    CGFloat amountPast = NSMaxX(subtitleRect) - NSMaxX(bounds);
    if (amountPast > 0) {
        subtitleRect.size.width -= amountPast;
    return subtitleRect;

The only other method that is missing is the attributedSubtitleValue method, which allows us to have the text of the second line drawn in a shade of grey.

- (NSAttributedString *)attributedSubtitleValue
    NSAttributedString *astr = nil;
    if (subtitle) {
        NSColor *textColour = [self isHighlighted] ? [NSColor lightGrayColor] : [NSColor grayColor];
        NSDictionary *attrs = [NSDictionary dictionaryWithObjectsAndKeys:textColour,
                               NSForegroundColorAttributeName, nil];
        astr = [[[NSAttributedString alloc] initWithString:subtitle attributes:attrs] autorelease];
    return astr;

And that is our cell. Now we need to connect it up so that the NSTableView will display it instead of the default text cell. In our simple application, we have dragged an NSTableView into the window, turned off headers and set the number of columns to 1.
To tell Cocoa to display our custom cell select the cell in the NSTableView and in the Class section of the Identify Inspector enter the name of the cell class. Make sure you've selected the cell, to select it you will need to click a few times on the NSTableView. The first click will select the scrollview, the second will select the NSTableView, 3rd will select the column, then another click on the cell will select it.

One final thing we need to do is to set the size of the row. If we were using Lion's NSView mode for our NSTableView cells, then it would be possible for the NSTableView to calculate the size automatically, but as we are using the old fashioned NSCell mode we need to set it explicitly. This is set in the Size Inspector by selecting the NSTableView. For our example the height of the NSCell is the height of the image plus two borders, which works out as 74.

We need a model object to display the tableview: 

@interface CellInfo : NSObject <NSCopying> {
    NSString *title;
    NSString *subtitle;
    NSImage *image;

@property (readwrite, retain) NSString *title;
@property (readwrite, retain) NSString *subtitle;
@property (readwrite, retain) NSImage *image;

- (id)initWithTitle:(NSString *)_title subtitle:(NSString *)_subtitle image:(NSImage *)_image;


If you look closely you'll see that the model item needs to implement the NSCopying protocol. This is because we will be returning the item as the object for the cell, and somewhere in the depths of Cocoa this will be copied. To copy the item though, we'll just create a new one:

- (id)copyWithZone:(NSZone *)zone
    CellInfo *cellInfo = [[CellInfo alloc] initWithTitle:[self title]
                                                subtitle:[self subtitle]
                                                   image:[self image]];
    return cellInfo;

In the application delegate we need to create the model, creating an NSMutableArray in the init method and filling it with our model items.

cellInfos = [[NSMutableArray alloc] init];
for (int i = 0; i < 10; i++) {
    CellInfo *ci = [[CellInfo alloc] initWithTitle:@"Custom Cell"
                                              subtitle:[NSString stringWithFormat:@"Row number %d", i + 1]
                                                 image:[NSImage imageNamed:NSImageNameApplicationIcon]];
    [cellInfos addObject:ci];
    [ci release];

The application delegate should needs to implement the NSTableViewDataSource and NSTableViewDelegate protocols.

@interface CustomCellAppDelegate : NSObject <NSApplicationDelegate, NSTableViewDataSource, NSTableViewDelegate>

There are a number of different ways to get the data into the view, but for this example we will use the NSTableViewDataSource. We could use bindings but bindings only allow one value to be set on a cell whereas we need three so we would still need to use the data source protocol for the other two.

At a minimum for NSTableViewDataSource we need to implement two methods; numberOfRowsInTableView: and tableView:objectViewForTableColumn:row: 

- (NSInteger)numberOfRowsInTableView:(NSTableView *)tableView
    return [cellInfos count];

- (id)tableView:(NSTableView *)tableView 
objectValueForTableColumn:(NSTableColumn *)tableColumn 
    return [cellInfos objectAtIndex:row];

But this doesn't set the values on the cell. For this we need to implement an NSTableViewDelegate method; tableView:willDisplayCell:forTableColumn:row:. This method gives us the cell that is going to be displayed, and tells us what row it is, so we can get the object from the model and set the appropriate values on the cell.

- (void)tableView:(NSTableView *)tableView 
   forTableColumn:(NSTableColumn *)tableColumn
    CellInfo *info = [cellInfos objectAtIndex:row];
    CustomCell *cCell = (CustomCell *)cell;
    [cCell setImage:[info image]];
    [cCell setTitle:[info title]];
    [cCell setSubtitle:[info subtitle]];

Now finally we just need to set the application delegate as the dataSource and delegate for the NSTableView in Interface Builder by ctrl-dragging from the NSTableView onto the application delegate object twice. The first time selecting dataSource, and the second time selecting delegate.

And there you have it, a custom NSCell for NSTableViews. In the future we'll look at the Lion NSView version of this, and see the extra things that it can do for us.

Sunday, 21 August 2011

Reading from external controllers with CoreMIDI

As a relative newcomer to Cocoa and OSX some of Cocoa's frameworks seem very good, some I've looked at are what I expected, but recently I had to do some MIDI work and frankly, CoreMIDI is just weird and confusing. The documentation is limited, to say the least, and really not what I've come to expect from Apple. But I got what I needed done and so this post, more than some others, is to serve two purposes. One, to help me remember what I worked out, and two, maybe it'll help someone else.

I have a few MIDI controllers lying around, and what I wanted to do was to read the MIDI data as the various knobs and faders and buttons are manipulated and have it tell me what was happening.

In CoreMIDI we create input and output ports, and connect them to other ports or endpoints. For our case we want to create an input port, and connect this to the external MIDI controller. When we create an input port we also give it a callback function that will get called when the MIDI controller sends data.

The first step to create an input port is to create a MIDIClient

MIDIClientRef midiClient;

OSStatus result;
result = MIDIClientCreate(CFSTR("MIDI client"), NULL, NULL, &midiClient);
if (result != noErr) {
    NSLog(@"Error creating MIDI client: %s - %s",

The first parameter gives the MIDIClient a name and as we're working with CoreFoundation here we use the CFSTR macro. The second and third parameters are for receiving notifications when the MIDI system changes, but we don't care about that here and so just pass NULL. The final parameter is for obtaining our MIDIClientRef. A lot of the CoreMIDI functions return OSStatus to indicate success or failure, and it's a good idea to check there wasn't any errors before continuing. In the following code snippets, I'll leave out the error checking as it's the same as the check above to save space, but it is present in the example code.

Once we have a MIDIClient, we can then create an input port.

MIDIPortRef inputPort;
result = MIDIInputPortCreate(client, CFSTR("Input"), midiInputCallback, NULL, &inputPort);

The first parameter it takes is our MIDIClientRef that we created earlier, and the second is the name of the port. At the end, the final parameter is for obtaining our newly created port. The third and fourth parameters are the callback for whenever there is MIDI data available on the port. The third is the function and the fourth is data that will be passed into that function as context. Here we don't have any context data, but in a more complicated objective C program we would pass usually pass self or some other object. It is in this callback that the MIDI parsing happens. Care needs to be taken as it is also called on a separate, high priority thread, which means that any data access needs to be done in a thread safe manner, and any UI updating needs to be triggered on the main UI thread using performSelectorOnMainThread.

So what does this callback function look like then? Something like this:

static void
midiInputCallback (const MIDIPacketList *list,
                   void *procRef,
                   void *srcRef)
    NSLog(@"midiInputCallback was called");

We'll go into detail about what goes into it, and what the various parameters are for later, as first I want to get the rest of the ports hooked up.

So we want to get an endpoint on the external MIDI device and functions such as MIDIGetSource seem perfect for it, but they take an index to where the device is in the system. This is useful when you want to offer a list of devices for the user to choice the device from, but makes things more complicated for us. For this initial example we're going to use MIDIObjectFindByUniqueID but it needs a unique ID. To get this we need a helper program first.

A Brief Interlude

Make a new command line project, I called it MidiLister. Basically it is going to list all the MIDI devices and print their Unique IDs. First we need to add the CoreMIDI framework to it. If you select the MidiLister target, and go to the Build Phases page. Then in the section titled Link Binary With Libraries, click the plus symbol, type coremidi and select it.

Now in the main.m file import CoreMIDI/CoreMIDI.h and add the following code to the main function:

ItemCount numOfDevices = MIDIGetNumberOfDevices();
for (int i = 0; i < numOfDevices; i++) {
    MIDIDeviceRef midiDevice = MIDIGetDevice(i);
    NSDictionary *midiProperties;
    MIDIObjectGetProperties(midiDevice, (CFPropertyListRef *)&midiProperties, YES);
    NSLog(@"Midi properties: %d \n %@", i, midiProperties);

We're getting all the MIDI devices that the system knows about, and printing out their properties. If you build, run it, and look at the output somewhere in the output will be the MIDI device you want to use, and it will print something like this

    entities =     (
            destinations =             (
                    uniqueID = "-2123048758";
            embedded = 0;
            maxSysExSpeed = 3125;
            name = LPD8;
            sources =             (
                    uniqueID = 1759718006;
            uniqueID = "-2133248137";
    image = "/Library/Audio/MIDI Devices/Generic/Images/USBInterface.tiff";
    manufacturer = "AKAI professional LLC";
    model = LPD8;
    name = LPD8;

There may be a lot of devices and there are quite a lot of unique IDs for each device, each entity has a one and the device has one of its own. Which one do we want? Well, if you look through the output for something related to the MIDI controller you want to use. I knew that the model name for my controller was the Akai LPD8, and in this case we're wanting to receive data from the controller, so we want to use a source entity. This device only has one source, and its UniqueID is 1759718006. It will probably be different on your machine and for your controllers. Write the UniqueID down, and go back to the original project.

Now that we can use MIDIObjectFindByUniqueID to get the endpoint to use with this.

MIDIObjectRef endPoint;
MIDIObjectType foundObj;
result = MIDIObjectFindByUniqueID(1759718006, &endPoint, &foundObj);

Obviously in the above code, you should replace the uniqueID with the one for your device. We don't actually care about the foundObj, what we're interested in is the endPoint parameter and once we have that, we can link the input port that we created earlier to this endpoint.

result = MIDIPortConnectSource(inputPort, endPoint, NULL);

This function is quite straight forward, it connects the port that we created earlier (inputPort) to the end point that we got from our device (endPoint) and the last parameter is for some context data. Finally by adding a CFRunLoop so that any input events will be processed we will see that our callback is triggered whenever you do something on the MIDI controller.


Processing MIDI Events

Now that we have a program that can read MIDI events, we need to work out what the MIDI data is actually telling us. Unfortunately CoreMIDI is only used for configuring, sending and receiving MIDI data, it doesn't have anything to help with creating or parsing that MIDI data. Luckily the upside is that MIDI is a very simple protocol and there are various places online that describe it (without you needing to spend big bucks on the official MIDI spec). CoreMIDI passes the data to our callback function in a MIDIPacketList. Remember this was our callback function's protocol:

static void 
midiInputCallback (const MIDIPacketList *list,
                   void *procRef,
                   void *srcRef)

The MIDIPacketList parameter contains all the data related to the control change, and the two void * parameters are two pieces of context data. The first (procRef) is the context data that was set when the port was created via MIDIInputPortCreate, the second (srcRef) is the context data that was set when the port was connected to the endpoint with MIDIPortConnectSource

To process a MIDIPacketList we iterate through the list, processing the MIDIPackets that are contained in it. A MIDIPacket contains at least one complete MIDI message, except for Sysex messages which can be spread over multiple packets.

A MIDI message consists of a status byte, and then fixed number of bytes, depending on the message (apart from SysEx messages again but we'll ignore them for now). The high bits of the status byte tell us the message type, and the low bits tell us the channel number. So, for example, the status 0x83 tells us that a Note Off message was received from channel 4 (in MIDI messages the channels are numbered from 0-F, but in MIDI terminology they are numbered from 1-16, so 3 is really the fourth channel).

To find out the length of each message, is a pretty good guide. Looking at the 0x83 message again, we can see that Note Off has 2 bytes of data afterwards. This way we can iterate over the data contained in the MIDIPacket finding all the messages and acting on them.

As mentioned above, the exception to all this is the SysEx message. This message starts with 0xF0 (SysEx Start) and all the following data until the message 0xF7 (SysEx End) is part of the SysEx message. This means it could be over multiple MIDIPackets, but all the data for a SysEx message will not be spread over multiple MIDIPacketLists. Also, if there is a SysEx message in a MIDIPacket there will not be any other messages in that MIDIPacket. This simplifies processing SysEx messages considerably.

Here is an example of how you could process some MIDI messages in the midiInputCallback function. I'm not claiming this is perfect ands there's probably lots of places where it could be improved. If you know an improvement or spot an error, feel free to leave a comment.

#define SYSEX_LENGTH 1024

We define a maximum length for a SysEx message.

void midiInputCallback (const MIDIPacketList *list,
                        void *procRef,
                        void *srcRef)
    bool continueSysEx = false;
    UInt16 nBytes;
    const MIDIPacket *packet = &list->packet[0];

We get the first MIDIPacket in the list. Although we use ->packet[0] to get the first packet, the other packets are not accessed by ->packet[1], [2] etc. We need to use MIDIPacketNext,  as you'll notice at the end of the function.

    unsigned char sysExMessage[SYSEX_LENGTH];
    unsigned int sysExLength = 0;
This is just a buffer for collecting the SysEx messages. 

We start by going through all of the MIDIPackets in the MIDIPacketList

    for (unsigned int i = 0; i < list->numPackets; i++) {
        nBytes = packet->length;

We want to check if we're gathering a SysEx message that is spread over many MIDIPackets. If it is, then we need to copy the data into the message buffer.

        // Check if this is the end of a continued SysEx message
        if (continueSysEx) {
            unsigned int lengthToCopy = MIN (nBytes, SYSEX_LENGTH - sysExLength);
            // Copy the message into our SysEx message buffer,
            // making sure not to overrun the buffer
            memcpy(sysExMessage + sysExLength, packet->data, lengthToCopy);
            sysExLength += lengthToCopy;

Now we've copied the data, we check if the last byte is the SysEx End message.
            // Check if the last byte is SysEx End.
            continueSysEx = (packet->data[nBytes - 1] == 0xF7);

If we've finished the message, or if we've filled the buffer then we have  a complete SysEx message to process. Here we're not doing anything with it, but in a proper application we'd pass it to whatever acts on the MIDI messages.

            if (!continueSysEx || sysExLength == SYSEX_LENGTH) {
                // We would process the SysEx message here, as it is we're just ignoring it
                sysExLength = 0;
        } else {

If we weren't continuing a SysEx message then we need to iterate over all the bytes in the MIDIPacket parsing the messages that are contained in it.

            UInt16 iByte, size;
            iByte = 0;
            while (iByte < nBytes) {
                size = 0;
                // First byte should be status
                unsigned char status = packet->data[iByte];
                if (status < 0xC0) {
                    size = 3;
                } else if (status < 0xE0) {
                    size = 2;
                } else if (status < 0xF0) {
                    size = 3;
                } else if (status == 0xF0) {
                    // MIDI SysEx then we copy the rest of the message into the SysEx message buffer
                    unsigned int lengthLeftInMessage = nBytes - iByte;
                    unsigned int lengthToCopy = MIN (lengthLeftInMessage, SYSEX_LENGTH);
                    memcpy(sysExMessage + sysExLength, packet->data, lengthToCopy);
                    sysExLength += lengthToCopy;
                    size = 0;
                    iByte = nBytes;

                    // Check whether the message at the end is the end of the SysEx
                    continueSysEx = (packet->data[nBytes - 1] != 0xF7);
                } else if (status < 0xF3) {
                    size = 3;
                } else if (status == 0xF3) {
                    size = 2;
                } else {
                    size = 1;
                unsigned char messageType = status & 0xF0;
                unsigned char messageChannel = status & 0xF;

Now we know the size of each message, what type it is, and what channel it was received on and we can pass it off to something that will parse it. For this example, here is some code that just prints the message and the values. Ideally this would happen on a low priority thread so that it doesn't block the thread that receives the MIDI messages, but for this example it doesn't matter too much.
                switch (status & 0xF0) {
                    case 0x80:
                        NSLog(@"Note off: %d, %d", packet->data[iByte + 1], packet->data[iByte + 2]);
                    case 0x90:
                        NSLog(@"Note on: %d, %d", packet->data[iByte + 1], packet->data[iByte + 2]);
                    case 0xA0:
                        NSLog(@"Aftertouch: %d, %d", packet->data[iByte + 1], packet->data[iByte + 2]);
                    case 0xB0:
                        NSLog(@"Control message: %d, %d", packet->data[iByte + 1], packet->data[iByte + 2]);
                    case 0xC0:
                        NSLog(@"Program change: %d", packet->data[iByte + 1]);
                    case 0xD0:
                        NSLog(@"Change aftertouch: %d", packet->data[iByte + 1]);
                    case 0xE0:
                        NSLog(@"Pitch wheel: %d, %d", packet->data[iByte + 1], packet->data[iByte + 2]);
                        NSLog(@"Some other message");
                iByte += size;

As mentioned above, to get the next MIDIPacket you need to use MIDIPacketNext.

        packet = MIDIPacketNext(packet);

And thats how you read data from a MIDI controller. Hopefully this is useful to some people.

Apple documentation for CoreMIDI -