I have this idea for an app that I would totally use myself. You know I started XcodeJobs.com and the @XcodeJobs twitter feed to go with it to have a channel to retweet all the iOS-related job offerings that flow before my very eyes. For the site I’ve been talking to people to create a login for themselves and post their jobs self-servingly. For the twitter feed I’ve been using the Twitter search features with a variety of search terms to find tweets where company owners of the developers themselves are tweeting about job postings.

Now the reality of the matter is that most of all Tweets are from recruiters, agencies, job sites and other kinds of services that a self-respecting iOS developer does not want anything to do with. So right now I’m manually filtering tweets. There are a variety of criteria that I want to be able to combine to end up at the real good retweet-worthy tweets.

For example I would do a search for “hiring iOS” …

Criteria for exclusion might include:

• Jack Patrick I found to be a recruiter
• Jobely.com and Jobrep.com are examples for job websites
• The jobmakesfun is pointing to a Freelancer site with a ridiculous project description.
• The one for Viki is pointing to a dubious URL that does not seem to correspond with the hiring company.

On the above screenshot the one tweet by Fable Technologies is the only one I deemed worthy of retweeting. It does not contain link bait, is by the guys for themselves, no commercially interested middle man in sight and they are so daring as to even post their e-mail address.

All of these manual steps surely can be automated. I’m envisioning several black lists of twitter accounts that I want to ignore. And the second big category would be to ignore tweets that are leading to job sites (like Bullhorn Reach) or agencies. The user (i.e. myself) could configure multiple elaborate search queries and have the app filter out all the crap. So ideally this would leave us with the true gems of Twitterdom.

What Twitter Engine to Use?

So I sat down in front of my home iMac and started to think to myself: “OMG that will be hell of a job. Maybe I should go look for a good Twitter engine to use first.”

The reason for this train of thought was that nowadays most of the interaction with the Twitter REST API has to be authenticated via OAuth. Which is why there are a bunch of OAuth frameworks in the wild, including one from Google because their APIs are OAuth-heavy as well. OAuth basically works by having a secret token that you have to add to all URL requests that need authentication. Of course one could program this from scratch but you’d certainly spend quite some time on this. Even using a third party OAuth library would not be straightforward because of the time it takes to figure out how to include it in your app.

All of these reasons would make this small project unrealistic for a couple of stolen hours on a Sunday.

Some people recommended MGTwitterEngine, but then it dawned on me that there was something new that Apple had introduced with iOS 5. So I checked the SDK documentation and – indeed! – besides the TWTweetComposeViewController that everybody now knows about there is a second class that is exactly what we need: TWRequest.

Tw…TW…TWRequest?

Hidden behind the simplest of names is a wrapper that you can do any kind of request with. Think of it as a block-based wrapper around NSURLConnection with all the trimmings needed to also do authentication where necessary.

Let’s dive right in, and start by doing a simple search. We added the Twitter.framework to the linked libraries and in the class where we want to call TWRequest we added <Twitter/Twitter.h>. I prefer to put that in the PCH file so that it is available everywhere without needing an extra inclusion of the header.

The search itself is deceptively simple:

NSURL *searchURL = [NSURL URLWithString:@"http://search.twitter.com/search.json"];
NSDictionary *parameters = [NSDictionary dictionaryWithObject:@"hiring ios" forKey:@"q"];
TWRequest *request = [[TWRequest alloc] initWithURL:searchURL parameters:parameters requestMethod:TWRequestMethodGET];
 
[request performRequestWithHandler:^(NSData *responseData, NSHTTPURLResponse *urlResponse, NSError *error) {
    if (responseData)
    {
        NSError *parseError = nil;
        id json = [NSJSONSerialization JSONObjectWithData:responseData options:0 error:&parseError];
 
        if (!json)
        {
            NSLog(@"Parse Error: %@", parseError);
        }
        else
        {
            NSLog(@"%@", json);
        }
    }
    else
    {
        NSLog(@"Request Error: %@", [error localizedDescription]);
    }
}];

To perform a Twitter request you need to have a couple of ingredients, which you can glean from the Twitter API Documentation, for example for the search function.

We needed the URL which is a composite of:

• the so-called endpoint: http://search.twitter.com
• the method name: search
• the response data format: json

The parameters need to be provided in a dictionary which is also straightforward, the keys are the parameter names and the values are the parameter values. Also we don’t need authentication for search, in fact it is not even supported according to the Twitter docs. This is to make it impossible for Twitter to track you based on what you search for… unlike Google.

TWRequests supports the three HTTP verbs GET, POST and DELETE. The documentation tells you which one to use. Note that different API functions might have different end points.

We could have also chosen xml format for the response format, but there is no simple way to transform that into something useful for us. NSXMLParser works event-based and we would have to do some work to get it into a structure. The second ingredient that we need is also new with iOS 5, Apple provides a handy high performance class to serialize and deserialize JSON data.By passing the responds of the TWRequest into NSJSONSerialization we get an NSDictionary containing the parsed JSON. With JSON and the de-serialization method provided we get a dictionary which is way more convenient.

If the deserialization worked you get a non-nil result, looking like this:

{
    "completed_in" = "0.027";
    "max_id" = 204150933297446912;
    "max_id_str" = 204150933297446912;
    "next_page" = "?page=2&max_id=204150933297446912&q=hiring%20ios";
    page = 1;
    query = "hiring+ios";
    "refresh_url" = "?since_id=204150933297446912&q=hiring%20ios";
    results =     (
                {
            "created_at" = "Sun, 20 May 2012 10:05:50 +0000";
            "from_user" = "fun_programming";
            "from_user_id" = 282845320;
            "from_user_id_str" = 282845320;
            "from_user_name" = jobmakesfun;
            geo = "";
            id = 204150933297446912;
            "id_str" = 204150933297446912;
            "iso_language_code" = en;
            metadata =             {
                "result_type" = recent;
            };
            "profile_image_url" = "http://a0.twimg.com/profile_images/1313148959/jobs_normal.gif";
            "profile_image_url_https" = "https://si0.twimg.com/profile_images/1313148959/jobs_normal.gif";
            source = "<a href="http://twitterfeed.com" rel="nofollow">twitterfeed</a>";
            text = "#ipad #project iOS Game Side Scrolling Battle by olimoli123: I require someone to make a game like ... http://t.co/LNqWsmLs #dev #hiring";
            "to_user" = "";
            "to_user_id" = 0;
            "to_user_id_str" = 0;
            "to_user_name" = "";
        },
                {
            "created_at" = "Sun, 20 May 2012 08:18:14 +0000";
            "from_user" = XcodeJobs;
            "from_user_id" = 539747758;
            "from_user_id_str" = 539747758;
            "from_user_name" = "Xcode Jobs";
            geo = "";
            id = 204123856812785665;
            "id_str" = 204123856812785665;
            "iso_language_code" = pl;
            metadata =             {
                "result_type" = recent;
            };
            "profile_image_url" = "http://a0.twimg.com/profile_images/1988345754/xcode-twitter_normal.png";
            "profile_image_url_https" = "https://si0.twimg.com/profile_images/1988345754/xcode-twitter_normal.png";
            source = "<a href="http://angel.co" rel="nofollow">AngelList</a>";
            text = "RT @CardFlick: CardFlick (@CardFlick) is hiring a iOS Engineer http://t.co/O7Wf0gtw";
            "to_user" = "";
            "to_user_id" = 0;
            "to_user_id_str" = 0;
            "to_user_name" = "";
        },...

We get the tweets in an NSArray below the “results” key, each tweet being represented by an NSDictionary. If we wanted to refresh the contents of this query we could use the value stated in “refresh_url”. Then we would only see tweets that were made after the ones we had already gotten here.

Also results are paginated. To retrieve subsequent pages there is a “next_page” value that would get us those. The default is to return 15 tweets per “page”. We could easily also increase this number by passing it as the optional “rpp” (results per page) parameter.

Later the same day …

I’m skipping over the part where I’m merging the result dictionary into a CoreData database and using an NSFetchedResultsController to display the tweets. That is a different story which I shall treat another day. Let’s just say, I got it to display in a very basic form and pushing a refresh button would redo the same query and merge the results into a database.

After getting all the basics wired up (CoreData Stack, methods to query by ID for user and message, merging function, table view controller, fetched results controller) I ended up with a view like this:

It is immediately apparent from this that we are seeing only tweets that we want to get rid of. 4 of these Twitter accounts would be blacklisted because of the reasons stated above. Also my own tweets are visible as they came from my own account and start with RT, those where the job tweets I had retweeted. OMG the amount of time this tool will save me not having to visually scan through all of this…

But like every TV cook I’ve already prepared this step as the details are of no interest for this tutorial. I added a long press gesture recognizer to each cell and a red “Blacklist” button that allows me to mark the TwitterUser as blacklisted and remove his tweets from the database. This results in a tremendous improvement.

Oh, look! Fable Technologies is at the top after having blacklisted all the clutter. They should pay me a referral fee for all this free advertising!

Now for the hard part of this tutorial. Now that we have leaned up the result, we also want to be able to retweet individual messages and of course somehow know if we have done so already to prevent a double-retweeting. Unfortunately there does not seem to be an easy we to get the retweet info included in the search results.

Dealing with Accounts

In order to do something that requires authentication we need to ask the global account store for a list of Twitter accounts and have the user pick one. First we add the Accounts.framework in our build phases. Then we add <Accounts/Accounts.h> to our PCH file.

The first time we want access to a given account type (as of iOS 5 only Twitter is supported) we need to ask the user’s permission. Subsequently we can query the ACAccountType if permission has already been given.

- (void)account:(UIBarButtonItem *)sender
{
    ACAccountStore *accountStore = [[ACAccountStore alloc] init];
    ACAccountType *accountType = [accountStore accountTypeWithAccountTypeIdentifier:ACAccountTypeIdentifierTwitter];
 
    if ([accountType accessGranted])
    {
        // have access already
        [self _showListOfTwitterAccountsFromStore:accountStore];
    }
    else
    {
        // need access first
        [accountStore requestAccessToAccountsWithType:accountType withCompletionHandler:^(BOOL granted, NSError *error) {
 
            if (granted)
            {
                [self _showListOfTwitterAccountsFromStore:accountStore];
            }
            else
            {
                UIAlertView *alert = [[UIAlertView alloc] initWithTitle:@"Error" message:@"Cannot link account without permission" delegate:nil cancelButtonTitle:@"Ok" otherButtonTitles:nil];
                [alert show];
            }
        }];
    }
}

There is no shared instance of the ACAccountStore, so we have to create our own with alloc/init. Then we create the ACAccountType for Twitter and check if we might already have access granted. If not then we ask for it passing a completion handler. For sake of simplicity I opted to just show an action sheet for all the configured accounts. This does not deal with the situation of no configured accounts.

- (void)_showListOfTwitterAccountsFromStore:(ACAccountStore *)accountStore
{
    ACAccountType *accountType = [accountStore accountTypeWithAccountTypeIdentifier:ACAccountTypeIdentifierTwitter];
    NSArray *twitterAccounts = [accountStore accountsWithAccountType:accountType];
 
    UIActionSheet *actions = [[UIActionSheet alloc] initWithTitle:@"Choose Account to Use" delegate:self cancelButtonTitle:@"Cancel" destructiveButtonTitle:nil otherButtonTitles:nil];
    actions.tag = 2;
 
    NSMutableArray *shownAccounts = [NSMutableArray array];
 
    for (ACAccount *oneAccount in twitterAccounts)
    {
        [actions addButtonWithTitle:oneAccount.username];
        [shownAccounts addObject:oneAccount];
    }
 
    _shownAccounts = [shownAccounts copy];
 
    [actions showInView:self.view];
}

Since I don’t have any block-based action sheet handy I need to remember the list of accounts presented in _shownAccounts IVAR and then in the delegate method for the action sheet I can pick the correct one based on the index. The important thing here is that we end up with an instance of a ACAccount.

Now Let’s Sign This…

Let’s quickly check the Retweet API and what we need to perform such a request.

Just like above for the search we need to construct a URL from these parts:

• the endpoint: http://api.twitter.com/1
• the method name: statuses/retweet
• the identifier of the tweet to retweet: some number
• the response data format: json
• no parameters necessary

Here I stumbled across something that was not immediately obvious to me. You have to store the ACAccountStore in an instance variable. Otherwise the ACAccount’s ACAccountType will be released by the autorelease pool. At let’s that’s what happened to me, I got a memory exception and an authentication error in alternation. Could it be that there’s a bug in the Accounts framework? Seems to me that the accountType property of ACAccount should be a strong reference, but it isn’t.

But that aside, there’s no hurt making an IVAR for the account store, the function to retweet would then look like this:

- (void)_retweetMessage:(TwitterMessage *)message
{
    NSString *retweetString = [NSString stringWithFormat:@"http://api.twitter.com/1/statuses/retweet/%@.json", message.identifier];
    NSURL *retweetURL = [NSURL URLWithString:retweetString];
    TWRequest *request = [[TWRequest alloc] initWithURL:retweetURL parameters:nil requestMethod:TWRequestMethodPOST];
    request.account = _usedAccount;
 
    [request performRequestWithHandler:^(NSData *responseData, NSHTTPURLResponse *urlResponse, NSError *error) {
        if (responseData)
        {
            NSError *parseError = nil;
            id json = [NSJSONSerialization JSONObjectWithData:responseData options:0 error:&parseError];
 
            if (!json)
            {
                NSLog(@"Parse Error: %@", parseError);
            }
            else
            {
                NSLog(@"%@", json);
            }
        }
        else
        {
            NSLog(@"Request Error: %@", [error localizedDescription]);
        }
    }];
}

There are three differences here versus the search request. 1) this one is a POST, 2) there are no parameters and 3) we set the account property to the ACAccount we have selected earlier. And if this works we indeed see the retweets show up immediately.

If you get an authentication error, then there’s a problem with the ACAccount. If you get some error complaining that this cannot be done with the given tweet, then most likely you have already retweeted it before. And in case of success you get a response with some retweet status.

Conclusion

There are a couple more touches necessary to make this app really useful. For one, we also need to retrieve the timeline of own retweets to be able to mark the tweets we already looked at. Also the retweet response should somehow be used to mark the tweet as retweeted.

And of course the UI could use much more work. We want to asynchronously show the profile pictures, access and edit the blacklist, decode the shortened URLs and make a blacklist based on those and of course be able to specify multiple search query for a given topic. For the UI I’ll probably use my DTCoreText method that lets me render the tweets with interactive hyperlinks.

But the gist of this post is that if you know about the Twitter and the Account frameworks then it is quite easy to build your own specialized Twitter client. We can extend a heart-felt “Thank You!” to the engineers at Apple who built something as useful as these into the operating system. Interacting with Twitter couldn’t be easier!

When shrink-wrapping your code for later reuse you inadvertenly will come into the situation that you have some resources (strings, XIBs, images et al) in your project that you also want to be reused. So what do you do?

If only we had frameworks on iOS … then we could bundle the resources together with the code in a framework. But Apple does not want us to compile frameworks in Xcode since these could potentially contain code downloaded after the app review process.

Popular projects like ShareKit or the Facebook iOS SDK have approached this dilemma by simply putting all resources into a folder, giving it the “.bundle” extension and instruct users of their SDK to also add this bundle to the “Copy Bundle Resources” step of their respective apps.

In this here blog post I will show you a smarter way.

There are several problems with having resources contained in a bundle and having this bundle be a member of any projects.

Not Visible

Xcode does not directly “see” the contents of the bundle, instead the pbxproj only contains a reference to the bundle folder. This causes trouble for apps like our Linguan that parses the project file to find strings (and soon XIB) files. It simply cannot see them.

Here’s how it looks in the ShareKit project. If you ever see a x.bundle in a project you open, armed with the knowledge in this article, your reaction should be “that’s bad”.

Of course the maintainers of these projects “have their reasons”. But I hope that by reading this you will agree with me that the reasons to not do it like this are better.

Not Optimized

Another issue is that this approach effectively disables the build time optimizations that Xcode carries out on the resources.

• strings files get converted into binary property lists
• XIB files get compiled into binary NIBs
• images get pngcrush-ed
• and many other actions for which there are build rules set up

That means resources that are simply bundled (by means of copying them together) are slower and not optimized for the mobile devices. You probably wouldn’t notice that for only a hand full of items, but if you have a large number of resources then these delays will add up slowing down your app. And even if that does not bother you very much then the engineer inside of you should cringe. It just feels so dirty…

Not Updating

Another reason for why it is a bad idea to hide files from Xcode is that it simply won’t know to rebuild your app if you make a change in one of the files hidden in such a bundle.

The Xcode build system has what is called dependencies. Implicit dependencies are source code files and resources that are part of certain targets. If such a dependence is modified then the incremental build process can determine which parts need to be re-compiled or re-optimized.

Say you change something in a single .m file. Xcode will not rebuild the entire app, but only create the .o for this updated file and then link it with the previously built (and unchanged .o files). Same with resources. Xcode only replaces resources in the output product .app bundle if it knows they where modified.

Resources in static bundles are invisible to Xcode and thus you always have to clean your build folder when doing a new build after changing them. Otherwise your updates will not propagate into your app.

Enter the Resource Bundle Target

Xcode, iOS and OS X have a mechanism to deal with folders that are looking like a single file but are actually containing multiple resources. This mechanism is modeled by the NSBundle class. You probably have worked with bundles before, namely the .app bundles that make up your app. Have you ever written [NSBundle mainBundle] before? I bet you did.

For NSBundle to be able to manage bundles it requires a special info.plist inside the bundle that contains some meta information, like an internal identifier. But once you have the bundle set up correctly you have multiple great options for getting at the files as I will show you below.

How to Set Up a Resource Bundle Product

A resource bundle is a product that we will set up a target for. It just so happens that my DTPinLockController project is in need of some love, so it will serve as our example for this tutorial.

As a first step I needed to move the files into the modern way of structuring my projects. That is, for component projects I have a Core and a Demo folder at the project root. Then each has a Source and a Resources sub-folder. DTPinLockController has XIBs, Images and Localizable.strings files.

The source code goes into a Static Library target. All resources go into a Resource Bundle target. I’ve omitted the setup for the library here as we want to focus our attention on the resources.

Next we need to set up the Resource Bundle target. The template for this can be found in the OS X section, under “Framework & Library”.

When removing the dynamic framework template Apple also removed the template for the Bundle. But with some minor modifications we can change the Mac bundle into one suitable for iOS. Since our bundle will not contain any executable code we don’t care about the settings for ARC and which foundation we want to use.

I like to name the bundle product that same as my component so that in the end I have a DTPinLockController.bundle containing my resources and a libDTPinLockController.a library to go with it.

The template creates 3 files in DTPinLockController:

• an empty InfoPlist.strings file, we don’t need that
• a DTPinLockController-Info.plist file, this is the META plist we need
• a DTPinLockController-Prefix.pch file. No code in bundle means we can also remove that.

I grab the plist and move it in to the root of my resources folder. The rest we can safely remove. I also rename it to Resources-Info.plist as to give an unsuspecting observer a hint that this is for the resource bundle.

We use this approach because the info plist for the resource bundle is too long to manually create. Note that there are several placeholders that get filled in during the build process, like the bundle name. All these settings come from the build settings which we are going to adjust next.

The default setting is to use the target name as name of products. I don’t like this because as I said about I want several products to be named DTPinLockController, but have the targets reflect their actual purpose.

Because of this I end up with 3 targets with descriptive names like “Demo App”, “Static Library” and “Resource Bundle”. These have different product names “Demo”, “DTPinLockController” and “DTPinLockController” respectively. The different extensions and the prefix “lib” are automatically added by Xcode.

There are all the modifications we need to do on the build settings for the resource bundle:

• CMD-Backspace on the Base SDK to have it be the same as for the entire project: Latest iOS
• Same on the “Mac OS X Deployment Target”
• Same on the Architectures
• Remove the reference to the PCH file
• Remove “Installation Directory”
• Set “Skip Install” to YES
• Adjust “Info.plist File” to the correct path, e.g. Core/Resources/Resources-Info.plist

The important part is the correct paths for the info plist and pch (none) and that Skip Install is YES because otherwise you get problems when trying to archive an app using the resource bundle. Since there is no code the compiler-related settings are really inconsequential, but I like to have them inherit from the project settings to have it look like a native iOS target.

In the build phases of the bundle target there is still a framework in “Link Binary with Libraries”. Remove this as well. It does not hurt as there is nothing to link it with, but might be confusing.

The final step for building the target is now to add the resources to it. You select the appropriate files in the project tree and set the checkmark in the right panel next to Resource Bundle.

Xcode has added a scheme for the new target with the original name. So I remove all schemes and auto generate them from scratch. This will create one scheme per target and name them the same as the targets.

Then we can try and see if the bundle builds correctly.

you can open up each build step and see that Xcode carried out the optimizations I alluded to above. In the case of this strings file you see –outputencoding binary that tells us that the strings file in the bundle will actually be binary.

You now have a DTPinLockController.bundle in the Products group that you can inspect to verify that this indeed is the case.

Using the Resource Bundle

There is a bug in Xcode 4.3.2 that might cause the list of resource to copy for the app target might get out of sync with the check marks in the project navigator. I had to manually go in the build phases of the demo app target and remove all the references to resources that are now part of the resource bundle.

Since the resource bundle is now a proper target we can add it as dependency to apps using it. This tells Xcode that if this dependency is somehow “dirty” then the app target is also in need of updating. So it will first build the dependent target and then include the product in the app build process.

This shows the Static Library set up as dependency and in “Link Binary With Libraries”. Now we also want the bundle product to be in the “Copy Bundle Resources” phase. Click the Plus button and add the bundle product.

This will make the bundle appear in the “Copy Bundle Resources”. If it wasn’t built before the app target then it will be built first, causing it to appear in the build products folder and from there it will be copied and included in the app product.

A Few Code Changes

If you reference images from a XIB and you put these images in the same resource bundle as the XIB then you don’t have to change anything. XIBs will load images from the same bundle that they were instantiated from.

However there are a couple of changes you need to make to your code so that the resources can be found at their new location.

Strings

The default macros for getting localized strings look for the strings files (aka “string tables”) in the main app bundle. These are the definitions of the 4 default macros and you can see that the bottom two have a way to specify the bundle to get the strings from whereas the first two hardcode the mainBundle.

#define NSLocalizedString(key, comment) \
	    [[NSBundle mainBundle] localizedStringForKey:(key) value:@"" table:nil]
#define NSLocalizedStringFromTable(key, tbl, comment) \
	    [[NSBundle mainBundle] localizedStringForKey:(key) value:@"" table:(tbl)]
#define NSLocalizedStringFromTableInBundle(key, tbl, bundle, comment) \
	    [bundle localizedStringForKey:(key) value:@"" table:(tbl)]
#define NSLocalizedStringWithDefaultValue(key, tbl, bundle, val, comment) \
	    [bundle localizedStringForKey:(key) value:(val) table:(tbl)]

You can create your own custom macro or category or what-have-you to have a shortcut, but to promote understanding here’s the entire code that we need to first get an NSBundle instance from our resource bundle and then get one string from it.

Note that specifying a table of nil means that the strings file is called “Localizable.strings”, for a table name of “Name” the file is called “Name.strings”

// get the resource bundle
NSString *resourceBundlePath = [[NSBundle mainBundle] pathForResource:@"DTPinLockController" ofType:@"bundle"];
NSBundle *resourceBundle = [NSBundle bundleWithPath:resourceBundlePath];
 
// get a string
NSString *string = NSLocalizedStringFromTableInBundle(@"Set Passcode", @"DTPinLockController", resourceBundle, @"PinLock");

We probably want to have a category on NSBundle specific to our project to load and cache in a static variable the resource bundle. And to go with that a localized string macro that hard codes this resource bundle.

XIBs

You probably have several places where you call [super initWithNibName:@"MyViewController" bundle:nil]. The nil in this case causes the NIB loader to assume that you mean the main bundle. Just as easily we can get the resource bundle and pass there here instead.

// get the resource bundle
NSString *resourceBundlePath = [[NSBundle mainBundle] pathForResource:@"DTPinLockController" ofType:@"bundle"];
NSBundle *resourceBundle = [NSBundle bundleWithPath:resourceBundlePath];
 
// load View Controller from that
UIViewController *vc = [[MyViewController alloc] initWithNibName:@"MyViewController" bundle:resourceBundle];

Do we see a pattern here? You betcha! We always first get the NSBundle instance for the resource bundle and then pass this as a parameter to some method that does something with the resource.

Images

Ah, Graphics. With strings and XIBs both working the same way you would only be right to assume that there’s a imageNamed:inBundle: method, BUT… it is private. Radar rdar://10250430 by Cedric Luthi addresses this.

But fortunately for us we don’t really need this special method. The regular imageNamed can work with our resource bundles too!

All you need to do is prefix your image names with the name of the bundle, like this:

// load image from resource bundle
UIImage *image = [UIImage imageNamed:@"DTPinLockController.bundle/Image.png"];

This works because besides being a bundle object the resource bundle is also folder that can be traversed via its path.

The cool thing about loading XIBs, strings and images from resource bundles like this is that you retain the awesome powers of these methods. Like pathForResource:ofType: will automatically deliver the correct language version for the device if the resource is localized. Or imageNamed: will still automatically load Retina graphics where applicable or device-specific images with ~ipad or ~iphone.

I told you before that you don’t need to do anything special if these images are referenced from XIBs contained in the same resource bundle. So this is only necessary for the cases where you load the images from code.

Conclusion

This simplified tutorial has all targets and resource consumers in the same project. But the same concepts also work if you add the component project as a sub-project.

I hope that I could convince you of the many advantages of resource bundles over static bundles. I’ve been successfully using them in almost all of my commercial components and many internal projects.

Bundle targets allow you to streamline the resource building process in a way that make larger projects way more effective and less error prone. Make it so!

I’m working on my own file and image cache that uses CoreData for storage. The same way that NSURLCache is doing it, but with some optimizations that I know and understand. So I created DTDownloadCache and got it all working, but there was one minor thing that I didn’t like: The usual method of creating a CoreData entity model is by the entity editor built into Xcode.

This meant that I had to include the .datamodeld file in apps using this. But I didn’t want to have to create a resource bundle just for this single file as you would have to do if you keep your reusable code in static libraries. Ugh!

Fortunately there is a way how you can create a static model entirely in code so that you can make use of CoreData without having to ship an XML description of the model.

First, lets have a quick look at how the model looked like in the editor.

Plain and simple. The only special thing is to allow external storage for the fileData attribute. This allows CoreData to decide where to most efficiently store the file data. Small files will be stored in the entity itself. Large files get a GUID and will be saved in a subfolder of Library/Caches. If we wanted to support iOS 4 then we would have to create this mechanism ourselves which is available as of iOS 5.

In XML this is represented as:

The model is what CoreData needs to know how to translate between the SQLite database on disk and the managed objects in memory. Of course we could embed this model as a binary resource, but that would make out code unnecessarily more convoluted. I found a good example (Core Data Utility Tutorial) in the Apple documentation that explains the process.

The method to create the same model as in the editor in code looks like this. ARC = On.

- (NSManagedObjectModel *)_model
{
	NSManagedObjectModel *model = [[NSManagedObjectModel alloc] init];
 
	// create the entity
	NSEntityDescription *entity = [[NSEntityDescription alloc] init];
	[entity setName:@"DTCachedFile"];
	[entity setManagedObjectClassName:@"DTCachedFile"];
 
	// create the attributes
	NSMutableArray *properties = [NSMutableArray array];
 
	NSAttributeDescription *remoteURLAttribute = [[NSAttributeDescription alloc] init];
	[remoteURLAttribute setName:@"remoteURL"];
	[remoteURLAttribute setAttributeType:NSStringAttributeType];
	[remoteURLAttribute setOptional:NO];
	[remoteURLAttribute setIndexed:YES];
	[properties addObject:remoteURLAttribute];
 
	NSAttributeDescription *fileDataAttribute = [[NSAttributeDescription alloc] init];
	[fileDataAttribute setName:@"fileData"];
	[fileDataAttribute setAttributeType:NSBinaryDataAttributeType];
	[fileDataAttribute setOptional:NO];
	[fileDataAttribute setAllowsExternalBinaryDataStorage:YES];
	[properties addObject:fileDataAttribute];
 
	NSAttributeDescription *lastAccessDateAttribute = [[NSAttributeDescription alloc] init];
	[lastAccessDateAttribute setName:@"lastAccessDate"];
	[lastAccessDateAttribute setAttributeType:NSDateAttributeType];
	[lastAccessDateAttribute setOptional:NO];
	[properties addObject:lastAccessDateAttribute];
 
	NSAttributeDescription *expirationDateAttribute = [[NSAttributeDescription alloc] init];
	[expirationDateAttribute setName:@"expirationDate"];
	[expirationDateAttribute setAttributeType:NSDateAttributeType];
	[expirationDateAttribute setOptional:NO];
	[properties addObject:expirationDateAttribute];
 
	NSAttributeDescription *contentTypeAttribute = [[NSAttributeDescription alloc] init];
	[contentTypeAttribute setName:@"contentType"];
	[contentTypeAttribute setAttributeType:NSStringAttributeType];
	[contentTypeAttribute setOptional:YES];
	[properties addObject:contentTypeAttribute];
 
	NSAttributeDescription *fileSizeAttribute = [[NSAttributeDescription alloc] init];
	[fileSizeAttribute setName:@"fileSize"];
	[fileSizeAttribute setAttributeType:NSInteger32AttributeType];
	[fileSizeAttribute setOptional:NO];
	[properties addObject:fileSizeAttribute];
 
	NSAttributeDescription *entityTagIdentifierAttribute = [[NSAttributeDescription alloc] init];
	[entityTagIdentifierAttribute setName:@"entityTagIdentifier"];
	[entityTagIdentifierAttribute setAttributeType:NSStringAttributeType];
	[entityTagIdentifierAttribute setOptional:YES];
	[properties addObject:entityTagIdentifierAttribute];
 
	// add attributes to entity
	[entity setProperties:properties];
 
	// add entity to model
	[model setEntities:[NSArray arrayWithObject:entity]];
 
	return model;
}

This looks more complicated than it really is because of the verbosity of Objective-C. The process is simply to create an empty model, create an entity, create the attributes, stir it all up and cook on medium flame for 15 minutes. If you compare this with the XML model above then you’ll see a couple of differences, but those are merely because I made my code model more strict, with less optional properties.

The bare bones CoreData stack – consisting of the model, the persistent store coordinator and the managed object context might look like this:

- (void)_setupCoreDataStack
{
	// setup managed object model
 
	/*
	NSURL *modelURL = [[NSBundle mainBundle] URLForResource:@"DTDownloadCache" withExtension:@"momd"];
	_managedObjectModel = [[NSManagedObjectModel alloc] initWithContentsOfURL:modelURL];
	 */
 
	// in code
	_managedObjectModel = [self _model]; 
 
	// setup persistent store coordinator
	NSURL *storeURL = [NSURL fileURLWithPath:[[NSString cachesPath] stringByAppendingPathComponent:@"DTDownload.cache"]];
 
	NSError *error = nil;
	_persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:_managedObjectModel];
 
	if (![_persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:storeURL options:nil error:&error]) 
	{
		// inconsistent model/store
		[[NSFileManager defaultManager] removeItemAtURL:storeURL error:NULL];
 
		// retry once
		if (![_persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:storeURL options:nil error:&error]) 
		{
			NSLog(@"Unresolved error %@, %@", error, [error userInfo]);
			abort();
		}
	}
 
	// create MOC
	_managedObjectContext = [[NSManagedObjectContext alloc] init];
	[_managedObjectContext setPersistentStoreCoordinator:_persistentStoreCoordinator];
}

Xcode turns the XML model into a momd file when building. The line that loads the model from this file is now commented out and instead the model is being constructed at runtime.

You can modify the model until the first time it is used to access the persistent store. After that time it is read-only because changing it then would make it impossible for CoreData to talk to the data it has already saved.

Also this approach does not bother with migrating models if something is changed. We simply remove the store file and retry adding the persistent store to the store coordinator. If that works then CoreData has created a fresh SQLite database on disk that fits our new model.

Conclusion

CoreData is powerful to be used for maintaining the META information of caches and with the above approach you can easily keep your custom caches in static libraries without the need of adding a clunky model file to apps that are linking in this lib.

The experts are still out as to the motivations behind Facebook’s iOS SDK strategy. But it is rather clear that if Facebook has their way then everybody is to be using their Single Sign-On (SSO) technique. Besides all potential advantages of having this SSO in place it has to leave your app for signing on.

Not exactly something that is useful for all use cases. We have one case (involving ShareKit) which works better with the old style of signing into Facebook. This “traditional approach” shows the login dialog in a web pop up instead of leaving the app.

In this post I’m sharing the 3 methods how to hack the Facebook class and bend it to our will.

When researching for a way to disable SSO and restore the “old style” you inadvertently come across several people telling you to simply hack the authorize: method in Facebook.m.

Original Code Hack

The code for it is Open Source and so that poses little problem for a simple modification. You simply replace the method with the version below:

- (void)authorize:(NSArray *)permissions 
{
	[self setValue:permissions forKey:@"permissions"];
 
	[self authorizeWithFBAppAuth:NO safariAuth:NO];
}

But unfortunately that means you’ll have to modify code that is owned by somebody else. And I’m not talking about licensing concerns but the simple fact that if Facebook updates the SDK you have to do this change every time after updating to the latest version.

I am using the Facebook SDK via a submodule in ShareKit and I generally want to avoid having to keep track of modifications in third-party code. Which brings us to method number 2.

Category Hack

We know that we can add methods to existing classes by simply adding them in categories. If you have a method that has the same name as an existing method then the new method simply replaces the old one. So we can wrap the above hack in a category like so:

Facebook+NoSSO.h

#import "Facebook.h"
 
@interface Facebook (NoSSO)
 
- (void)authorize:(NSArray *)permissions;
 
@end

Facebook+NoSSO.m

#import "Facebook+NoSSO.h"
 
#import "Facebook.h"
 
@interface Facebook ()
 
- (void)authorizeWithFBAppAuth:(BOOL)tryFBAppAuth
                    safariAuth:(BOOL)trySafariAuth;
@end
 
@implementation Facebook (NoSSO)
 
- (void)authorize:(NSArray *)permissions 
{
	[self setValue:permissions forKey:@"permissions"];
 
	[self authorizeWithFBAppAuth:NO safariAuth:NO];
}
 
@end

Notice that we need to define the authorizeWithFBAppAuth:safariAuth: in an anonymous category so that the compiler accepts us calling it from our own authorize: method. Without that we would get a compiler warning. This method is internal to the Facebook class and not exposed via the header.

This method has two problems: 1) I don’t think that Apple actually guarantees a certain order in which classes and categories are loaded. So this is a bit risky to rely on it having always been working. 2) As of Xcode 4.3.2 the compiler will actually warn you “Category is implementing a method which will also be implemented by its primary class”

Which brings us to the third – most elegant method.

Swizzeling Hack

Being dynamic in nature the Objective-C runtime allows us to dynamically changes classes and methods which our app is running. One such technique is called “Method Swizzeling” which basically means to exchange two methods. With this method you can substitute your own method implantation.

First I borrowed some code from ShareKit and adapted it for general use in DTFoundation:

NSObject+DTRuntime.h

@interface NSObject (DTRuntime)
 
+ (void)swizzleMethod:(SEL)selector withMethod:(SEL)otherSelector;
 
@end

NSObject+DTRuntime.m

#import <objc/runtime.h>
 
@implementation NSObject (DTRuntime)
 
+ (void)swizzleMethod:(SEL)selector withMethod:(SEL)otherSelector
{
	// my own class is being targetted
	Class c = [self class];
 
	// get the methods from the selectors
	Method originalMethod = class_getInstanceMethod(c, selector);
	Method otherMethod = class_getInstanceMethod(c, otherSelector);
 
	if (class_addMethod(c, selector, method_getImplementation(otherMethod), method_getTypeEncoding(otherMethod)))
	{
		class_replaceMethod(c, otherSelector, method_getImplementation(originalMethod), method_getTypeEncoding(originalMethod));
	}
	else
	{
		method_exchangeImplementations(originalMethod, otherMethod);
	}
}
 
@end

That takes care of the actual exchanging. I have to admit that I don’t quite understand the if, because in my tests it always ended up in the method_exchangeImplementations part. But as long as it is working…

Now we have to only have a method with a unique name (to avoid the warning) and then we can exchange this method for the original authorize: method by means of this swizzle. The modified category now looks like this:

Facebook+NoSSO.h

#import "Facebook.h"
 
@interface Facebook (iCatalog)
 
- (void)authorize_noSSO:(NSArray *)permissions;
 
+ (void)toggleSingleSignOn;
 
@end

Facebook+NoSSO.m

#import "Facebook+iCatalog.h"
#import "NSObject+DTRuntime.h"
 
@interface Facebook ()
 
- (void)authorizeWithFBAppAuth:(BOOL)tryFBAppAuth
                    safariAuth:(BOOL)trySafariAuth;
@end
 
@implementation Facebook (iCatalog)
 
- (void)authorize_noSSO:(NSArray *)permissions 
{
	[self setValue:permissions forKey:@"permissions"];
 
	[self authorizeWithFBAppAuth:NO safariAuth:NO];
}
 
+ (void)toggleSingleSignOn
{
	[Facebook swizzleMethod:@selector(authorize:) withMethod:@selector(authorize_noSSO:)];
}
 
@end

You see that I have also added a class method to toggle SSO on and off. I named it “toggle” because each call will exchange the two method implementations. While the first toggle will disable SSO the second call restores the original method and thus re-enables it.

With this you only have to call the toggleSingleSignOn once per app launch, the best place is the app delegate’s +initialize which is guaranteed to be called just once and also in a thread-safe manner.

@implementation MyAppDelegate
+ (void)initialize
{
	// disable Facebook SSO
	[Facebook toggleSingleSignOn];
}
@end

That’s all it takes.

Conclusion

Of the three ways how you can modify somebody else’s code’s behavior the one that actually replaces the implementation via swizzeling is the most convenient and least dangerous, at least if you know what you are doing.

This alternative does not have the drawback of original code hacking that you incur an additional maintenance overhead. And it does not have the drawback of compiler warnings for the category hack approach.

For a project that I am currently working on I needed to implement a custom container view controller. I was feeling my way forward in the dark for the most part because this does not seem to be a widely used technique. Developers – understandably – favor reusing and skinning existing view controllers over creating new containers.

However there are some scenarios where you should prefer to make your own container because it greatly simplifies your code over trying to bend a UINavigationController or UITabBarController to your will. Do you remember the times when those two where the only two containers available?

I distinctly remember using a UINavigationController with hidden nav bar as a view controller to contain multiple full views. And you probably had do do your own view juggling and animations because for the most part the standard transitions would be useless. Fortunately we can no file this as a fond memory of the past and move on to implementing our own containers.

The first thing to wrap your head around is the notion that besides a hierarchy of views you now also have to have a consistent hierarchy of view controllers. Before iOS 5 people would often create a new view controller and then slap this VC’s view into an existing view hierarchy. Now more!

Nowadays you never would resort to this. Instead you use the mechanisms afforded by UIViewController to add and remove child view controllers.

Another notion is that we’ve trained ourselves to think of view controllers as being responsible for one entire full screen, think of the sub view controllers of a tab bar controller. But ever since UISplitViewController which arrived with the iPad this is no longer the true mantra. Viewcontrollers are supposed to manage a coherent region on the screen, that can be the entire screen for lack of space on an iPhone, but it can also be a content bar at one of the sides of the screen. Like UISplitViewController which has two sub-viewcontrollers, one for the left (“master”) panel and one for the right (“detail”) panel.

UIViewController provides two methods to add a view controller as a child and to remove it later. Those are part of the “UIContainerViewControllerProtectedMethods” category extension for UIViewController:

@interface UIViewController (UIContainerViewControllerProtectedMethods)
 
- (void)addChildViewController:(UIViewController *)childController;
- (void)removeFromParentViewController;
 
@end

These two methods do exactly what their names suggest. Though what’s not quite obvious is how you are supposed to use them. And if and how you should combine these with addSubview and removeFromSuperview. Hence this exploration. Note: All this assumes we use ARC.

According to the docs it is up to us to define the kinds of relationships we want to model. Be it only a single VC visible at the time like nav controllers, or multiple that can be reached through tabs. Or even multiple VCs that are sort of like pages.

There are three possible things that you want to be able to do with your sub-viewcontrollers that have slightly different semantics:

• Add it to your container
• Remove it from your container
• Transition to another view controller (i.e. add the new and remove the old)

For any of these you also want to be be assured that the 4 view delegate methods get properly called, as well as the new 2 delegate methods that fire before and after a VC moved to a new parent. Note that a parent of nil means that it was removed.

Why is this attention to the delegate messaging necessary? You probably use the view(Did|Will)(A|Disa)pear methods to do some last setup or teardown and so you are interested that they get properly called. And also there is an ugly warning in the console about unbalanced messages if you get something wrong here.

We’ll dive into greater detail with a sample. Let’s say we want to get an effect similar to a tabbed view controller. i.e. we have an array of view controllers and we want to switch between these. Since this container VC will be our app’s root view controller we want to show the first sub-VC when it shows.

Basic Setup

Let’s put the necessary bare bones IVARs in the implementation because we only need to have access to these inside our own ContainerViewController.

@implementation ContainerViewController
{
	NSArray *_subViewControllers;
	UIViewController *_selectedViewController;
	UIView *_containerView;
}

The subVCs will be a static array of view controllers that the developer can set. The selected VC will hold a reference to the currently showing VC and the container view will be the area where we want our sub VC’s view to be positioned. Let’s start by setting up the container view in the container’s loadView:

- (void)loadView
{
	// set up the base view
	CGRect frame = [[UIScreen mainScreen] applicationFrame];
	UIView *view = [[UIView alloc] initWithFrame:frame];
	view.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
	view.backgroundColor = [UIColor blueColor];
 
	// set up content view a bit inset
	frame = CGRectInset(view.bounds, 0, 100);
	_containerView = [[UIView alloc] initWithFrame:frame];
	_containerView.backgroundColor = [UIColor redColor];
	_containerView.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
	[view addSubview:_containerView];
 
	// from here on the container is automatically adjusting to the orientation
	self.view = view;
}

I’ve colored the main view blue and the container view red for clarity. The sub-VCs will go in the red area and be automatically resized if we rotate the device.

The integration in the app delegate is a mere formality, add the import for the header, allocate an instance and set it as root VC.

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    self.window = [[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]];
 
	ContainerViewController *container = [[ContainerViewController alloc] init];	
 	self.window.rootViewController = container;
 
    [self.window makeKeyAndVisible];
    return YES;
}

Next we need a couple of view controllers to play the role of the sub-VCs. Let’s create a simple UIViewController subclass that has a UILabel as it’s main view, so that we can display something there to tell them apart. Do avoid overcomplicating things for this example we simple take the view’s own description in there. This way we see if the display changes.

- (void)loadView
{
	// set up the base view
	CGRect frame = [[UIScreen mainScreen] applicationFrame];
	UILabel *label = [[UILabel alloc] initWithFrame:frame];
	label.numberOfLines = 0; // multiline
	label.textAlignment = UITextAlignmentCenter;
 
	// let's just have this view description
	label.text = [self description];
	self.view = label;
}

I bet you never before had a view controller which consisted only of a UILabel. For Science!

Adding

Next we need to put a couple of these PageViewControllers into an array and have a method that allows us to set this array into our container. Let’s assume that you know how to create a property for the subViewControllers. In the app delegate we have these additional lines:

// make an array of 5 PageVCs
NSMutableArray *tmpArray = [NSMutableArray array];
 
for (int i=0; i<5; i++)
{
	PageViewController *page = [[PageViewController alloc] init];
	[tmpArray addObject:page];
}
 
// set these as sub VCs
[container setSubViewControllers:tmpArray];

So much for basic setup. Now let’s override the setter for the sub VCs to select the VC at index 0 and present it. Though we cannot present it in the setter, because the view might not have been loaded yet and so our _containerView IVAR is still nil.

- (void)setSubViewControllers:(NSArray *)subViewControllers
{
	_subViewControllers = [subViewControllers copy];
 
	if (_selectedViewController)
	{
		// TODO: remove previous VC
	}
 
	_selectedViewController = [subViewControllers objectAtIndex:0];
 
	// cannot add here because the view might not have been loaded yet
}
 
@synthesize subViewControllers = _subViewControllers;

Instead we do it at the latest possible moment, that would be in the viewWillAppear because here we are guaranteed that the loadView has taken place already. A nice lazy thing, if we find that the selected VC already has self as the parent then there’s nothing to do.

- (void)viewWillAppear:(BOOL)animated
{
	[super viewWillAppear:animated];
 
	if (_selectedViewController.parentViewController == self)
	{
		// nowthing to do
		return;
	}
 
	// adjust the frame to fit in the container view
	_selectedViewController.view.frame = _containerView.bounds;
 
	// make sure that it resizes on rotation automatically
	_selectedViewController.view.autoresizingMask = _containerView.autoresizingMask;
 
	// add as child VC
	[self addChildViewController:_selectedViewController];
 
	// add it to container view, calls willMoveToParentViewController for us
	[_containerView addSubview:_selectedViewController.view];
 
	// notify it that move is done
	[_selectedViewController didMoveToParentViewController:self];
}

The above shown sequence takes care of calling the viewWillAppear, viewDidAppear, willMoveToParentViewController and didMoveToParentViewController. Notice that all but the last are done for you automatically, but for some strange reason the didMove is not. So we have to do that manually. Upon starting our Demo we now see the VC at index 0.

Next we add the capability to transition from one VC to the next.

Transitioning

To move between the child VCs we’ll add a swipe gesture recognizer to our container. If we swipe left we want to go to the VC with a lower index in the array. If we swipe right we want to go higher. No wrapping. In loadView we add:

// add gesture support
UISwipeGestureRecognizer *swipeLeft = [[UISwipeGestureRecognizer alloc] initWithTarget:self action:@selector(swipeLeft:)];
swipeLeft.direction = UISwipeGestureRecognizerDirectionLeft;
[view addGestureRecognizer:swipeLeft];
 
UISwipeGestureRecognizer *swipeRight = [[UISwipeGestureRecognizer alloc] initWithTarget:self action:@selector(swipeRight:)];
swipeRight.direction = UISwipeGestureRecognizerDirectionRight;
[view addGestureRecognizer:swipeRight];

And the implementation of swipe is as follows. For simplicity’s sake we use two separate gesture recognizers because Apple does not provide an easy way to determine the swipe direction if you combine to directions.

- (void)swipeLeft:(UISwipeGestureRecognizer *)gesture
{
	if (gesture.state == UIGestureRecognizerStateRecognized)
	{
		NSInteger index = [_subViewControllers indexOfObject:_selectedViewController];
		index = MIN(index+1, [_subViewControllers count]-1);
 
		UIViewController *newSubViewController = [_subViewControllers objectAtIndex:index];
 
		[self transitionFromViewController:_selectedViewController toViewController:newSubViewController]; 
	}
}
 
- (void)swipeRight:(UISwipeGestureRecognizer *)gesture
{
	if (gesture.state == UIGestureRecognizerStateRecognized)
	{
		NSInteger index = [_subViewControllers indexOfObject:_selectedViewController];
		index = MAX(index-1, 0);
 
		UIViewController *newSubViewController = [_subViewControllers objectAtIndex:index];
 
		[self transitionFromViewController:_selectedViewController toViewController:newSubViewController]; 
	}
}

The logic to transition from one VC to another we package in transitionFromViewController:toViewController:. This is the really interesting part. There is a convenient method that again takes care of most of the boring work of adding and removing views. And again some non-obvious additional messaging is necessary to get it perfect.

- (void)transitionFromViewController:(UIViewController *)fromViewController toViewController:(UIViewController *)toViewController
{
	if (fromViewController == toViewController)
	{
		// cannot transition to same
		return;
	}
 
	// animation setup
	toViewController.view.frame = _containerView.bounds;
	toViewController.view.autoresizingMask = _containerView.autoresizingMask;
 
	// notify
	[fromViewController willMoveToParentViewController:nil];
	[self addChildViewController:toViewController];
 
	// transition
	[self transitionFromViewController:fromViewController
					  toViewController:toViewController
							  duration:1.0
							   options:UIViewAnimationOptionTransitionCurlDown
							animations:^{
							}
							completion:^(BOOL finished) {
								[toViewController didMoveToParentViewController:self];
								[fromViewController removeFromParentViewController];
							}];
}

And with that we have our transitions in the can.

You have a number of UIViewAnimationOptionTransition-s available, but you don’t have to settle for these. You can also pass 0 for the options and instead provide all animations you like the two views to execute in the animations block.

Before I discovered is method I was using the previous method of animating the views. Though this has a side-effect that we might not like in this case. Typically you want the “will” delegate methods to fire before the transition and the “did” to follow afterwards. If you animate the views yourself then iOS 5 will take care of sending these messages for you but it does so together. This looses us the ability to differentiate between stuff we want to do before and after the appearing and disappearing of the view controller.

Conclusion

It took me quite a bit of experimenting to also get all the messaging happening and equally balanced. The above sample has that working as it should.

But once you do figure out the two techniques outlined in this article you are on the road to implement your own view controller containment like you never did anything else.

One thing that I wasn’t able to figure out so far is why the transition method always adds the new view controller’s view to the container view controller’s main view. This simplifes the process somewhat because you don’t have to know at which stage in the transition it is ok to add and remove the views. But at the same time you might have a situation where you don’t want the animation to occur over the entire area of the container view controller.

For this scenario I can only think of covering up the parts with extra sub views. Or we could stack multiple container view controllers and have one to only cover the region where where have the container view. This could then clip its subviews and thus be only care for this area.

The main advantage of any kind of view controller containment is that rotation messages (should|will|did) reach the lowest leaves of your view controller tree. Unless you disable that by means of overriding automaticallyForwardAppearanceAndRotationMethodsToChildViewControllers and returning NO.

But who would really want that when we have waited for so long for public API to have these being forwarded? Working with view controller containers I get a feeling that they greatly simplified creating complex user interfaces consisting of multiple parts.

The project for this tutorial is in my Examples GitHub repository.

Inspired by the Gmail Tap April Fools joke by Google I felt inspired to program the same thing for iOS. There we have custom input views as well as the UIKeyInput protocol and so I figured it should be an easy undertaking.

The whole affair took slightly more than one hour and I was hoping to record it in 1 second intervals with ScreenNinja. Unfortunately it seems that this otherwise fabulous app crapped out on me. I later discovered that the MOV file had actually finished before the crash, so to my delight (and hopefully yours too) you can follow this tutorial on YouTube.

In the video you see the whole process from start to finish. Let me just offer some clarifications in this post so that you better understand what I am doing.

http://www.youtube.com/watch?v=w5lRUE1Xet8

In iOS Apple provides us with two protocols, UIKeyInput for inputting single key strokes and UITextInput for everything that’s more than single keys, like text selection, auto correction etc. As part of UITextInput each conforming UIView implements an inputView property that can be any UIView.

If set this inputView is shown instead of the standard keyboard, if not then you see the keyboard. So for our example all we need is a UIView with one button for the Morse button.

The Gmail Tap prank interface uses two buttons, one for a dot, one for a dash. When researching Morse Code on Wikipedia I found one slight problem with this approach. Individual letters have to have a slight pause from each other because otherwise we wouldn’t know if by a dot the user intends to write an E or one of the other dozen or so combinations that have a dot as the first part of a sequence.

The Morse alphabet doesn’t just consist of 2 letters, but rather it uses 3 which are “dit”, “dah” and a silent pause. Given that “dit” is the shortest element, a “dah” is three times as long. Sequence elements are spaced one “dit” apart, Letters are spaced one “dah” apart and the length of 7 “dits” signifies a word break. Really, I am not making this up!

First step is to override canBecomeFirstResponder and return YES.

For simplicy I just wired up the UIControlEventTouchUpInside and the UIControlEventTouchDown to two methods and I’m using the time interval between the touch down and the touch up to determine if we saw a “dit” or “dah”. A bit of pause handling with performSelector:withObject:afterDelay: was necessary to get the breaks decoded.

Then there is a lookup table from which to draw the actual letters once a letter break is encountered.

+ (void)initialize
{
	morseLookup = [NSDictionary dictionaryWithObjectsAndKeys:@"a", @"··−·",
						@"b",@"−···",
						@"c",@"−·−·",
						@"d",@"−··",
						@"e",@"·",
						@"f",@"··−·",
						@"g",@"−−·",
						@"i",@"··",
						@"j",@"·−−−",
						@"k",@"−·−",
						@"l",@"·−··",
						@"m",@"−−",
						@"n",@"−·",
						@"o",@"−−−",
						@"p",@"·−−·",
						@"q",@"−−·−",
						@"r",@"·−·",
						@"s",@"···",
						@"t",@"−",
						@"u",@"··−",
						@"v",@"···−",
						@"w",@"·−−",
						@"x",@"−··−",
						@"y",@"−·−−",
						@"z",@"−−··",
						@"0",@"−−−−−",
						@"1",@"·−−−−",
						@"2",@"··−−−",
						@"3",@"···−−",
						@"4",@"····−",
						@"5",@"·····",
						@"6",@"−····",
						@"7",@"−−···",
						@"8",@"−−−··",
						@"9",@"−−−−·",
						nil];
 
	for (NSString *oneSequence in [morseLookup allKeys])
	{
		_longestMorseSequence = MAX(_longestMorseSequence, [oneSequence length]);
	}
}

The final piece of the puzzle is that we somehow need to have a pointer to the text control so that we can insert the detected characters. I was puzzled by the fact there does not seem to be an obvious method to do that. So I went for the obvious method of setting it in the init.

The input delegate (e.g. a UITextField) has to be compliant with the UIKeyInput protocol and therefore we know that it must have an insertText: method. This will insert the passed string at the current cursor position.

Conclusion

Turns out the by far hardest part of making our own custom keyboard is the input logic, which itself becomes way more complicated by having to invent algorithms to decode Morse sequences.

There are several improvements that spring to mind right away. Auto-Completion would be one where it could show a decreasing number of possible sequences based on the sequence for a single letter so far. Maybe that could be shown if the finger is lifted outside of the button.

Another question to tackle is how to actually delete text, the backspace key if you will. Maybe by doing a “dit” that begins on the button, but ends outside to the left of it?

Finally there should be no set amount of time for a “dit”, experienced Morse-o-graphers can have much shorter dits as long as the relation of dits, dats and pauses remains mostly constant. In this tutorial I went for the easiest method of simply using the latest dit’s duration, but it would be nice to average this over time.

Source code of the full project is available on GitHub.

The file systems of iOS and Mac both use HFS+ as file system, with only one small difference. iOS uses case-sensitive file names, Mac doesn’t by default. Both have a feature called “Extended File Attributes” that allow you to set custom values by key.

Apple generally ignored this functionality and it was only briefly – in 5.0.1 – that they actually used an extended attributed for something. The “com.apple.MobileBackup” extended attribute served as a stop-gap-measure to mark files that should not be backed up. Though this was very short lived.

I was facing the problem myself that I needed to save the ETag for an image that I downloaded from the web somewhere. And I wanted to do that elegantly, somehow together with the file itself and in a way that would simply work.

Well, the best way for achieving this simple goal was to save the ETag value in the extended file attributes. This way you don’t have to keep track of a separate file or storage for the value. If the file is deleted, then that takes care of the extended attributes as well.

Extended attributes are managed with C-API defined in sys/xattr.h, so most of the work in translating to and from Objective-C is converting strings.

To set an attribute you do this:

const char *attrName = [attribute UTF8String];
const char *filePath = [_path fileSystemRepresentation];
 
const char *val = [value UTF8String];
 
int result = setxattr(filePath, attrName, val, strlen(val), 0, 0);

Note that result has to be a signed integer because it returns -1 in case of error.

The above assumes that you want to set a string as value which is the most versatile. But in fact you can set any kind of byte data because you specify a pointer to the first byte and then number of bytes to write. So you can also set an integer like this:

int val=1;
int result = setxattr(filePath, attrName, &val, sizeof(val), 0, 0);

In my special case I was content with setting an NSString. I didn’t see any way how you could know from the data from what bytes it was actually set. I guess you have to make your own rules and only access/decode your own extended attributes.

To retrieve the extended attribute value you do this:

const char *attrName = [attribute UTF8String];
const char *filePath = [_path fileSystemRepresentation];
 
// get size of needed buffer
int bufferLength = getxattr(filePath, attrName, NULL, 0, 0, 0);
 
// make a buffer of sufficient length
char *buffer = malloc(bufferLength);
 
// now actually get the attribute string
getxattr(filePath, attrName, buffer, 255, 0, 0);
 
// convert to NSString
NSString *retString = [[NSString alloc] initWithBytes:buffer length:bufferLength encoding:NSUTF8StringEncoding];
 
// release buffer
free(buffer);

Here you see getxattr actually being called twice. If you don’t set a pointer to a buffer this retrieves the size of the value in bytes for the attribute. So you can malloc a sufficiently large buffer and then get this filled to the brim on the second call.

All commands in xattr.h have two variants, one with a file path and a second with with a file descriptor. The latter would be used in cases where you want to keep the file descriptor (sort of a handle to the file) around doing more work. But we can ignore these.

There are also two more functions, for sake of completeness, one to remove an extended attribute, one to get a list of all keys. I contemplated for a while to make an API similar to NSFileManager, but since we don’t know the data type (unless we save that somewhere) there is little use to build a method that sets or retrieves and entire dictionary of keys and values. Also, for now, I only need to save and retrieve a single string.

I created a documented wrapper for handling extended file attributes in DTFoundation and I named it DTExtendedFileAttributes.

On my iCatalog project I have more than 40 targets (FOURTY!), all with individual info.plist. Let me share a quick command line script that lets me simultaneously bump the software version to a different value for all targets at the same time. This has proven extremely helpful.

The script I use uses the defaults tool to write the new value into the plists and then converts it back to text xml plist format.

bump_version_to.sh

#!/bin/sh
 
if [ "$#" -ne 1 ]
then
        echo "Usage: bump_version_to <version>"
        exit 1
fi
 
for FILE in `ls */*Info.plist`
do
        INFO_PLIST=`pwd`/${FILE%.*}
        defaults write "$INFO_PLIST" CFBundleVersion "$1"
        plutil -convert xml1 "$INFO_PLIST.plist"
done

Put this script into your project root. It expects the info.plist files to be one level below it. Make it executable with chmod +x. Run it passing the new version string. Boom!

I was getting reports about DTCoreText having leaks. Not something I like to wake up to, to be honest. So I dived right in with Instruments and the Leaks tool. I am going to share with you something that I learned about Leaks and Blocks that might save you much trouble if you check for that first the next time you profile any app.

I fired up Instruments with the DTCoreText demo app running in simulator and I was dumbfounded. Lots of small red bars told of unchecked leaks. But what was even more unnerving was the sheer amount of different objects that got leaked. No way that all these different “Responsible Frames” where all leaking.

I did what every engineer does at first: find somebody else to blame. So I pointed the finger at Apple, because obviously NSScanner, NSDictionary and NSString are “leaking like crazy”. What other reason could there be for NSDictionary leaking 64 Bytes on every mutableCopy?

In my desperation I wasted about half a day trying to rewrite methods to leak less, like replace NSDictionary’s mutableCopy with initWithDictionary. But the only thing this achieved was to move the leaks elsewhere. Like a sinking boat I found myself unable to plug any holes because for each plugged one a new leak would form.

Rule number 2 for engineers is: if you cannot achieve perfection, then at least rationalize it as “good enough”. Surely it cannot be a problem leaking 192 Bytes every now and then. We all shipped apps leaking more than that, right?

So, it’s all Apple’s fault and my code is good enough….

The Retain Loop Loupe

But then – while aimlessly clicking around the Instruments interface – I discovered the “Cycles & Roots” tool. Bam!

Next to an amazing Picasso I saw some Complex and some Simple Cycles. But the flow chart made a light go on in my head. Obviously the root cause was DTHTMLAttributedStringBuilder having a dictionary that held onto an NSMallocBlock that referred back to the builder class, a classical retain loop. They should name this graphic “The Retain Loupe” because it magnified the problem sufficiently so that even us dumb developers would get it.

To understand what the above means let me briefly explain an architecture choice I made in the attributed string builder class. In the original code I had an enormous if-else if-else if-etc. tree checking for all different kinds of HTML tag names. I changed all this by having two dictionaries of handlers for starting and ending of tags. Each dictionary contains the blocks to be executed when the opening or closing of certain tags are encountered.

@implementation DTHTMLAttributedStringBuilder
{
	// parsing state, accessed from inside blocks
	DTHTMLElement *currentTag;
 
	// lookup table for blocks that deal with begin and end tags
	NSMutableDictionary *_tagStartHandlers;
	NSMutableDictionary *_tagEndHandlers;
}
 
- (void)_registerTagStartHandlers
{
	void (^blockquoteBlock)(void) = ^
	{
		currentTag.paragraphStyle.headIndent += 25.0 * textScale;
		currentTag.paragraphStyle.firstLineHeadIndent = currentTag.paragraphStyle.headIndent;
		currentTag.paragraphStyle.paragraphSpacing = defaultFontDescriptor.pointSize;
	};
 
	[_tagStartHandlers setObject:[blockquoteBlock copy] forKey:@"blockquote"];
 
	// ...
}

So in the parser delegate method for opening of tags I can see if there is a block registered for a given tag and execute it if it is.

- (void)parser:(DTHTMLParser *)parser didStartElement:(NSString *)elementName attributes:(NSDictionary *)attributeDict
{
	// find block to execute for this tag if any
	void (^tagBlock)(void) = [_tagStartHandlers objectForKey:elementName];
 
	if (tagBlock)
	{
		tagBlock();
	}
}

Isn’t it beautiful?

When the blocks are created the compiler checks which variables are used. If you use a local variable then it is copied. Any objects used thusly are also retained during the lifetime of the block. If you use an instance variable (currentTag) then this also means that self will be captured and retained. The reason for this is that the runtime needs to make sure that all needed objects stay around while the block lives.

Michael Mangold points us to Lecture 10 of Standford’s iOS Development course, which explains the block basics as well.

Simple Fix

The problem with this is that each of these blocks that accesses an ivar retains self, but the string builder class also retains the dictionaries which in turn retain the blocks. Vicious loop, fortunately with a simple fix.

The simple fix for this is to make sure that the two dictionaries are released when no longer needed. In ARC you force a release by nilling the only reference to them.

- (BOOL)buildString
{
	// register default handlers
	[self _registerTagStartHandlers];
	[self _registerTagEndHandlers];
 
	// string building
 
	// clean up handlers because they retained self
	_tagStartHandlers = nil;
	_tagEndHandlers = nil;
 
	return result;
}

That’s all it takes, boom! Gone are all leaks.

So these “Leaks” where no real leaks. They where just the interim objects that made up the NSAttributedString that the string builder produced while building the string from HTML.

Weak Self

The more complicated option revolves around having a way to access IVARs and properties without retaining self. Since the blocks are retained by the dictionaries which are retained by the string builder there is no scenario where the block would still live without the string builder still around. So it is perfectly safe to convert self into a weak reference.

__weak DTHTMLAttributedStringBuilder *weakself = self;
 
void (^someBlock)(void) = ^
{
	weakself.someProperty = @"bla";
	[weakself callMethod:@"bar"];
};

Using this approach you can access all properties and methods of self without causing self to be retained. The final piece of the puzzle is how you can access IVARs for which you didn’t create a property. Maybe you don’t want to have the unnecessary Objective-C messaging overhead for just setting a value?

For accessing IVARs Objective-C borrows the arrow operator from C++. This operator is usually used in C++ if you have a pointer to a struct or C++ class and want to access one of the member variables. In Objective-C any IVAR really is just a variable inside a structure too, one that self points to.

__weak DTHTMLAttributedStringBuilder *weakself = self;
 
void (^someBlock)(void) = ^
{
	weakself->ivar = @"bla";
};
 
// for example:
void (^blockquoteBlock)(void) = ^
{
	weakself->currentTag.paragraphStyle.firstLineHeadIndent = weakself->currentTag.paragraphStyle.headIndent;
	weakself->currentTag.paragraphStyle.paragraphSpacing = weakself->defaultFontDescriptor.pointSize;
};

This is your option number 2, prefixing every property with “weakself.” and every IVAR with “weakself->”.

Ugh, so much extra code! I wish there was some option in ARC to tell the compiler not to retain self to be able to avoid all this extra code.

Conclusion

When debugging/profiling an application that makes use of blocks be sure to check the “Cycles&Roots” view first because those the the easiest to fix. Don’t waste your time fiddling around with the “obvious leaks” and also refrain from blaming Apple.

Of the two methods to avoid a self-retain-cycle I personally prefer the one using less code, i.e. explicitly releasing a container holding onto blocks. But in some situations this might not be an option, so you have to resort to the weakself strategy.

Steve Flack recommends reading Mike Ash’s piece on retain cycles and how these behave with ARC.

I was looking for a safe way to initialize a property on individual instances of an object. “Safe” as in “thread-safe”, because nowadays you never know… with GCD it could just happen that an object is used from multiple threads concurrently.

I felt the need to chronicle the findings here one the different approaches I tried.

In DTCoreText I have a class that represents one line of text. I wanted to protect the method to get the text metrics for this line.

@synchronized

The classical approach is to use the @synchronized keyword and provide any object to synchronize against. Usually you would use self for this purpose, but you don’t have to.

- (void)calculateMetrics
{
	@synchronized(self)
	{
		if (!_didCalculateMetrics)
		{
			// do calc
 
			_didCalculateMetrics = YES;
		}
	}
}

These days if you show such code to somebody with at least a passing knowledge of Grand Central Dispatch we will frown and turn his nose on @synchronize. “Man, synchronize is so much slower than GCD.”

Fiery Robot did a bit of benchmarking which shows that indeed GCD is marginally faster than synchronize, probably because of the more light-weight locking that is available through GCD. This is why David Hoerl changed it to using GCD locking when he updated the project to ARC.

GCD Semaphores

This adds quite a bit more code as you have to create the locking token (aka semaphore), keep it in an IVAR and release it at the end of the object’s lifetime.

@interface DTCoreTextLayoutLine ()
@property (nonatomic, assign) dispatch_semaphore_t layoutLock;
@end
 
@implementation DTCoreTextLayoutLine
 
- (id)init...
{
	if ((self = [super init]))
	{
		layoutLock = dispatch_semaphore_create(1);
	}
	return self;
}
 
- (void)dealloc
{
	// clean up semaphore
	dispatch_release(layoutLock);
}
 
- (void)calculateMetrics
{
	// wait for lock if active
	dispatch_semaphore_wait(layoutLock, DISPATCH_TIME_FOREVER);
 
	if (!_didCalculateMetrics)
	{
		// do calc
 
		_didCalculateMetrics = YES;
	}
 
	// release lock
	dispatch_semaphore_signal(layoutLock);
}

This achieves the same degree of safety as the synchronize version, but using GCD and – marginally – faster.

dispatch_once … NOT

Though there was something about this that bugged me. Previously we learned that if you want something to occur only exactly once you would use dispatch_once.

- (void)calculateMetrics
{
	static dispatch_once_t _onceToken;
	dispatch_once(&_onceToken, ^{
		// do calc
 
		_didCalculateMetrics = YES;
	});
}

That looks quaint, but it does not work for the simple reason that static variables are essentially global. Creating the dispatch once token inside the method only creates on token for all instances of this class. So it would calculate the text metrics only for the very first layout line.

So my next attempt was to make the once token a private instance variable. That’s exactly the same as the above example, but instead of static you move the token variable into the curly braces after the implementation.

I got even cockier than that. I still had a BOOL to keep track of wether the calc had already been done, so that I don’t get an overhead of many unnecessary Objective-C function calls. Since the dispatch once token is essentially an integer itself you can also use it in an if. It will be 0 when created and -1 once it was used once.

@implementation DTCoreTextLayoutLine
{
	dispatch_once_t _didCalculateMetrics;
}
 
- (void)calculateMetrics
{
	dispatch_once(&_didCalculateMetrics, ^{
		// do calc
	});
}
 
- (CGFloat)ascent
{
	if (!_didCalculateMetrics)
	{
		[self calculateMetrics];
	}
 
	return ascent;
}

This approach looked to me the most elegant involving the least amount of code. So I patted myself on my back and pushed these changes to GitHub for everybody to marvel at my ingenuity. Even some true experts called it “unconventional”.

BUT …

Unfortunately it’s not as good as it looks. Somebody had the audacity to read the f’in manual (RTFM) and point out this sentence to me:

The predicate must point to a variable stored in global or static scope. The result of using a predicate with automatic or dynamic storage is undefined.

OH NO! This sentence basically means that you cannot safely use a dispatch once token that is an instance variable or even a regular local variable. Instead it has to be either global outside of your implementation or with the static keyword.

The reason for this restriction must be in the way how GCD can access global or static storage in a thread-safe way that it cannot for other types of variables. Maybe something to do with memory locking, that only works properly for these cases.

The “elegant” method does indeed work, but falls apart in concurrency.

There’s yet another possibility for synchronizing, dispatch_sync.

dispatch_sync

This is similar to locking semaphores because internally dispatch_sync uses these for synchronizing. Though there is a bit of an advantage of using a sync dispatching over locking. You might end up with a dead lock if a lock-protected section tries to wait for the lock as well.

 
@implementation DTCoreTextLayoutLine
{
	BOOL _didCalculateMetrics;
	dispatch_queue_t _syncQueue;
}
 
- (id)init...
{
	if ((self = [super init]))
	{
		// get a global queue
		_syncQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
	}
	return self;
}
 
- (void)calculateMetrics
{
	dispatch_sync(_syncQueue, ^{
		if (!_didCalculateMetrics)
		{
			// do calc
 
			_didCalculateMetrics = YES;
		}
	});
}

The previous example uses a global queue (available in low, default and high priority) and as such does not need to worry about retaining or releasing it, as these live as long as your app does.

Again the RTFM contains a suggestion for us:

Although dispatch queues are reference-counted objects, you do not need to retain and release the global concurrent queues. Because they are global to your application, retain and release calls for these queues are ignored. Therefore, you do not need to store references to these queues. You can just call the dispatch_get_global_queue function whenever you need a reference to one of them.

But personally I’d rather set up the reference to the queue I want to use for synching in an IVAR because when I can be sure to have the exact same queue available in different methods of the same object. If you are doing some very tight and extensive looping then the overhead of unnecessary function calls can pile up.

Conclusion

It is unfortunate that dispatch_once does not work in the context of lazy property initialization. For this scenario we have 3 options at our disposal. The original synchronize, locking with semaphores and sync dispatching.

One use was intentionally omitted from this: dispatch_async. The reason being that the above mentioned benchmark showed it as substantially slower than the other variants because of the system having to copy the blocks to the heap. Sync dispatching can keep the block on the stack and execute it right away.

While doing some performance tuning on the iCatalog.framework I stumbled upon a method of about 7 statements where a single line was responsible for more than a third of all CPU time.

This basically was only getting the path to the app’s Library/Caches folder. By itself this statement looks very innocent and I had it in about a dozen places all around the app. But it turns out that if you calling it hundreds or thousands of times then the time it takes to search for the Caches (and Documents) path sums up enormously.

Interestingly it does not seem like Apple implemented any caching for them so they seem to use around the same time all the time. But these values are prime candidates for caching because they won’t change while your app is running. Also the objc function call to get a cached version of the paths is several orders of magnitude faster than determining it in the first place.

This is what I saw in Instruments:

Granted, in most apps you would probably never notice a difference, but if you work with the documents or caches folders frequently in loops and/or many places then this is for you.

I added two category methods to my DTFoundation project to add caching.

@implementation NSString (DTPaths)
 
+ (NSString *)cachesPath
{
	static dispatch_once_t onceToken;
	static NSString *cachedPath;
 
	dispatch_once(&onceToken, ^{
		cachedPath = [NSSearchPathForDirectoriesInDomains(NSCachesDirectory, NSUserDomainMask, YES) lastObject];	
	});
 
	return cachedPath;
}
 
+ (NSString *)documentsPath
{
	static dispatch_once_t onceToken;
	static NSString *cachedPath;
 
	dispatch_once(&onceToken, ^{
		cachedPath = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) lastObject];	
	});
 
	return cachedPath;
}
@end

I introduced this pattern using a “onceToken” previously and it proves to be a boon. So you should learn it by heart. You create a static dispatch once token as well as a static pointer to something that you want to cache. Then you wrap the creation of the item in a dispatch_once. Et voila!

Before GCD we would have done this sort of caching with a if (!cachedPath) but the GCD-way has the added benefit of being thread-safe.

As a general rule of thumb you should cache everything that is expensive to get (calculation or via network) and does not change much. If it is just a single pointer, like an NSString * the above approach works marvelously. If you want to see how this can be combined with the use of NSCache for multiple values, check out …

OMG, I was about to write “checkout the DTFontDescriptor class in DTCoreText” as a shining example of how to initialize a font cache. But here’s what I found:

+ (NSCache *)fontCache
{
	if (!_fontCache)
	{
		_fontCache = [[NSCache alloc] init];
	}
 
	return _fontCache;
}

This is exactly the non-GCD-way that I alluded to above. This hasn’t been a problem so far because all calls to it would have come from the same thread, but while we are at its a great exercise to show that we have properly inhaled the knowledge imparted upon us above.

+ (NSCache *)fontCache
{
	static dispatch_once_t onceToken;
 
	dispatch_once(&onceToken, ^{
		_fontCache = [[NSCache alloc] init];
	});
 
	return _fontCache;
}

Ah, much better. Works just like before, but now thread-safe.

There is one riddle that I stumbled up I was not able to explain. In the same file DTFontDescriptor I also tried to adjust the second lookup table fontOverrides in the same fashion, but when trying to run the Demo it stalls somehow deadlocked. Could it be that the current way of using semaphores to synchronizing interferes with the dispatch_once?

Comment below if you know how to fix that…

As a hobby project I am working on uncovering hidden treasures that exist on all your iOS devices. Hidden, because there is no Objective-C API for them, Existing, because Apple includes a great deal of open source libraries in iOS, compiled as a dynamic library.

What items exist you can see if you check out what dylibs are there to be seen in “Link Binary with Libraries”. Most entries beginning with lib and ending with dylib can be used. Some people have reported getting rejected for adding the static variants of libraries like libxslt or libarchive, but that’s probably because Apple sees these symbols as duplicate to the ones contained in the dynamic libraries.

We previously looked at libxml2 for parsing HTML (and part 2), today we’ll familiarize ourselves with zlib for decompressing .gz and .zip files.

The first time I came in contact with decompressing files was on my MyAppSales open source project. There I was scraping iTunes Connect and the downloaded files were compressed in ZIP format. The new unofficial iTunes Connect API compresses the daily and weekly reports in GZIP format, which no longer works. This is why I set out to find a solution that works for both.

There are several compression schemes out there, but the most prevalent two are PKZIP (as popularized by WinZIP) and GZIP (as in GNU Zip). The former is a wrapper around the latter. GZIP only supports a single file, whereas PKZIP adds a special file header that has an index of the included files with pointers to the locations of the corresponding GZIP chunks.

In Memory Decompression: GZIP

zlib by itself can only deal with zlib-compressed streams (“deflated”), GZIP in turn adds a minimal header for this deflated content. Decompressing streams with pure zlib-compression or zlib+GZIP header is relatively straightforward. Here’s the method I gleaned (and cleaned up) from CocoaDev’s NSData category.

NSUInteger dataLength = [_data length];
NSUInteger halfLength = dataLength / 2;
 
NSMutableData *decompressed = [NSMutableData dataWithLength: dataLength + halfLength];
BOOL done = NO;
int status;
 
z_stream strm;
strm.next_in = (Bytef *)[_data bytes];
strm.avail_in = (uInt)dataLength;
strm.total_out = 0;
strm.zalloc = Z_NULL;
strm.zfree = Z_NULL;
 
// inflateInit2 knows how to deal with gzip format
if (inflateInit2(&strm, (15+32)) != Z_OK)
{
	return;
}
 
while (!done)
{
	// extend decompressed if too short
	if (strm.total_out >= [decompressed length])
	{
		[decompressed increaseLengthBy: halfLength];
	}
 
	strm.next_out = [decompressed mutableBytes] + strm.total_out;
	strm.avail_out = (uInt)[decompressed length] - (uInt)strm.total_out;
 
	// Inflate another chunk.
	status = inflate (&strm, Z_SYNC_FLUSH);
 
	if (status == Z_STREAM_END)
	{
		done = YES;
	}
	else if (status != Z_OK)
	{
		break;	
	}
}
 
if (inflateEnd (&strm) != Z_OK || !done)
{
	return;
}
 
// set actual length
[decompressed setLength:strm.total_out];

This method works by setting up a zstream struct with a pointer and length to the data bytes of the file. Then it initializes the decompressor with inflateInit2. The trailing 2 is important in this function as this is the version that knows how to deal with the GZIP header. The decompression occurs by calling inflate until this returns Z_STREAM_END. Finally the decompressor is freed up by calling inflateEnd.

I liked this approach because it keeps adding half of the compressed data size to the output mutable data object. This is way more efficient than constantly adding each decompressed chunk to the data forcing it to constantly reallocate larger bits of memory and copying the contents. At the end it uses setLength to specify the actual length of data.

In Memory Decompression: PKZIP

You can tell a GZIP and PKZIP file apart by inspecting the first two bytes of it. If these are ‘PK’ then you have a ZIP file.

PKZIP adds a special header so that multiple GZIPped files can peacefully coexist in a single .ZIP file. Dealing with this header is quite tedious so people are happy to use Minizip. This is another C-library that wraps this complexity up. If you take it as it is there are some compiler warnings, so – lazy me – I used the cleaned up version by Sam Soffes.

The Objective-C versions of decompressing PKZIP seem all to be more or less based on ZipArchive project by “Aish”. You can tell that this is the case because they generally contain the same bug dealing with the file date of the zipped files. If you find a reference to Jan 1st 1980 in there, you know what I mean.

The – simplified – structure of dealing with a PKZIP in memory is this:

unsigned char buffer[BUFFER_SIZE] = {0};
 
// open the file for unzipping
unzFile _unzFile = unzOpen((const char *)[_path UTF8String]);
 
// return if failed
if (!_unzFile)
{
	return;
}
 
// get file info
unz_global_info  globalInfo = {0};
 
if (!unzGetGlobalInfo(_unzFile, &globalInfo )==UNZ_OK )
{
	// there's a problem
	return;
}
 
if (unzGoToFirstFile(_unzFile)!=UNZ_OK)
{
	// unable to go to first file
	return;
}
 
// enum block can stop loop
BOOL shouldStop = NO;
 
// iterate through all files
do 
{
	unz_file_info zipInfo ={0};
 
	if (unzOpenCurrentFile(_unzFile) != UNZ_OK)
	{
		// error uncompressing this file
		return;
	}
 
	// first call for file info so that we know length of file name
	if (unzGetCurrentFileInfo(_unzFile, &zipInfo, NULL, 0, NULL, 0, NULL, 0) != UNZ_OK)
	{
		// cannot get file info
		unzCloseCurrentFile(_unzFile);
		return;
	}
 
	// reserve space for file name
	char *fileNameC = (char *)malloc(zipInfo.size_filename+1);
 
	// second call to get actual file name	
	unzGetCurrentFileInfo(_unzFile, &zipInfo, fileNameC, zipInfo.size_filename + 1, NULL, 0, NULL, 0);
	fileNameC[zipInfo.size_filename] = '\0';
	NSString *fileName = [NSString stringWithUTF8String:fileNameC];
	free(fileNameC);
 
	NSMutableData *tmpData = [[NSMutableData alloc] init];
 
	int readBytes;
	while((readBytes = unzReadCurrentFile(_unzFile, buffer, BUFFER_SIZE)) > 0)
	{
		[tmpData appendBytes:buffer length:readBytes];
	}
 
	// decompressed file now in tmpData, name in fileName
 
	// close the current file
	unzCloseCurrentFile(_unzFile);
}
while (!shouldStop && unzGoToNextFile(_unzFile )==UNZ_OK);

Now about these file dates … For some unfortunate historical reasons Microsoft didn’t think to include time zone support in DOS file dates. We will never now why. There are two ways how a file date can be represented in PKZIPped files. The zipInfo header struct both contains a dosdate value as well as a tmu_date struct. The spec states that if the dosdate is 0 then the tmu_date is to be used which has individual fields for hour, min, sec, day, month and year. But no time zone either.

Now the bug I alluded to above is to assume that – like on Unix – the dosdate is a time stamp, a number of seconds since a reference date. The implementations that get this wrong just assume that the dosdate is a number of seconds since beginning of 1980. The problem is that there might still be some files out there that ONLY use the dosdate, so we cannot just ignore that and go with the tmu_date.

I found the spec for the dos date hidden deep in Microsoft’s web. It simply compresses all the date parts into 2 bytes, using only as many bits as necessary for each value. And to save one bit on the seconds these are divided by 2.

long l = 1078768689; // a dosdate
 
int year = ((l>>25)&127) + 1980;  // 7 bits
int month = (l>>21)&15;  // 4 bits
int day = (l>>16)&31; // 5 bits
int hour = (l>>11)&31; // 5 bits
int minute = (l>>5)&63;	// 6 bits	
int second = (l&31) * 2;  // 5 bits

Crazy, isn’t it? I can see how somebody might have assumed that this is a number of seconds since 1980 as the year value is at the highest order bits.

If the dosdate is 0 then you can trust the tmu_date to contain the local time where the ZIP file was created. But I’ll leave this exercise to you. Just one hint: January is month 0.

We usually don’t care about getting the exact file stamps correct. If you do you need to either save the time zone inside the ZIP file as well, possibly as a plain text file. Or alternatively if you know that the files came from a certain server you can assume this time zone.

PKZIP also supports encrypting files with a password. If you never deal with thus encrypted files you can disable crypt support by defining NOCRYPT and NOUNCRYPT. This omits the crypt code from the compiled binary. The header minizip/crypt.h claims:

The encryption/decryption parts of this source code (as opposed to the non-echoing password parts) were originally written in Europe. The whole source package can be freely distributed, including from the USA. (Prior to January 2000, re-export from the US was a violation of US law.)

But then again, the original encryption is relatively weak, several cracking tools exist which can brute force it. There are two stronger encryption schemes introduced by WinZip and PKWare PKZip which are not even supported by minizip. So you are probably safer if you just omit the encrypting parts if you don’t want to jump hoops presented by Apple or the US Government or face the grief of not-supported encryption schemes.

Conclusion

The source code featured in this article is available as part of DTFoundation, check out the DTZipArchive class there.

There isn’t really much there once you gotten used calling C functions. And unfortunately documentation or tutorials on the subject matter are pretty hard to come by. But thankfully – almost always – somebody has blazed the trail and provided something that we can use to pattern our approach after.

Especially in Unix circles there is a third decompression scheme that I neglected to mention: tar.gz. This works around the single-file limitation of GZIP by concatenating the files first and then deflating them. The methods to deal with these files are available in libarchive.dylib, available in the dylibs I mentioned above. I am looking for somebody who can confirm that this is indeed app-store-legal before I add support for tar.gz to DTZipArchive.

I would also be interested to hear from somebody using minizip in app store apps and whether the crypt code was omitted and/or the encryption exporting process was required.

I got a strange bug report last week for iCatalog. Deleting of outdated catalogs takes too long, if we couldn’t show a HUD with a spinner while the deletion occurs. That was definitely one of these HUUUUU?! moments. I always thought that file deletion is instant on Unix since only an entry in a file table needs to be removed.

I grabbed an iPad 1 and deleted a 160 MB catalog. Only to find that the whole deletion – a simple NSFileManager removeItemAtPath – took 50 seconds. Uhm, no that is far from ideal to be blocking the main thread and interface for that long.

I played around a bit and over the course of the day, with some great help from several GCD experts on twitter, I pieced together a solution that might interest you if you ever have to delete large amounts of files in an instant. Before Cocoa, on Carbon, OSX offered a method called FSPathMoveObjectToTrashAsync, this is sort of the equivalent for iOS.

I didn’t debug or instrument too much into NSFileManager and why it performs so badly in this case. I suspect that this might be because NSFM supports setting a delegate, whose delegate methods get called multiple times during file operations. Possibly this isn’t optimized such that the check which methods are implemented (all are optional) is done when setting the delegate. If NSFM does tons of respondsToSelector: then that would explain the lag.

The goal in this was was to get rid of a folder (and its contents) as fast as possible and to have any longer operations go on in the background. So my first instinct was to create a category on NSFileManager.

Go Undercover

Removing the above mentioned 160 MB folder would take 50 seconds (on iPad 1), but it would only require 8 ms to rename it. So the strategy became: 1) move it away into a temp location 2) actually delete it.

// move it to a tmp name to that it appears gone
CFUUIDRef newUniqueId = CFUUIDCreate(kCFAllocatorDefault);
CFStringRef newUniqueIdString = CFUUIDCreateString(kCFAllocatorDefault, newUniqueId);
NSString *tmpPath = [NSTemporaryDirectory() stringByAppendingPathComponent:(__bridge NSString *)newUniqueIdString];
CFRelease(newUniqueId);
CFRelease(newUniqueIdString);
 
// make a file manager just for this thread/queue
NSFileManager *fileManager = [[NSFileManager alloc] init];
 
if (![fileManager moveItemAtPath:path toPath:tmpPath error:NULL])
{
	// looks like the file is no longer there
	return;
}
 
[fileManager removeItemAtPath:tmpPath error:NULL];

Now you see that I am creating a new fileManager. In the initial category I would simply use self here. But it was pointed out to me that it is unsafe to use the same NSFM instance from multiple threads. Also we don’t know “where it has been” – i.e. somebody might have set the delegate – so we make our own.

A Touch of GCD

The trusty performSelectorOnBackgroundThread: is so 2010, so we’ll make full use of the facilities provided by Grand Central Dispatch (GCD). Trust me, it’s easier than it sounds. Don’t let yourself be intimidated by the C and Blocks. We covered blocks before, today we shall dispatch them asynchronously.

We will employ 3 GCD techniques in tandem:

1. dispatch_async
2. GCD groups
3. dispatch_once

In true GCD terminology everything happens on so-called Queues. As far as I can tell you can use Queue and Thread interchangeably. There are ways to get the main queue or background queues with certain priorities. But the one attribute of queues that we shall make use of is that they process one block of code at a time.

This means you can feed multiple operations (packed in blocks) onto a background queue and be certain that they will be worked off sequentially. Sounds like an NSOperationQueue, doesn’t it? Well, NSOQ was reimplemented on GCD as soon that entered the language because it is way more efficient than regular threading.

Now you can create anonymous queues, or you can group queues under a queue name. The latter is advantageous if you want to be able to wait for the queue to finish its work.

The DTAsyncFileDeleter class that we are creating here is supposed to work off the items we give it to remove in sequence so that we can be certain that we don’t have two instances try to remove the same file at once. I initially thought about a simple @synchronize but that would have blocked if you wanted to remove two items in rapid succession. As a matter of fact I found that just mentioning the possibility of @synchronize in informed circles will get you many frowns and invariably several people will step forward to lecture you on how much more efficient GCD is.

Let’s summarize the strategy: All instances of DTAsyncFileDeleter should use the same background queue. This queue should only be created once globally for all instances. And we need to somehow know if all operations are done. Oh, and as a bonus it would be nice if the deletion would continue even after the app is suspended …

“use the same background queue” and “create once globally” are the terms that should trigger two ideas: static global variable and dispatch_once.

dispatch_once is a way in GCD to have something occur exactly once. Before GCD we would possibly have a static variable, check that against nil and only instantiate it if it is. The problem with this approach is that there would potentially be racing conditions where two threads would call the sharedInstance method at the same time and both would create the shared instance, but one would leak.

With GCD you define a static token and you are guaranteed that the dispatch_once block using this token is only going to be executed once. Really.

So in code, we have the static global variables at the top of our class, before the @implementation:

static dispatch_queue_t _delQueue;
static dispatch_group_t _delGroup;
static dispatch_once_t onceToken;

And in the init we dispatch_once the creation of a queue and a group.

dispatch_once(&onceToken, ^{
	_delQueue = dispatch_queue_create("DTAsyncFileDeleterQueue", 0);
	_delGroup = dispatch_group_create();
});

In this instance we don’t care about releasing these resources because these items should stay available until the app is terminated. GCD objects are very lightweight anyway, so not to worry.

And with the GCD trimmings set up we can wrap the above rename and delete code in a GCD block:

- (void)removeItemAtPath:(NSString *)path
{
	dispatch_group_async(_delGroup, _delQueue, ^{
		// rename and delete
	});
}

Of course if you don’t need to wait on the queue then you can dispense with the group thing, a regular dispatch_async without group will work just the same. But we want to have a way for the outside world to wait, so we implement a method for that.

- (void)waitUntilFinished
{
	dispatch_group_wait(_delGroup, DISPATCH_TIME_FOREVER);
}

If the queue in our _delGroup is done, this function returns immediately. Otherwise it will wait forever. It’s almost too easy.

At this point our class is fully functional, but for convenience we also want to have a shared instance. So using what we now know about a token for dispatch_once we can construct our +sharedInstance method thusly:

static DTAsyncFileDeleter *_sharedInstance;
 
@implementation DTAsyncFileDeleter
 
+ (DTAsyncFileDeleter *)sharedInstance
{
	static dispatch_once_t instanceOnceToken;
	dispatch_once(&instanceOnceToken, ^{
		_sharedInstance = [[DTAsyncFileDeleter alloc] init];
	});
 
	return _sharedInstance;
}

Oh how convenient that is, anywhere we import the header we can do

[[DTAsyncFileDeleter sharedInstance] removeItemAtPath:path]:

It’s fast, it’s convenient, it’s safe. It’s totally Apple.

Bonus: Automatic Background Task Completion

If we left here then the deletion process would get suspended together with the app and possibly aborted if the app is killed. Of course it would resume normally the next time the app is brought to the foreground but such a deletion might take in the vicinity of one minute on a older iOS device, so that’s an ideal use case for iOS multitasking, aka background task completion.

A UIApplication is not suspended but left alive if it has one or more registered background tasks running. For this purpose you need to get a background task id from the shared UIApplication instance of your app. You need to set a completion handler that gets called if the task completion runs into the 10 min timeout. And you need to invalidate the background task in both cases, when timed out and when complete.

We just need to register a method for getting informed for the UIApplicationDidEnterBackgroundNotification and do that all. Of course we also check if multi tasking is available, just to be sure. (It is probably superfluous because it wouldn’t get the notification if it weren’t, right?)

#pragma mark Notifications
- (void)applicationDidEnterBackground:(NSNotification *)notification
{
	UIDevice *device = [UIDevice currentDevice];
 
	if ([device respondsToSelector:@selector(isMultitaskingSupported)])
	{
		if (!device.multitaskingSupported)
		{
			return;
		}
	}
 
	UIApplication *app = [UIApplication sharedApplication];
	__block UIBackgroundTaskIdentifier backgroundTaskID;
 
	void (^completionBlock)() = ^{
		[app endBackgroundTask:backgroundTaskID];
		backgroundTaskID = UIBackgroundTaskInvalid;
	};
 
	backgroundTaskID = [app beginBackgroundTaskWithExpirationHandler:completionBlock];
 
	// wait for all deletions to be done
	[self waitUntilFinished];
 
	// ... when the syncing task completes:
	if (backgroundTaskID != UIBackgroundTaskInvalid)
	{
		completionBlock();		
	}
}

Notice the __block which tags the backgroundTaskID variable such that it can be modified from inside the blocks. Without this attribute the value of the variable would be captured at the time of creation of the block. You can also see that I reuse the completionBlock variable because the same code is to be executed on the timeout as is on successful completion. The actual work is just to wait for the _delQueue to finish.

Update: Gavin McKenzie correctly pointed out to me that the above mentioned approach causes subsequent renames to have to wait for the entire async block to finish. This is why the public version of DTAsyncFileDeleter now has a second rename queue and performs the renames on a sync queue too. Gavin explains it thus:

In the original implementation, additional callers would wait until all queued renames were finished — which is exactly what your code comment said. Which, could (in theory) mean that if 20 renames were queued, even the first rename operation would have to wait for the 20th queued rename to finish. That potential worried me, of having basically a block-wait on all the renames.

With the dispatch_sync approach each rename only has to wait for the previous rename to finish.

Update 2: Steve Weller was the only one spotting the major problem with my approach and I needed some lab tasting to understand why he was right. The wait was occurring on the main thread of the application. So if that was blocking then the watchdog timer would see your app as not responding after 10 minutes and kill it. So the finally perfect approach was to do away with the notification and instead wrap each block with a task completion id, which – according to the documentation – can be called from background threads safely. Please look at the DTAsyncFileDeleter class in DTFoundation for the latest version.

Conclusion

GCD offers us a possibility to stick multiple items into a background queue that we don’t want to happen concurrently. The background queue will then happily work off these items and if we have it in a group then we can also wait for the completion in a safe way.

dispatch_once is a safe and convenient method of making sure that something only occurs a single time and thus ideal to create caches or shared instances.

Finally – according to an Apple engineer who was asked this question at a TechTalk – you can have as many background task completions as you wish, provided you clean up properly. Because if you don’t then your app will be killed.

One question that I couldn’t find an answer for at the time of this writing is if you can do away with the notification but instead “warp the operations in a background task” from the get go. My feeling is that this would be less elegant because you would have to dispatch onto the main thread to get a background task ID and to invalidate it in the end. But I am interested if you know any more elegant solutions…

The source code for DTAsyncFileDeleter can be found in my DTFoundation project on GitHub.

When I moved the default CSS rules into a separate file I was facing the old dilemma: how can I embed this in the static library but still be able to easily add contents to it via Xcode. I previously explained how you can turn any file into a c-Byte-array. But this still required manual work.

I did a bit of researching and found that on regular Linux systems people seem to have a tool named objcopy which can copy files as their are into an object file (.o) which can be linked together to the final binary by the linker. But unfortunately this tool does not come with Xcode. So it is out of the question because I want everybody to be able to build DTCoreText.

Xcode Build Rules come to the rescue. They can automate any kind of preprocessing you like and it turns out they are an easy solution for this very problem.

There are essentially three kinds of source file that Xcode knows how to deal with (greatly simplified):

• c, c++ or Objective-C source code, or any code that can be somehow compiled
• header files
• other files

“Other files”, if you add them to a target, are treated as resources that simply get copied into your app bundle.

And now I’ll teach you how you can make any kind of file compilable. Be it some textures that you are compressing for building your game, or be it some resource that you want to embed into your binary.

Make a Build Rule

When Xcode is asked to compile a file it has a set of rules how it does that. So what we are going to do is to add our own rule how a css file is turned into a .c file. And for .c files are something that Xcode has a built-in rule for. So it can happen that you have two or more daisy chained rules that hand off their results to each other.

It is somewhat inconvenient that you cannot define rules globally. Instead you have to repeat your custom rules for each target.

To add your own rule, go into the projects info where you see your targets and on the target you want the rule for add it like shown.

This rule matches all files that end with .css (asterisk is a wildcard) and executes a custom script:

cd "$INPUT_FILE_DIR"  # move into file dir, otherwise xxd takes the full path for the symbol
/usr/bin/xxd -i "$INPUT_FILE_NAME" "$DERIVED_SOURCES_DIR/$INPUT_FILE_BASE.css.c" # builds a c file with a hex array

I’m moving into the folder of the input file first because xxd will always take the full passed file path and turn it into the name of the c-array. This way it will just be named default_c and the length will be in default_c_len.

The above rule is all it takes to teach Xcode how to turn a .css file into a .c file. Now the next step is to make sure that we are actually compiling the default.css file as opposed to copying it together with the other resources.

Compile not Copy

You can see that default.css is compiled by having it in the Compile Sources section of the Build Phases.

If you build the project you see two hints about what happens now. Towards the top of the build log you see the preprocessing of the file. A little further down you see how the intermediate c file is being compiled.

Further down the library tool puts all object files into the library archive. If you don’t believe me, you can employ the nm tool to inspect the symbols in the final product.

nm DemoApp | grep default_c
00045474 D _default_css
00045fe0 D _default_css_len

These are the two symbols from the default.css.c file that made it into the final product.

This guide wouldn’t be complete if I didn’t show you how to get to these symbols.

Accessing the Objectified File

I told you above that xxd uses the passed file name to make two variables, one for the c-style array, one for the length. Only change is that it has to turn the file name characters into a legal symbol, so dots are turned into underscores.

In DTCSSStylesheet.m I am accessing this array like so. I define these two variables as external which prompts the linker to insert their correct address.

+ (DTCSSStylesheet *)defaultStyleSheet
// external symbols generated via custom build rule and xxd
extern unsigned char default_css[];
extern unsigned int default_css_len;
}

And to get the string out of them is just as easy:

+ (DTCSSStylesheet *)defaultStyleSheet
{
   // get the data from the external symbol
   NSData *data = [NSData dataWithBytes:default_css length:default_css_len];
   NSString *cssString = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
 
   return [[DTCSSStylesheet alloc] initWithStyleBlock:cssString];
}

This works because I know that all my source files are UTF8 encoded, i.e. one byte per (normal) character. It is quite handy to have the default_css_len integer telling us the length of the array, because in c an array is just a pointer to the first byte.

Conclusion

If you know how to employ custom build rules to your advantage many tasks that you would have previously run external scripts for now become part of the build process. What you can do with this knowledge is only limited by your imagination … and your scripting skills.

But the general advantage of build rules versus external programs is that they are part of the project. So if they use standard commands they are portable to other developer’s machines. Perfect for Open Source projects.

In this post I will to demonstrate how you can contribute to an Open Source project hosted on GitHub. I previously blogged about how you can make and apply patches, which is certainly one way. But on GitHub there is an even cooler method, one that is also less work on the project maintainer.

For the sake of this tutorial I shall pretend that there is an issue with a project by Matt Galloway, then I’ll fork his project. After fixing the issue I will push my changes online to my fork and finally notify Matt about these changes via a so-called “Pull Request”.

A prerequisite for this tutorial is to have your GitHub account and local git set up previously.

Step 1 – We have an Issue

Your first thought should be to document that in GitHub’s excellent bug tracking system. Each Open Source project there has a list of Issues (which are the bugs and feature requests). I found two things to improve in the project and so I added two issues.

We could hope for the project’s owner to fix that for us, but – being smart developers – we can also proceed to fix the issues ourselves. From the get go we won’t have writing access to the GitHub repository (unless the owner gives it to us, which is rare). You can see that this is the case if you only see the HTTP and Git Read-Only options on the repo URL:

The usual method is to make a copy of this project in our own area on GitHub where we CAN write to. This copy is what they call a “Fork”.

Step 2 – Stick a Fork in it

To make a fork you klick the “Fork” button at the top right of the project main page. After some “hardcore forking action” you end up on your own copy of the project. And there is now an extra option “SSH” telling you that you have read and write access to that.

A fork is a complete copy of the original project including all previous history. The reason for this that with Git every copy of a repository is always a full clone, so that each clone can also serve as a backup. This illustrates the distributed nature of Git, as opposed to centralized Source Code Management (SCM) systems like Subversion which has a central server.

Step 3 – Clone Wars

To work on our copy of the project we need to clone it to our hard disk. You can use the excellent official GitHub app but personally I prefer to work in terminal. I open terminal, go to a place where I want the copy of the project to be and issue the command:

git clone git@github.com:Cocoanetics/ios-library-with-resources.git

The URL used is the SSH one I copied from the web page. This results in a ios-library-with-resources folder appearing in the current working directory.

Step 4 – Fix-Tours

Now we can do our changes. So I open the Xcode project.

The first thing I am fixing for this example is to add the two sub-project products as dependency so that they get automatically built if somebody builds the containing app.

You see an M (for Modified) appear next to the root of the project tree which means that the project file has been modified. Each such fix should become one commit. So we right-click on MyLibraryTest, Source Control, Commit Selected Files. This opens a dialog that shows which files were modified and prompts for a commit message.

Since we fixed issue number #001 with that we also add this tag, because later we will see that GitHub conveniently links this commit with the issue via this number.

This committing basically saves the current state in your local Git repository (your clone) and adds your message to that. Actually it did a bit more than that, because Git has a an extra step called “staging” before committing.

In staging you select the modifications you want to be part of the commit by Adding them to the staging area. The second part then does this saving. If you do this via the Xcode interface you don’t need this staging phase because you specifically selected the files to commit already via the project navigator.

The second issue to fix is to change a build setting on the sub-project.

This time the M appears next to the MyLibrary.xcodeproj which really means the project.pbxproj file contained in the MyLibrary.xcodeproj bundle. (Remember that a bundle is a folder that looks like a file)

This time let’s do the committing on the command like, like a real pro. First let’s see the status.

git status
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.
#
# Changes not staged for commit:
#   (use "git add ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
#	modified:   MyLibrary/MyLibrary.xcodeproj/project.pbxproj
#
no changes added to commit (use "git add" and/or "git commit -a")

This shows that we are ahead of our fork (origin/master) by 1 commit, which is the previous one we did. And it shows that the project file for the sub-project was modified. We issue a “git commit -a” which stages all modified files for the commit and immediately prompts us for a commit message in the currently set up text editor.

There is a second shortcut that GitHub recognizes besides the #002 tag. If you write “closes” or “fixes” in front of the hashtag it not only references the issue, but also marks it as closed. How convenient!

“Set Skip Install YES for resource bundle to prevent copying that into Archive. closes #002″

I type my message into the VIM that opened and save/exit via ESC, :wq, ENTER.

Now we have two local commits (git log to see them), next we need to push them to our online repository. We just had modifications to the project file in these two examples, but – trust me – it works just as well for code changes.

Step 5 – Push Up

When we cloned the project to our local hard disk Git was nice enough to automatically configure the location where it came from as so called “Remote”, the default name is “origin”. You can have multiple remote places set up and choose which to push.

The command for us is:

git push origin master

“origin” is the remote I told you about, “master” means that you are pushing this to the master branch, which is the default branch.

To check if our changes have really arrived online I like to check the commits view on GitHub. And indeed they have.

We could end here and many people do. They just make a copy of an Open Source project for the case that the original repository disappears. But since we are playing nice we want to offer our improvements to the original project’s owner.

Step 6 – I just called to say “Please Pull Me!”

Just like for the fork we find a “Pull Request” button at the top right of the GitHub website of our fork. Push it and you get a nice overview of what is contained in the request.

There are some advanced options regarding milestones and assignments, but we’ll ignore these for now. You’ll notice that the pull request automatically generates a new number, #3 in our case. So you can not only reference issues with your commit messages, but the same thing is possible for pull requests.

The final step is not for us, but for the original project’s owner to perform.

Step 7 – Accepted!

As owner of a GitHub repo it is up to you whether you want to accept the changes into your repository. If you kept your commits small and – for code changes – added some good comments to aid understanding then the owner will be happy to accept them. It saves him from doing the work that you did and all he needs is to push the green merge button.

The button to accept the changes is only green if the merge of the code can be performed automatically. If the author did some changes on his own that conflict with yours then the button is gray stating that the changes cannot be merged automatically. In this unfortunate situation either one of you needs to pull the other’s changes and manually resolve the merging conflicts. But this is a topic for another day.

Conclusion

I suggest you find yourself a a GitHub project to contribute to and try out this process to shed the timid feeling. It really is quite idiot proof.

People put their projects on GitHub not only so that you get free source code to use in your apps. They also hope that other developers might find and fix bugs. And every once in a while you see a pull request that makes you think “OMG I’m not worthy!” because those are the gems that really teach you something.

And this is why they call it collaborative coding.

The parsing of HTML necessary for my DTCoreText open source project is done entirely with the sophisticated use of NSScanner. But it has been long on my list to rewrite all of this parsing with a the industry standard libxml2 which comes preinstalled on all iOS devices. Not only is this potentially much faster in dealing with large chunks of HTML. It probably is also more intelligent in correcting structural errors of the HTML code you throw at it.

In part 1 of this series I showed you how to add the libxml2 library to your project and explained the basic concepts of using libxml2 for parsing any odd HTML snippet into a DOM. In this here part 2 we will create an Objective-C based wrapper around libml2 so that we can use it just like NSXMLParser, only for HTML.

There are basically two methods – approaches – to handling XML/HTML: DOM or SAX. DOM (short for Document Object Model) refers to building a tree of nodes and leaves which you then can query for information based on their path (with XPath). SAX (short for Simple API for XML) basically walks through the the document and sends you events as it discovers things. For example the begin of an element, the end of it, when a comment was found, etc.

NSXMLParser is a SAX parser for well-formed XML. That means you can use it only on XML, it cannot deal with the nature of HTML where some elements might not require that you close them. But if you look at the delegate protocol and compare it to the SAX interface of libxml2 you find an eerie resemblance. It might as well be that NSXMLParser is just a wrapper around the XML SAX interface of libxml2.

As we have previously learned we know that libxml2 also possesses a HTML parser which reuses most of the same internal data structures as its XML cousin. So we shall endeavor to create DTHTMLParser as a SAX parser for HTML.

Don’t worry about piecing together all the source code from this article, I will put that online in my DTFoundation project. Instead I want you to follow my lead and understand the pieces.

The libxml2 SAX Interface

The SAX interface works such that you register a handler function – in C – for each SAX event that you are interested in. This is done via a C-structure of type htmlSAXHandler which is nothing more than an aggregate of multiple C-function pointers.

In the libxml headers you often see some “html…” data structure typedef’d into one with “xml” as prefix. The reason for this is that it reuses whatever it can from the XML parser. But this also adds confusion – as I found through my experimenting – because some of the events that make sense for parsing XML don’t fire for HTML, like for example the one that reports CDATA elements. HTML does not have those, but the function pointer is still present in the htmlSAXHandler struct.

By CMD+Clicking on a type we can swim upstream to find the original definition of htmlSAXHandler:

htmlSAXHandler = xmlSAXHandler = _xmlSAXHandler =

struct _xmlSAXHandler {
    internalSubsetSAXFunc internalSubset;
    isStandaloneSAXFunc isStandalone;
    hasInternalSubsetSAXFunc hasInternalSubset;
    hasExternalSubsetSAXFunc hasExternalSubset;
    resolveEntitySAXFunc resolveEntity;
    getEntitySAXFunc getEntity;
    entityDeclSAXFunc entityDecl;
    notationDeclSAXFunc notationDecl;
    attributeDeclSAXFunc attributeDecl;
    elementDeclSAXFunc elementDecl;
    unparsedEntityDeclSAXFunc unparsedEntityDecl;
    setDocumentLocatorSAXFunc setDocumentLocator;
    startDocumentSAXFunc startDocument;
    endDocumentSAXFunc endDocument;
    startElementSAXFunc startElement;
    endElementSAXFunc endElement;
    referenceSAXFunc reference;
    charactersSAXFunc characters;
    ignorableWhitespaceSAXFunc ignorableWhitespace;
    processingInstructionSAXFunc processingInstruction;
    commentSAXFunc comment;
    warningSAXFunc warning;
    errorSAXFunc error;
    fatalErrorSAXFunc fatalError; /* unused error() get all the errors */
    getParameterEntitySAXFunc getParameterEntity;
    cdataBlockSAXFunc cdataBlock;
    externalSubsetSAXFunc externalSubset;
    unsigned int initialized;
    /* The following fields are extensions available only on version 2 */
    void *_private;
    startElementNsSAX2Func startElementNs;
    endElementNsSAX2Func endElementNs;
    xmlStructuredErrorFunc serror;
};

All these function pointer types are aptly named SomethingFunc and we’ll get to how these need to look like in a minute. The problem for us to solve how to bridge from our Objective-C paradigm (using self, properties, IVARs) into the C-function paradigm (no access to our parser object).

This problem is compounded by the fact that we are building this with ARC so that we don’t have to deal with memory management any more. Our resulting class will run on iOS 4 and above. Somehow we need to pass enough information to each event-function so that we can communicate with the parser object.

There are two reasonable possibilities: either we pass a reference to the parser instance or we pass a pointer to a struct that contains multiple values that we might need. I decided for the first because it seems cleaner to me.

libxml2 provides a concept of a user_data pointer that is passed to the event-functions together with other relevant information. We can use (abuse?) this for passing a reference to self. ARC requires this to be be cast to (__bridge void *) which in plain English means: “please treat this as just some typeless memory pointer and also don’t worry about retaining or releasing. I’ll take care of that, don’t panic.”

Like any self-respecting class we’ll start out with a good block of IVAR definitions and and init method.

@implementation DTHTMLParser
{
    htmlSAXHandler _handler;
 
    NSStringEncoding _encoding;
    NSData *_data;
 
    __unsafe_unretained id <DTHTMLParserDelegate> _delegate;
    htmlParserCtxtPtr _parserContext;
}

Notice that I’m defining the IVARs in the implementation, not the interface (in the header). This is possible as of using the new Apple LLVM compiler and provides for a nice way to hide the IVARs that we don’t want to be visible.

- (id)initWithData:(NSData *)data encoding:(NSStringEncoding)encoding
{
	self = [super init];
	if (self)
	{
		_data = data;
		_encoding = encoding;
 
		xmlSAX2InitHtmlDefaultSAXHandler(&_handler);
	}
 
	return self;
}

We save the data and the encoding into our IVARs, _data being a strong reference to it. That means that it retains it during the lifetime of our parser and will only release it when the parser goes away.

We also initialize the _handler structure via the xmlSAX2InitHtmlDefaultSAXHandler function. This sets up some initial internals and also chooses the SAX2 method for our work. There’s also an older SAX1 method, but we obviously prefer the newer one. This setup has to happen inside the init method because right after such an instance of a DTHTMLParser is create we want to be able to set the delegate which should take care of wiring up the needed functions.

Each event-function should have a corresponding delegate method in our DTHTMLParserDelegate protocol, but to avoid unnecessary function-calling we only add the functions to the handler for which the delegate actually implements the function.

#pragma mark Properties
 
- (void)setDelegate:(__unsafe_unretained id<DTHTMLParserDelegate>)delegate;
{
	if (delegate != _delegate)
	{
		_delegate = delegate;
 
		if ([_delegate respondsToSelector:@selector(parserDidStartDocument:)])
		{
			_handler.startDocument = _startDocument;
		}
		else
		{
			_handler.startDocument = NULL;
		}
 
		if ([_delegate respondsToSelector:@selector(parserDidEndDocument:)])
		{
			_handler.endDocument = _endDocument;
		}
		else
		{
			_handler.endDocument = NULL;
		}
// ...

This way when you set the delegate it is inspected one by one whether it responds to certain methods of the protocol. And whenever it does the corresponding function pointer is set in the _handler IVAR.

At the top of the file we need to define these functions. If we do just that then the compiler will warn us for all of these implementations that there was “no previous prototype”. In C a you call the first line of a C-function it’s prototype. So we just have an extra block for these.

#pragma mark Event function prototypes
 
void _startDocument(void *context);
void _endDocument(void *context);
void _startElement(void *context, const xmlChar *name,const xmlChar **atts);
void _endElement(void *context, const xmlChar *name);
void _characters(void *context, const xmlChar *ch, int len);
void _comment(void *context, const xmlChar *value);
void _error(void *context, const char *msg, ...);

Wherever you see a context parameter this will be the user_data pointer we will provide. And for example the _startElement function gets a name and an array of attributes as well. The xmlChar type is just a char. This is actually always provided as UTF8-encoded string regardless of what the data is encoded as.

Let’s look first at an easy event-function and then at a more complex one.

void _startDocument(void *context)
{
    DTHTMLParser *myself = (__bridge DTHTMLParser *)context;
 
    [myself.delegate parserDidStartDocument:myself];
}

You see how I cast the void pointer back into a pointer of type DTHTMLParser. I have to name the variable different than “self” because that is a reserved word. Then I can simply call the delegate method that corresponds to this event function.

By the way, these are the delegate methods we want to implement:

@class DTHTMLParser;
 
@protocol DTHTMLParserDelegate <NSObject>
 
@optional
- (void)parserDidStartDocument:(DTHTMLParser *)parser;
- (void)parserDidEndDocument:(DTHTMLParser *)parser;
- (void)parser:(DTHTMLParser *)parser didStartElement:(NSString *)elementName attributes:(NSDictionary *)attributeDict;
- (void)parser:(DTHTMLParser *)parser didEndElement:(NSString *)elementName;
- (void)parser:(DTHTMLParser *)parser foundCharacters:(NSString *)string;
- (void)parser:(DTHTMLParser *)parser foundComment:(NSString *)comment;
- (void)parser:(DTHTMLParser *)parser parseErrorOccurred:(NSError *)parseError;
 
@end

The @class is necessary before the protocol definition because a good delegate protocol also passes a reference to the caller. This also enables the casual reader of your code to know at a glance that this method belongs to a protocol connected to this class.

Now for something more complex, let’s look at didStartElement. The complexity of this method results from the arts parameter being alternating key, value, key, value etc.

void _startElement(void *context, const xmlChar *name,const xmlChar **atts)
{
	DTHTMLParser *myself = (__bridge DTHTMLParser *)context;
 
	NSString *nameStr = [NSString stringWithUTF8String:(char *)name];
 
	NSMutableDictionary *attributes = nil;
 
	if (atts)
	{
		NSString *key = nil;
		NSString *value = nil;
 
		attributes = [[NSMutableDictionary alloc] init];
 
		int i=0;
		while (1)
		{
			char *att = (char *)atts[i++];
 
			if (!key)
			{
				if (!att)
				{
					// we're done
					break;
				}
 
				key = [NSString stringWithUTF8String:att];
			}
			else
			{
				if (att)
				{
					value = [NSString stringWithUTF8String:att];
				}
				else
				{
					// solo attribute
					value = key;
				}
 
				[attributes setObject:value forKey:key];
 
				value = nil;
				key = nil;
			}
		}
	}
 
	[myself.delegate parser:myself didStartElement:nameStr attributes:attributes];
}

The while loop walks through the attributes and and when a NULL is encountered where a key would be then it breaks out of the loop. It is also necessary to deal with a situation where there is only an attribute name, but no value because HTML also allows that. Theoretically one attribute could also be occurring multiple times, but we ignore this. For later convenience we want the key/values for the attributes in an NSDictionary.

The final piece is how the parsing actually is started. For this we have a parse method, just like NSXMLParser.

- (BOOL)parse
{
	void *dataBytes = (char *)[_data bytes];
	int dataSize = [_data length];
 
	// detect encoding if necessary
	xmlCharEncoding charEnc = 0;
 
	if (!_encoding)
	{
		charEnc = xmlDetectCharEncoding(dataBytes, dataSize);
	}
	else
	{
		// convert the encoding
		// TODO: proper mapping from _encoding to xmlCharEncoding
		CFStringEncoding cfenc = CFStringConvertNSStringEncodingToEncoding(_encoding);
		CFStringRef cfencstr = CFStringConvertEncodingToIANACharSetName(cfenc);
		const char *enc = CFStringGetCStringPtr(cfencstr, 0);
 
		charEnc = xmlParseCharEncoding(enc);
	}
 
	// create a parse context
	_parserContext = htmlCreatePushParserCtxt(&_handler, (__bridge void *)self, 
		dataBytes, dataSize, NULL, charEnc);
 
	// set some options
	htmlCtxtUseOptions(_parserContext, HTML_PARSE_RECOVER | HTML_PARSE_NONET |
		HTML_PARSE_COMPACT);
 
	// parse!
	int result = htmlParseDocument(_parserContext);
 
	return (result==0);
}

There is a xmlDetectCharEncoding function in libxml to try and detect the encoding of the data which we invoke if the passed encoding is 0. The way to convert the NSStringEncoding into the xmlCharEncoding is only work shift, that needs some improving, possibly by simply having a big switch statement. Maybe later.

There are multiple ways how you can parse a document, some of which take individual parameters, some take a html parsing context. I settled on using the one for the push interface because there the htmlParseDocument function does not return a pointer to an html document tree (DOM), but instead a result that is 0 for success or -1 for error.

It is my understanding that this variant does not actually build the entire tree but keeps just enough state information to track the current hierarchy level inside the HTML tree. The push parser variant also has the ability to parse chunks of HTML data as they become available, e.g. while downloading. But exploring that is outside our scope for now.

Conclusion

Once your eyes have gotten used to the coding style of libxml2 you can start to appreciate the plethora of functions that it provides. You can also peruse the official reference manual online.

Finally I invite your collaboration on using and helping to polish DTHTMLParser if you have any need for parsing HTML text from within your iOS apps. Look for it on the GitHub DTFoundation project as soon as I can add a bit of AppleDoc-style documentation to it.

In my previous article about Sub-Projects in Xcode I showed you how you can have an Xcode project as a dependency. The advantage of this is that you can update your library from within the same project and debug into the sub-project’s code.

One thing that was not immediately obvious was how you deal with the problem that Xcode cannot find your sub-project’s library headers. Hence this writeup.

A header file generally informs the compiler what c-functions, global variables, Object-C methods and classes exist in an object file. All these are generally called “symbols”.

Build Process Review

The build process consists of these three steps:

1. Preprocessor – this takes care of including/importing header files into your source code, essentially pasting the content of the specified file right into your source code for building. This step also replaces all instances of #defined values with what you defined them to be.
2. Compiler – this goes through all the .m files and builds an object file with .o extension for each. Each item that is not coming from the same .m file is marked as an external symbol.
3. Linker – this merges all the object files into one big binary while at the same time linking the symbols with the actual implementations.

Header files do not actually contain executable code, but instead they tell the compiler which names and functions it should accept. It is the linker then that resolves these references. In short: If you want to use something which is implemented outside the current .m file than you need the appropriate import for that.

There are two kinds of imports:

• <path/header.h> with angle brackets – these are meant for system or “global” includes.
• “path/header.h” with double quotes – these are meant for items that are limited in scope to your current project

You have probably used both already, the one with the angle brackets for including headers that Apple provided. The one with the double quotes for accessing your own classes. Paths are usually relative paths because everybody has a different folder structure on his hard disk.

Get the Headers into Your App

The easiest way to get the headers into your project is to just add them. But you should generally refrain from duplicating your code for convenience, because then you have to perform an update on every copy of the file. A variation of this theme would be to not add the file itself, but just a reference. But this approach would also make your project dependent on your folder structure. Don’t do it, just don’t.

If you have followed the Sub-Projects guide you are already perfectly set up to get to the header while still keeping the project self-contained and independent from the file system layout. You only need to tell the outer Xcode project – probably your app that includes sub-projects for some static libraries – where it can find the necessary headers from the sub-project. Again you have multiple options: you can point it to the source code folder where the .h files are located. Or you can point it to the build output folder. Both are equally viable but if you work with static universal frameworks – as I do – then you can only use option two without losing your mind.

On my projects that contain reusable code I usually have multiple targets. One for a universal static framework – which is very convenient to add to a project because it automatically sets up the library and header paths for you when you drag it into your project. I have a second target for a static library because this allows for using the project as a sub-project and have the outer project link to the static library product. Because the framework conveniently bundles together multiple .h files you usually have the general name of the framework as path, i.e. you find <MobFox/MobFox.h> as the import of choice. For for example an Apple framework <Foundation/Foundation.h>.

Contrasting the MobFox/ path the actual location of the header file is PROJECT_ROOT/Core, so no MobFox path component there. This is the reason why we cannot point the containing app to the source folder because there is no header file MobFox.h inside a folder named MobFox. So via approach number 1 we would never find the header, we want to take door number two which is to point Xcode to the static library build output folder. But for this to work some setup is necessary.

Specifying Public Header Output Location

After building the static library Xcode will also copy certain headers to the output folder. A header can be public, project or private. Public means that it will be distributed together with the library. Project and Private won’t. You can set this for every header file individually. Headers for objects that you want to be able to see from your app should be Public.

If you now build the static library and look at the build log you will see that at the very end the public headers are copied somewhere:

We are astonished to find a path ending in Build/Products/Debug-iphoneos/usr/local/include for the two public header files. The reason for this being the default setting is that traditionally people would build binary files for the system they are working on. But that’s not what we want. So we go into the build settings for the static library target and adjust this. Don’t put this entry into either the Debug or Release lines, but into the line above these so that it may override both.

When we now build we see the output go to Build/Products/Debug-iphoneos/MobFox – as we want it. It is this extra MobFox path component that enables us to use the <MobFox/MobFox.h> style of including later on.

Now back to our app that is including this sub-project’s static library. There we now need to tell the compiler where it can find the above folder. It so happens that when building Xcode usually uses the same output folder for all products it needs to build, namely whatever is in the $(CONFIGURATION_BUILD_DIR) variable at the time. This in turn is set by default to $(BUILD_DIR)/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME).

The final piece of the puzzle to know is where to put that. This depends on the include style that I explained above. Double quoted includes need the path in “User Header Search Paths”. Angle-bracketed includes need the regular “Header Search Paths”. We are forced to use the latter due to the framework. We don’t need to check the Recursive box because at the given location there is a MobFox folder.

The <Multiple values> might cause you to pause for a second, but remember what I told you above. The configuration (Debug/Release) is part of this path and so even though you put the same variable in there, the actual result is a different one. Note that these values are bold because the target settings override the settings specified by the project. You can revert to the project settings by selecting the line and hitting CMD+Backspace.

There is an alternate option to put this path in the “User Header Search Paths” instead, but then you need to set “Always Search User Paths” to Yes. This will cause the compiler to also search the user header paths for angle bracket includes. Though personally I prefer the other option that needs less settings.

The other steps for building are explained in the Sub-Projects posts. If you carry them out (link with target), you will find that now your app finds the headers and successfully builds.

Conclusion

If Xcode complains that it cannot find a header then you should make sure that it is indeed in a place where it can find it. There is an abundance of default settings that may seem daunting, but if you know what to look for you can quickly find why your header cannot be found.

I wrote this article because I myself was stumped by not being able to explain why a project couldn’t find the MobFox header. So another learning of this episode is that you should not assume that the owner of your sub-project has set everything up correctly. In my case I too was missing the custom header output path setting. Errare humanum est.

Sometimes you want to tell somebody how you fixed a problem in their code, but for some reason the code is not on github so you cannot send them a pull request. If you felt really smart then you might put the changes you made into an e-mail, like “in file1.m:102 you change it to x, in file2.m:54 you make change y”.

Thought this doesn’t really help the developer you are trying to help. Even when following your change instructions to the letter it is a tedious and error-prone method of applying your changes.

Thanks to M. Douglas McIlroy, Adjunct Professor at Dartmouth Colleague – one of the early Unix pioneers – there is a better way. He invented the form of diff that we are using today to get the Difference between two versions of a file. And Larry Wall who invented the patch command which can take a diff file and effectively apply it to files.

Honestly I was afraid of using patch before, never having played with it. What you don’t know you fear…. but since you are reading this article I now finally did, so that I can explain its usage.

Making a Patch

Let’s try with a simple text file first.

file1.txt

Here's some text.
And a second line.
And a third line.
A fourth line.

file1_changed.txt

Here's some text.
And a third line.
A fourth line changed.

To get the difference between these files we just diff them:

diff file1.txt file1_changed.txt

This produces this output:

2d1
< And a second line.
4c3
< A fourth line. --- > A fourth line changed.

We see that diff features each changed block with a sort of address. We removed line two, so we see that the d in 2d1 probably means deleted. And the c in 4c3 probably means changed. If a line was changed you first see the original line prefixed by a less-than, then three dashes and then the changed line prefixed by a greater-than. diff also has a -y flag that puts the changes side-by-side, which more clearly visualizes the changes.

This kind of output is called the “normal” format. To be able to use the diff for patching this is missing some vital information, namely which file the changes belong to. For this purpose you have to use the “unified” format which you get with the -u flag. We also pipe the output into a new file.

diff -u file1.txt file1_changed.txt > patch.diff

This results in the patch file to have this content:

--- file1.txt	2011-12-25 12:17:30.000000000 +0100
+++ file1_changed.txt	2011-12-25 12:01:30.000000000 +0100
@@ -1,4 +1,3 @@
 Here's some text.
-And a second line.
 And a third line.
-A fourth line.
+A fourth line changed.

You can see that has information about which files are involved. Also you have minus and plus prefixes. A minus with a plus following means that the line was changed. Just a minus means that the line was only present in the original, i.e. deleted. Just a plus means that line was only present in the modified version, i.e. inserted.

Let’s Patch

Appling the patch we just created is very simple. If the current working directory contains both the patch and the to-be-patched file all we need is:

patch < patch.diff

You get an output telling you while files where touched. As soon as you have applied a patch, calling the patch command again with the same patch will get you a message about this giving you the option to ignore individual “hunks” which is what these blocks of changes are called.

Multiple Files

For multiple files the process is only slightly more complex. Say you have a a folder “original” and a folder “changed” and the latter contains multiple changed versions of the files in the former.

If you call diff passing a folder name instead of a file name then this causes diff to compare file1 in one folder against file1 in the second folder, exactly what we need. There are a couple of additional flags for diff that are of use in this case:

• -r to recurse through all subfolders of the passed folders
• -u to get the “unified format” output
• -p prints the c-function a change was contained in
• -N treats absent files as empty

So we go into the folder where our “original” and “changed” folders are located in and do:

diff -rupN original changed > original.patch

With this patch file it is easiest if you just move into the root of the original and copy the patch there. Then you can apply it without any additional parameters. Even though the diff contains the original and changed path prefixes it figures out that you must have meant the file contained in the current working directory.

cd original
cp ../original.patch .
patch < original.patch

There is one parameter of the patch command that you might need if the paths mentioned in the diff refer to a different folder structure than you have on your own machine. The man page of patch has this to say about -p:

-pnum or –strip=num
Strip the smallest prefix containing num leading slashes from each file name found in the patch file. A sequence of one or more adjacent slashes is counted as a single slash. This controls how file names found in the patch file are treated, in case you keep your files in a different directory than the person who sent out the patch. For example, supposing the file name in the patch file was

/u/howard/src/blurfl/blurfl.c

setting -p0 gives the entire file name unmodified, -p1 gives

u/howard/src/blurfl/blurfl.c

without the leading slash, -p4 gives

blurfl/blurfl.c

and not specifying -p at all just gives you blurfl.c. Whatever you end up with is looked for either in the current directory, or the directory specified by the -d option.

When I saw someone use patch for the first time he totally confused me by claiming that -p is short for the “patch level” which – as you can see above – is utter nonsense. It is probably short for prefix and the long form of it –strip actually means something entirely different.

It is essentially the number of slashes that are to be ignored, including the part of the path to the left of each ignored slash. A strip level of 0 does not modify the paths at all. A level of 1 ignores the path up to and including the first slash. Omitting the p parameter causes patch to ignore the complete paths and only look at the file names.

If a patch without specified strip level doesn’t work then you will have to take a look at the patch file and figure out how much of the paths you need to ignore so that these paths fit with the situation on your own file system. In our example above (with the original and changed folders) we could have used -p1 to have patch ignore the relative paths, but it seems to do that by itself.

Creating Patches from Repositories

Above we discussed one way to create diffs that can be used for patches by having two versions of your files in two folders and diff these two. This typically suites those people who have not yet warmed to the use of source code management systems (SCM) like git or Subversion.

In an SCM you don’t have multiple copies of each file, instead you have only one current version of the source code in the project directory and all previous versions are somehow hidden from view, but can be retrieved. In git this versions are called “commits” in Subversion “revisions”.

For the sake of trying it out, let’s create an empty Xcode project with a git repo. On the dialog where you specify the folder for the project you can set a checkbox to have Xcode also put it under version control.

It is probably an oversight in Xcode but if you create the project like this the first commit is being done for you with comment “Initial Commit”, however there are changes to the project file adding a “lastKnownFileType” to the project.pbxproj – which is contained in the xcodeproj bundle – that causes the project file to show as modified if you do “git status”.

To get to a clean starting state for the project I go to the folder where I created the project and do an additional commit.

cd ~/scmtest
git status
git commit -a
git log

I have vi set up as my editor so on the file that opens from the commit I enter some comment and then ESC : w q to save and finish the commit. The git log now shows no uncommitted changes, two commits and two untracked files containing some user workspace settings which you probably don’t need or want in your repo.

Now let’s modify some code. For this test I just added an NSLog into the app delegate file. What’s really interesting now is how to create a patch that contains the changes from our second commit to the current modified state of our working directory. Simple, really:

git diff --no-prefix > diff.patch

We need the option to omit the prefix because otherwise git diff adds an “a” and a “b” prefix in the paths which never correspond to actual paths. Of course you can also apply patches that still contain the prefixes if you use the -p1 strip level.

Our patch now looks like this:

diff --git scmtest/DTAppDelegate.m scmtest/DTAppDelegate.m
index 7730864..795fa6d 100644
--- scmtest/DTAppDelegate.m
+++ scmtest/DTAppDelegate.m
@@ -20,6 +20,8 @@

 - (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
 {
+    NSLog(@"Hello World");
+
     self.window = [[[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]] autorelease];
     // Override point for customization after application launch.
     self.window.backgroundColor = [UIColor whiteColor];

You can see the inserted lines prefixed with plus, also there are a couple of lines of code before and after the insertion so that patch can find the place even when there have been other modifications to the file that changed the line numbers.

A simple git diff always contains all modifications made on the latest commit. To test this patch we discard our changes and see that we ended up on the second commit.

git reset --hard
git log

Then we apply the patch, we need a strip level of zero so that the relative paths are used as they are.

patch -p0 < diff.patch
git status

After this patch we see that the changes appeared again.

You can also create a patch between any two commits by specifying the commit identifiers as parameters. You can get the identifiers from git log.

git log
git diff 92cef602e685970e6d4316cd383217d39d169b89 ec24c6d3435a2ae9ddaabd76f958561e86f9f02e > b.patch

Finally let’s also see how the same can be gotten from an SVN repo. Same technique:

svn status
svn diff > diff.patch

Between two specific revisions the same is possible. Subversion numbers revisions incrementally instead of with identifiers as git does. So the command for the changes from a specific version to the next is this:

svn diff -r 126:127 > c.patch

Contrary to git the diff command for svn does not prefix the file paths, instead the revision info is contained in brackets after the files which is ignored by patch.

Conclusion

Knowing how to create and apply patches is a great skill to have as a programmer. So the next time you find that you want to precisely communicate what changes you made in somebody else’s code you should send them a patch instead of an email containing line numbers and code fragments.

And if they don’t know what to do with that, also send them a link to this here tutorial.

PS: AlBlue points out that on git your can also directly send patches via e-mail. Those even include your commit messages and identity.

Honestly I was very much excited when I found that I can use my current knowledge of Objective-C and Foundation classes like NSString to also build nifty little tools. Previously I had to resort to bash script to perform one-off operations on files that where too tedious to do manually. But knowing what I am going to show you in this article will enable you to also write these littler helpers.

I believe that beginners should rather start with writing a couple of command line tools before diving into building user interface driven apps. Commend line tools are linear and thus their function is easier to grasp. They are more akin to “functional programming” then “object oriented programming” if you will.

I am going to show you what goes into building a simple command line tool and you be the judge whether you agree with my assessment.

First we need a purpose for our tool. It just so happens that I have a great idea!

Purpose: Join Several Files in a Structure

Somebody handed me a set of all the Localizable.strings files from Apple’s built-in apps.  Those are divided by app and there you have one lproj sub-folder per language. I want to join all of these together in a big file so that I want to look up one translation I get all languages at the same time.

The strings files themselves are binary property lists, basically a big NSDictionary each. That’s the same format that they get converted to if you build your app. Xcode takes each text strings file, omits the comment and packs them into the bplist format for faster loading.

Before you set out to coding anything it is a good idea to reflect on which steps we think we require to achieve our goal:

1. Produce a command line utility that can be called from the terminal and passed one directory as parameters.
2. Iterate through the directories and get the contained files
3. Open each file ending in .strings as dictionary
4. Assemble in memory the tokens contained in each dictionary in a way that will facilitate generating the output we require

Ok, let’s get to it.

Step 1: Command Line Tool with Parameters

We create a new project with the “Command Line Tool” from the Mac OS X/Application category. Under “Type” we see a couple of different templates. We want to use the one that says “Foundation” because this gives us all the Objective-C classes we know and love, like NSArray, NSString etc. Let’s also make it an ARC project so we don’t have to worry about memory management. I called my tool stringsextract.

As with all C-based applications the starting point for the program is in the main function which you can find in main.m.

int main (int argc, const char * argv[])
{
    @autoreleasepool {
        // insert code here...
        NSLog(@"Hello, World!");
    }
    return 0;
}

If it weren’t for the autorelease pool this would look like a very normal C-function. In objc methods generally are either class (+) or instance methods (-). In c functions are standing alone.

This function main has an int as return value. If the program runs without error you want it to return 0, otherwise you want it to return something else, like 1.

It has two parameters argc and argv. The first (argument count) is the number of parameters contained in the second (argument values). Int is just a number, but the second looks a bit more completed to the untrained eye. It is an array of char pointers. Since in c arrays are same as pointers you sometimes see this also written as char **argv, because in essence it is a pointer to a pointer to a char.

Arrays in C are not secured against being overrun which is why we need the argc. Otherwise we would have no way of knowing how many entries this array has. C-style arrays are just some memory where the elements are in sequential order. To access the nth item, you use array[n-1] and the compiler will convert that into the correct memory address. (Note: in c indexes begin with 0)

Another thing to know about argc/argv is that there is always at least one parameter at index 0: the name and path of the executable. So even if you call this program without any parameters on the command line argc will be 1 and there will be a value in argv[0].

The next tricky thing besides having to deal with a c-style array is that we also have to know c-style strings to make sense of the argument values. I told you above that an array is just a bunch of items. C-strings are just a bunch of characters where the last one is a binary zero, aka ‘\0′, to delimit the string. So whenever you see char * this could either be a pointer to one character or it could be a pointer to the first character of a string.

Since we want to use Foundation constructs for our program we want to convert all c-stuff to the more intelligent variants like NSString, by means of stringWithUTF8String.

As a first experiment let’s output all the arguments. We do a normal for loop beginning with 0, convert each argument to an NSString and log it.

for (int i=0; i<argc; i++)
{
    NSString *str = [NSString stringWithUTF8String:argv[i]];
    NSLog(@"argv[%d] = '%@'", i, str);
}

If you hit Run now you will see the following output on the console:

2011-12-11 09:21:58.636 stringsextract[18003:707] argv[0] = '/Users/oliver/Library/Developer/Xcode/DerivedData/stringsextract-dlkdghopgfoghoanjhmahhrlotsp/Build/Products/Debug/stringsextract'

So really what I told you is true, the parameter at index 0 has the path of our binary.

How can we test our program while we are still working on it? Of course you could open a terminal and run it by hand:

iair:~ oliver$ cd /Users/oliver/Library/Developer/Xcode/DerivedData/stringsextract-dlkdghopgfoghoanjhmahhrlotsp/Build/Products/Debug/
iair:Debug oliver$ ./stringsextract bla
2011-12-11 09:25:03.357 stringsextract[18017:707] argv[0] = './stringsextract'
2011-12-11 09:25:03.363 stringsextract[18017:707] argv[1] = 'bla'

You see how I changed into the Debug output directory and executed the program there passing one extra parameter. The dot slash is necessary to tell the shell that it should take the stringsextract binary right there in the current directory. Otherwise it would go off searching the set up system paths. You also see that argv[0] does not actually contain the absolute path, but always how it was called.

Now this works, but it might be a bit tedious to work with while we are still testing and debugging our tool. Fortunately Xcode allows us to specify command line arguments in the schema. If one argument contains whitespace you need to put it in quotes.

If we hit Run like this then we can also debug and step through our code as if we had passed this parameter on the command line ourselves.

As a final step in this chapter let us add a usage text which is to be output if somebody calls this program of ours without the required one parameter.

// one parameter is required
if (argc!=2)
{
    // output usage
    printf("Usage: stringsextract <dir>\n");
 
    // leave with code 1
    exit(1);
}

Note the use of the c-function printf which unlike NSLog just outputs the stuff we want and no extra stuff.

Now we’re set to move to Step 2.

Step 2: Getting the Files

Foundation gives us some awesome utilities when working with the file system, most notably NSFileManager which groups together most of them. We have our parameter specifying an absolute path. So next we need to list the .app bundles contained in there.

Usually you would use [NSFileManager defaultManager] to get a file manager instance. Only if you need to set a call-back delegate you would go about alloc/init’ing it.

So following our check for the existence of at least one parameter we add:

// convert path to NSString
NSString *path = [NSString stringWithUTF8String:argv[1]];
 
// get file manager
NSFileManager *fileManager = [NSFileManager defaultManager];
 
// get directory contents
NSError *error;
NSArray *contents = [fileManager contentsOfDirectoryAtPath:path
                                                     error:&error];
 
// handle error
if (!contents)
{
    printf("%s\n",[[error localizedDescription] UTF8String]);
    exit(1);
}
 
NSLog(@"%@", contents);

This converts the parameter to an NSString, sets up a file manager and then gets the contents of the specified path. If this is successful then there is a non-nil value in contents. If not then error will point to a new NSError instance which gives us a nice localized description to tell the user. Again we use printf because we want the direct output. UTF8String is the method to get from an objective-C NSString to back to a regular C-string which is required by the %s print format.

The NSLog at the bottom is just temporary so that we can see if we indeed got a result. And so we did.

2011-12-11 10:08:47.023 stringsextract[18231:707] (
    ".DS_Store",
    "AppStore.app",
    "Calculator.app",
    "Camera.app",
    "Compass.app",
    "Contacts~iphone.app",
    "Game Center~iphone.app",
    "Maps~iphone.app",
    "MobileCal.app",
    "MobileMail.app",
    "MobileNotes.app",
    "MobilePhone.app",
    "MobileSafari.app",
    "MobileSlideShow.app",
    "MobileSMS.app",
    "MobileStore.app",
    "Music~iphone.app",
    "Nike.app",
    "Preferences.app",
    "Reminders.app",
    "Setup.app",
    "Stocks.app",
    "Videos.app",
    "Weather.app",
    "YouTube.app"
)
Program ended with exit code: 0

We also see that there is one entry that we want to ignore later on, the “.DS_Store”. So let’s filter out everything that does not end in .app as I showed in a previous post.

Now we need to iterate through the array of app paths and get the contents of these, filtering for lproj. Since the contents are without the full path we need to concatenate the individual names with the path.

// filter
NSArray *filter = [NSArray arrayWithObject:@"app"];
 
// execute filter in place
contents = [contents pathsMatchingExtensions:filter];
 
// iterate through .app
for (NSString *oneApp in contents)
{
    // make full path
    NSString *appPath = [path stringByAppendingPathComponent:oneApp];
 
    // get contents, ignore error
    NSArray *appContents = [fileManager contentsOfDirectoryAtPath:appPath
                                                            error:NULL];
 
    NSLog(@"app: %@, %@", oneApp, appContents);
 
}

Again we have a bunch of files that we one and one that we don’t, in this case I have have a _CodeResources. Granted I could have removed these by hand but we’re writing this program here to be smart and save us manual labor. So we code around that just the same.

Rinse and repeat. We end up with three loops, one for the apps, one for the lprojs and one for the strings.

int main (int argc, const char * argv[])
{
    @autoreleasepool 
    {
        // one parameter is required
        if (argc!=2)
        {
            // output usage
            printf("Usage: stringsextract <dir>\n");
 
            // leave with code 1
            exit(1);
        }
 
        // convert path to NSString
        NSString *path = [NSString stringWithUTF8String:argv[1]];
 
        // get file manager
        NSFileManager *fileManager = [NSFileManager defaultManager];
 
        // get directory contents
        NSError *error;
        NSArray *contents = [fileManager contentsOfDirectoryAtPath:path
                                                             error:&error];
 
        // handle error
        if (!contents)
        {
            printf("%s\n",[[error localizedDescription] UTF8String]);
            exit(1);
        }
 
        // filter
        NSArray *filter = [NSArray arrayWithObject:@"app"];
 
        // execute filter in place
        contents = [contents pathsMatchingExtensions:filter];
 
        // iterate through .app
        for (NSString *oneApp in contents)
        {
            // make full path
            NSString *appPath = [path stringByAppendingPathComponent:oneApp];
 
            // get contents, ignore error
            NSArray *appContents = [fileManager contentsOfDirectoryAtPath:appPath
                                                                    error:NULL];
 
            // new filter
            filter = [NSArray arrayWithObject:@"lproj"];
 
            // execute filter in place
            appContents = [appContents pathsMatchingExtensions:filter];
 
            // iterate through .lproj = one language
            for (NSString *oneLang in appContents)
            {
                // make full path
                NSString *langPath = [appPath stringByAppendingPathComponent:oneLang];
 
                // get contents, ignore error
                NSArray *langContents = [fileManager contentsOfDirectoryAtPath:langPath
                                                                         error:NULL];
 
                // new filter
                filter = [NSArray arrayWithObject:@"strings"];
 
                // execute filter in place
                langContents = [langContents pathsMatchingExtensions:filter];
 
 
                for (NSString *oneStrings in langContents)
                {
                    // make full path
                    NSString *stringsPath = [langPath stringByAppendingPathComponent:oneStrings];
 
                    // do work here
                    NSLog(@"%@", stringsPath);
                }
            }
        }
    }
    return 0;
}

If this code makes your stomach churn then this is because this more and more turns into spaghetti code. This can still be fine for a one-off tool, but should be avoided if you plan to hand out this tool for others to use.

For one thing I would add a category to NSFileManager that does the listing, filtering and full-path-assembling in one step. Or possibly even go one further to have the working code in a block and have this category iterate over the strings files.

This we will be exploring in the next installment of this two part tutorial.

A very interesting yet very undocumented functionality of Xcode is that you can have sub-projects in your project tree. You can add an xcodeproj to your project and link to this project’s output.

This is exceptionally useful if you are developing some functionality in a contained project and now want to access this polished functionality from another project. Like for example you want to add to your app the capability of accepting HTML code copied from Safari and use my DTWebArchive classes for that. You could either copy all classes to your project, build two libraries (one for Simulator and one for Device, or lipo these two together), or build a static universal framework.

Or there is an option number 4 which I want to tell you about in this post. This option does neither copy source code nor does it involve building something upfront.

Note that Xcode 4.2 might cause you a bit of trouble until this works right. Some people told me that they did what I am about to describe in Xcode 3 to get it working. One way or the other this here should work, and if it doesn’t work for you then “you’re holding it wrong”.

Sub-Projects, An Overlooked Option

This is the first case where I myself am using sub-projects. My DTRichTextEditor framework is making heavy use of my two open source projects. This is how the project tree looks on my machine:

The file system structure and the group structure are more or less identical. I have a folder Externals that contains clones of these two GitHub projects. When you add an xcodeproj to your project then it will show up as shown in the picture, allowing you to browse the source code contained in this sub-project as well. Better yet, when you’re debugging you get to step through the sub-project’s code as if it where local.

There are several caveats, besides sometimes having to restart Xcode several times for the project trees to show up. Xcode seems to take objection to you having the sub-project also opened at the same time. If you did that you will get an error message that the consistency of the sub-project cannot be verified. So make sure you don’t have the sub project open at the same time. You can edit the code from within the larger project anyway.

Preparing the Sub-Project

For being able to use an xcodeproj like shown above you need a couple of things, let me walk you through these.

You need at least one target that builds an iOS static library. Go to your project root and add such a target. Then pick all the headers and implementation files that you want to be part of this library.

I like to name the target different from the product. I want the target to be called “Static Library” and then I go into the build settings and call the product DTWebArchive, resulting in a product libDTWebArchive.a showing up under Products.

You do not need to modify the header use which can be project, private or public. Changing it from the default project just copies these headers into the product output path which is of little use. You might also want to adjust the name of the Scheme, or auto generate schemes for your targets. These schemes are just a convenience for you personally, they are not even saved in the xcodeproj file.

Make sure that your library builds without warnings or errors. Note that if you didn’t select any source files then the libtool will fail, too, without any useful error message. That happened to me on several occasions.

Adding the Sub-Project

Once you are satisfied with the static library close this project and open the other one that should contain it. Depending on how you work you could either keep all your projects in the same location next to each other or have the larger project actually also contain the smaller one. It’s up to you. I recommend the second option because your file system structure might be different from somebody else building this project. If you use absolute paths, or even relative ones pointing outside of the project root folder then you are asking for trouble.

When adding the sub-project you don’t need to select it to be a member of any targets just yet. Best if you just drag the xcodeproj from Finder into the project.

Next you need to add the path to look for the sub-project’s headers in to your build settings. You cannot add the .h files to your main project, but you can tell the compiler where it can find them. Specify the path relative to your project’s root folder. I chose recursive because I have multiple sub-projects and this way I only need to specify this setting once.

With this setup you can use the same import directives that you are used to.

You tell the linker to link against the static library target in the sub project. You set this up on the Build Phases tab, under “Link Binary With Libraries”. You probably want it to be “Required” for static libraries because we cannot build dynamic libraries (dylib) as Apple is using in their frameworks.

You also want to add the static library targets as dependencies for your app. This way the build system knows that it needs to rebuild the app object code if code in the dependency changes.

If you were only programming in c then that would be all, but since we are coding in Objective-C (and possible use ARC) there are “Other Linker Flags” we need to add:

• -ObjC because otherwise the linker does not properly load your classes
• -all_load if you have categories in the library, because otherwise those are not loaded
• -fobjc-arc if you are linking against a library built with ARC from an app that does not yet use ARC

Actually the first two are pretty standard, so if you have ever done anything with static libraries before you already know about these.

Conclusion

You see, there is not very much too it and you can stop copying your reused source code all over the place. Better you put it in your own foundation framework (as I have begun with DTFoundation), document it and add the appropriate unit tests. This way any new methods or polishing benefits all your apps instead of just the current one.