When I moved the default CSS rules into a separate file I was facing the old dilemma: how can I embed this in the static library but still be able to easily add contents to it via Xcode. I previously explained how you can turn any file into a c-Byte-array. But this still required manual work.

I did a bit of researching and found that on regular Linux systems people seem to have a tool named objcopy which can copy files as their are into an object file (.o) which can be linked together to the final binary by the linker. But unfortunately this tool does not come with Xcode. So it is out of the question because I want everybody to be able to build DTCoreText.

Xcode Build Rules come to the rescue. They can automate any kind of preprocessing you like and it turns out they are an easy solution for this very problem.

There are essentially three kinds of source file that Xcode knows how to deal with (greatly simplified):

• c, c++ or Objective-C source code, or any code that can be somehow compiled
• header files
• other files

“Other files”, if you add them to a target, are treated as resources that simply get copied into your app bundle.

And now I’ll teach you how you can make any kind of file compilable. Be it some textures that you are compressing for building your game, or be it some resource that you want to embed into your binary.

Make a Build Rule

When Xcode is asked to compile a file it has a set of rules how it does that. So what we are going to do is to add our own rule how a css file is turned into a .c file. And for .c files are something that Xcode has a built-in rule for. So it can happen that you have two or more daisy chained rules that hand off their results to each other.

It is somewhat inconvenient that you cannot define rules globally. Instead you have to repeat your custom rules for each target.

To add your own rule, go into the projects info where you see your targets and on the target you want the rule for add it like shown.

This rule matches all files that end with .css (asterisk is a wildcard) and executes a custom script:

cd "$INPUT_FILE_DIR"  # move into file dir, otherwise xxd takes the full path for the symbol
/usr/bin/xxd -i "$INPUT_FILE_NAME" "$DERIVED_SOURCES_DIR/$INPUT_FILE_BASE.css.c" # builds a c file with a hex array

I’m moving into the folder of the input file first because xxd will always take the full passed file path and turn it into the name of the c-array. This way it will just be named default_c and the length will be in default_c_len.

The above rule is all it takes to teach Xcode how to turn a .css file into a .c file. Now the next step is to make sure that we are actually compiling the default.css file as opposed to copying it together with the other resources.

Compile not Copy

You can see that default.css is compiled by having it in the Compile Sources section of the Build Phases.

If you build the project you see two hints about what happens now. Towards the top of the build log you see the preprocessing of the file. A little further down you see how the intermediate c file is being compiled.

Further down the library tool puts all object files into the library archive. If you don’t believe me, you can employ the nm tool to inspect the symbols in the final product.

nm DemoApp | grep default_c
00045474 D _default_css
00045fe0 D _default_css_len

These are the two symbols from the default.css.c file that made it into the final product.

This guide wouldn’t be complete if I didn’t show you how to get to these symbols.

Accessing the Objectified File

I told you above that xxd uses the passed file name to make two variables, one for the c-style array, one for the length. Only change is that it has to turn the file name characters into a legal symbol, so dots are turned into underscores.

In DTCSSStylesheet.m I am accessing this array like so. I define these two variables as external which prompts the linker to insert their correct address.

+ (DTCSSStylesheet *)defaultStyleSheet
// external symbols generated via custom build rule and xxd
extern unsigned char default_css[];
extern unsigned int default_css_len;
}

And to get the string out of them is just as easy:

+ (DTCSSStylesheet *)defaultStyleSheet
{
   // get the data from the external symbol
   NSData *data = [NSData dataWithBytes:default_css length:default_css_len];
   NSString *cssString = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
 
   return [[DTCSSStylesheet alloc] initWithStyleBlock:cssString];
}

This works because I know that all my source files are UTF8 encoded, i.e. one byte per (normal) character. It is quite handy to have the default_css_len integer telling us the length of the array, because in c an array is just a pointer to the first byte.

Conclusion

If you know how to employ custom build rules to your advantage many tasks that you would have previously run external scripts for now become part of the build process. What you can do with this knowledge is only limited by your imagination … and your scripting skills.

But the general advantage of build rules versus external programs is that they are part of the project. So if they use standard commands they are portable to other developer’s machines. Perfect for Open Source projects.

This is a maintenance update. There was a bug in DTAboutViewController which caused a crash under iOS 5.

Change

• Fixed: Crash on iOS 5 when opening About Section

The update has been submitted to Apple for approval.

My iOS-related Podcast started out as “Dr. Touch Podcast” and later we rebranded ourselves to the more unique “Cocoanetics Podcast”. Though the topics always revolved around iOS development, Apple in general ans the people involved in our profession.

After an extended hiatus planning has begun to revive the podcast. I like to invite your comments, wishes and ideas as to what you would like to see – or more precisely hear – in a podcast that focusses on your daily bread and butter.

This impetus actually came from a composer by the name of Jamie Harper. A couple of weeks ago he contacted me offering his services. I thought to myself “well if I get new theme music for the podcast then I’ll relaunch it”.

I was kind of thinking that I could probably either not afford the composition or that this would take a long time to get done. Often you are dealing with people who vocally offer some great services only to disappoint you and never deliver.

But Mr. Harper delivered and he delivered very well. Not only did he compose a very nice intro jingle, but also made me a shorter and longer outro. And he did that at an exceptionally affordable rate. You’ll hear this music on the next show that I shall record.

Speaking of which … In the past I would always to a summary of iOS development news focussing on the kinds of news that would only half-heartedly be covered by the mainstream media which focusses on Apple products and the consumer end.

Sometimes I had the opportunity to interview industry leaders like Ortwin Gentz from Futuretap. Those podcast episodes where always good fun.

Finally I did speculate quite a bit on the future direction that Apple hints at with their actions. Crystal ball looking if you will. I like to talk and voice my opinion, that is one half why I want to do a podcast (because the established networks keep ignoring me).

The other reason for “why now?” is that there really is no dedicated iOS development news podcast. If you are interested in how great Marco Arment or his coffee is, then there’s a show for you. But nothing really covers stuff that I as a pro developer find interesting.

So here’s your chance to help shape the future of the Cocoanetics podcast. What are you interested in? What is niche enough to not be covered by others?

Also, let me know if you want to be on the show and get some free advertising for your apps and services. You get airtime if you let me ask you some in depth questions in an interview.

Your comments welcome, either via the public comments or if you don’t want anybody to know just drop me a mail.

There’s a category of developers – probably coming from other Linux-based desktops – who are feeling at home in the console (aka terminal). Especially when working with SCM systems like Git or Subversion you’re often faster committing or updating doing that via short commands as compared to Xcode.

Unfortunately Apple did not think to make the console a nice thing to look at from the get go. Let me show you quickly how you can geek up your terminal as well. Your colleagues will be impressed.

It’s really simple. You need to add these lines to your .profile which resides in your user home directory. If the file does not exist, create it.

export CLICOLOR=1
export LSCOLORS=Gxfxcxdxbxegedabagacad

The first line activates color support for the CLI (command line interface), the second line specifies which colors to use for certain kinds of things. When this is set every new console window will emit special ANSI sequences to set the colors.

You also need to enable these colors for your terminal. Make sure you have set your terminal emulation to xterm-color in Preferences – Advanced in the Terminal app. While you are here you can also set up some other things for how the Terminal should be styled. I prefer the “Pro” style which has a tinted glass background, sort of like a holographic display which you can see through.

The above settings will make your terminal look like this:

If these colors are not your style, don’t fret, Geoff Greer has build a cool online generator to piece together the LSCOLORS code. Just make your own color finger print and paste it into your .profile.

There’s more! Git also sports several color options:

git config color.branch auto
git config color.diff auto
git config color.interactive auto
git config color.status auto

… or you use the catch-all setting which includes all of the above:

git config color.ui true

This adds splashes of colors here and there, for examples a git diff:

Many other tools also support a color option, for example viewing a command’s man page is so much more interesting with a touch of color. If you have additional suggestions on how to make better use of color in terminal please don’t hesitate to comment below.

In this post I will to demonstrate how you can contribute to an Open Source project hosted on GitHub. I previously blogged about how you can make and apply patches, which is certainly one way. But on GitHub there is an even cooler method, one that is also less work on the project maintainer.

For the sake of this tutorial I shall pretend that there is an issue with a project by Matt Galloway, then I’ll fork his project. After fixing the issue I will push my changes online to my fork and finally notify Matt about these changes via a so-called “Pull Request”.

A prerequisite for this tutorial is to have your GitHub account and local git set up previously.

Step 1 – We have an Issue

Your first thought should be to document that in GitHub’s excellent bug tracking system. Each Open Source project there has a list of Issues (which are the bugs and feature requests). I found two things to improve in the project and so I added two issues.

We could hope for the project’s owner to fix that for us, but – being smart developers – we can also proceed to fix the issues ourselves. From the get go we won’t have writing access to the GitHub repository (unless the owner gives it to us, which is rare). You can see that this is the case if you only see the HTTP and Git Read-Only options on the repo URL:

The usual method is to make a copy of this project in our own area on GitHub where we CAN write to. This copy is what they call a “Fork”.

Step 2 – Stick a Fork in it

To make a fork you klick the “Fork” button at the top right of the project main page. After some “hardcore forking action” you end up on your own copy of the project. And there is now an extra option “SSH” telling you that you have read and write access to that.

A fork is a complete copy of the original project including all previous history. The reason for this that with Git every copy of a repository is always a full clone, so that each clone can also serve as a backup. This illustrates the distributed nature of Git, as opposed to centralized Source Code Management (SCM) systems like Subversion which has a central server.

Step 3 – Clone Wars

To work on our copy of the project we need to clone it to our hard disk. You can use the excellent official GitHub app but personally I prefer to work in terminal. I open terminal, go to a place where I want the copy of the project to be and issue the command:

git clone git@github.com:Cocoanetics/ios-library-with-resources.git

The URL used is the SSH one I copied from the web page. This results in a ios-library-with-resources folder appearing in the current working directory.

Step 4 – Fix-Tours

Now we can do our changes. So I open the Xcode project.

The first thing I am fixing for this example is to add the two sub-project products as dependency so that they get automatically built if somebody builds the containing app.

You see an M (for Modified) appear next to the root of the project tree which means that the project file has been modified. Each such fix should become one commit. So we right-click on MyLibraryTest, Source Control, Commit Selected Files. This opens a dialog that shows which files were modified and prompts for a commit message.

Since we fixed issue number #001 with that we also add this tag, because later we will see that GitHub conveniently links this commit with the issue via this number.

This committing basically saves the current state in your local Git repository (your clone) and adds your message to that. Actually it did a bit more than that, because Git has a an extra step called “staging” before committing.

In staging you select the modifications you want to be part of the commit by Adding them to the staging area. The second part then does this saving. If you do this via the Xcode interface you don’t need this staging phase because you specifically selected the files to commit already via the project navigator.

The second issue to fix is to change a build setting on the sub-project.

This time the M appears next to the MyLibrary.xcodeproj which really means the project.pbxproj file contained in the MyLibrary.xcodeproj bundle. (Remember that a bundle is a folder that looks like a file)

This time let’s do the committing on the command like, like a real pro. First let’s see the status.

git status
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.
#
# Changes not staged for commit:
#   (use "git add ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
#	modified:   MyLibrary/MyLibrary.xcodeproj/project.pbxproj
#
no changes added to commit (use "git add" and/or "git commit -a")

This shows that we are ahead of our fork (origin/master) by 1 commit, which is the previous one we did. And it shows that the project file for the sub-project was modified. We issue a “git commit -a” which stages all modified files for the commit and immediately prompts us for a commit message in the currently set up text editor.

There is a second shortcut that GitHub recognizes besides the #002 tag. If you write “closes” or “fixes” in front of the hashtag it not only references the issue, but also marks it as closed. How convenient!

“Set Skip Install YES for resource bundle to prevent copying that into Archive. closes #002″

I type my message into the VIM that opened and save/exit via ESC, :wq, ENTER.

Now we have two local commits (git log to see them), next we need to push them to our online repository. We just had modifications to the project file in these two examples, but – trust me – it works just as well for code changes.

Step 5 – Push Up

When we cloned the project to our local hard disk Git was nice enough to automatically configure the location where it came from as so called “Remote”, the default name is “origin”. You can have multiple remote places set up and choose which to push.

The command for us is:

git push origin master

“origin” is the remote I told you about, “master” means that you are pushing this to the master branch, which is the default branch.

To check if our changes have really arrived online I like to check the commits view on GitHub. And indeed they have.

We could end here and many people do. They just make a copy of an Open Source project for the case that the original repository disappears. But since we are playing nice we want to offer our improvements to the original project’s owner.

Step 6 – I just called to say “Please Pull Me!”

Just like for the fork we find a “Pull Request” button at the top right of the GitHub website of our fork. Push it and you get a nice overview of what is contained in the request.

There are some advanced options regarding milestones and assignments, but we’ll ignore these for now. You’ll notice that the pull request automatically generates a new number, #3 in our case. So you can not only reference issues with your commit messages, but the same thing is possible for pull requests.

The final step is not for us, but for the original project’s owner to perform.

Step 7 – Accepted!

As owner of a GitHub repo it is up to you whether you want to accept the changes into your repository. If you kept your commits small and – for code changes – added some good comments to aid understanding then the owner will be happy to accept them. It saves him from doing the work that you did and all he needs is to push the green merge button.

The button to accept the changes is only green if the merge of the code can be performed automatically. If the author did some changes on his own that conflict with yours then the button is gray stating that the changes cannot be merged automatically. In this unfortunate situation either one of you needs to pull the other’s changes and manually resolve the merging conflicts. But this is a topic for another day.

Conclusion

I suggest you find yourself a a GitHub project to contribute to and try out this process to shed the timid feeling. It really is quite idiot proof.

People put their projects on GitHub not only so that you get free source code to use in your apps. They also hope that other developers might find and fix bugs. And every once in a while you see a pull request that makes you think “OMG I’m not worthy!” because those are the gems that really teach you something.

And this is why they call it collaborative coding.

A month ago I was contacted by PacktPub with a “Review Request” and was provided with a ePub copy of the book for this exact purpose. PacktPub – which I had never heard of before – apparently is trying to get traction on the iOS developer market with a dozen books on the subject matter. But this pales in comparison to the hundreds of books they published for non-iOS ecosystems (Microsoft, Web, Java, etc.)

Is it just me or does it seem like more and more iOS developer are hoping to supplement their living from getting book royalties?

At the end of my procrastination I sat down and forced myself to read the first half of the book, taking notes to give this book a fair and balanced review. A word of caution: this will be a tragic comedy of epic proportions.

In Austria we have a saying: “Nothing is useless, it can always serve as a bad example”. It is my hope that this bad example serves to improve the overall quality of literature on our favorite subject matter.

The title of the work suggests that this book is not meant to be read in a linear fashion. You’d expect from a “Cookbook” to find “Recipes” for individual things that you would look up should you need to cook them. In the chapter about the book’s copyright you can read:

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied.

So far so good, but who is this book meant for? In the words of the author:

The iPhone Interface Cookbook is written from the ground up for people who are new to iOS or application interface design in general. Each chapter discusses the reasoning and design strategy behind critical interface components, as well as how to best integrate each into any iPhone or iPad application. Detailed and straight-forward recipes help give guidance and build understanding far beyond what is offered through Apple’s Human Interface Guidelines.

From these two parts of information – and possibly the short author’s bio – you come to expect a highly polished work that may sit on the shelf of a fledgling iOS developer getting his feet wet on his first couple of apps. A reference manual for design, if you will.

This puts me at odds with the premise. Neither am I fledgling, nor am I designer. I love to code and generally I like to delegate designing UI and UX to the companies that hire me. After having realized this I gave the book a second chance in my head: if it is worth the expense maybe I could recommend it to those designers sitting in my client’s companies for reference?

As an iOS contractor you often have to deal with companies that have established themselves on non-iOS platforms and now – due to public and marketing pressures – they also want an iOS app. Generally their in-house designers will know how to design a Windows app or website. Often I have wished for being able to recommend to them a guide as to what they have to do differently when designing for our platform.

This book isn’t that either.

Maybe I was “holding it wrong” because I read the book from the front cover to slightly more than half when I had to abort the mission. I’m detail oriented by nature and so I found boatloads of errors, formatting problems and blatant omissions with which I filled a note on Evernote with.

For example this is how e-mail addresses are formatted throughout. What do you think if you see this?

At first I attributed the problems I was seeing to the ePub format. Apparently the publisher did not invest in this being properly edited. Neither from a formatting point of view nor in terms of technical correctness.

Each chapter in the book slavishly follows this structure: Getting ready, How to do it…, How it works…, There’s more…, See also. The general style hides some interesting information in a tone ripe with fluff and half-truths.

And what’s even more annoying is that these mini-sections all take up space in the index which is destroying that as well. In this example you can see the worthless index. Why in god’s name would I want to directly jump to a There’s more in a section? You have no way of quickly getting an overview of what “Recipe” would be the right one for a real-life design case. Except wade through these superfluous entries of sub-sections.

I admitted that I am probably a) not the right person and b) not reading it correctly. But the problems with this book go far beyond the formatting.

Somehow I am getting the feeling that the individual chapters in this book might have been articles on a blog, that somebody just copy/pasted together and replaced all instances of “article” with “recipe” and “blog” with “book”. Is that truly the way how modern developers expect their information to be diced?

For one thing you’d expect from an experienced iOS developer to know how to spell the name of our IDE correctly. Instead Cameron (or is it a lawman editor?) insists on spelling it XCode. This book couldn’t have been edited on a Mac because even my WordPress autocorrected that to Xcode. Purists might call this a blatant disregard for what is holy.

There are many more examples of misspelled classes, apps or terms we iOS developers deal with every day. To give a second example: WebKit is one word, not two.

Ok, there are formatting problems (ePub’s fault), there are technical editing problems. To top it off there are factual problems as well.

The author details how to generated certificates on the provisioning portal, but omits the simple information that you can just connect your device via USB, click on the “User for Development” button in Xcode Organizer and have Xcode auto-provision the device in your “iOS Team Provisioning Profile”.

Then there is much commingling. Alert Views are thrown in together with modal view controllers. Later they are mashed up with push notifications and local notifications. And the rationalization for why everything is modal:  “iOS is a highly modal operating system, forcing one application at a time upon the user.”

There are 4 step guides where the second and third step are identical, just worded differently. There are in-depth explanations about topic that a designer has no interest in, and the interesting topics are covered incompletely. In truth I found two or three nuggets of information buried deep in what I read. For example I found the explanation on Fitt’s Law quite interesting, didn’t know that.

Sooner or later you realize that the statement from the Copyright section is a lie. “Every effort” was definitely not made and the information isn’t presented at an acceptable level of “accuracy”.

The second promise that the publisher did not keep is who the book is for. Where are the explanations “far beyond” of what you can find in Apple’s HIG? Where are the chapters that I can recommend to the UI/UX designer working at my client’s?

I had to stop reading half way through, when I had accumulated 3 screenfuls of “bugs”. I am not getting paid to correct this book, so that amount exceeded the level of bullish!t I was willing to put up with. My job is to evaluate whether this work of art is worthy of your attention.

By now you should fathom my opinion.

I chose this article’s title to try and grab your attention. Well, the product is still the same and does the same. The only difference is one that is under the hood. And as such it is your job – should you choose to accept it – to marvel at the benefits that the new old parsing engine brings us.

Ever since my friends at scribd showed me a first prototype for HTML parsing based on libxml2 I was – admittedly – jealous. This prototype basically worked by having libxml2 parse out the individual paragraphs of an article and then display each paragraph in its own cell of a UITableView. Back then I brushed off the suggestion and went with something I understood: NSScanner.

All the C-code necessary to deal with libxml2 seemed overly daunting, so I went with a simple basic structure for the parsing, something like this (pseudocode)

while (there_is_more)
{
   if (we_scanned_a_tag)
   {
      if (is_tag_a)
      {
         // a specific code
      }
      else if (is_tag_b)
      {
         //  b specific code
      }
      else if ...
   }
   else
   {
      // we must be inside a tag
 
      // deal with skipping over a tag that is incomplete, i.e. crap
 
      if (we_scanned_some_text_before_the_next_tag_open)
      {
         // append the text with correct formatting to the output
      }
   }
}

This structure had grown organically as I added support for additional tags and it went into the initWithHTML category method because that seemed to be the logical place. Little did I think ahead that this approach would preclude the possibility of having an event-based parser do callbacks into my code, because that would mean adding these event-handling methods to NSAttributedString.

Even the above pseudocode  is long, you can imagine how much of a Spaghetti the origin code became. Much of the problem didn’t actually come from all these tags, those where simply a big if-statement. Complexity came from having to deal with all these special cases where HTML might not be well-formed.

In my Open Source genstrings2 I saw how much faster pure C-code performs than NSScanner which also served to rekindle my wish to switch to libxml2 because this is also written in low-level highly optimized C.

I approached libxml2 in several steps:

1. Jealousy – “Boy wouldn’t it be great, but I’m afraid that this is out of my league”
2. Announce Intention – I wrote an issue on GitHub hoping for somebody to step forward
3. Do an Experiment and Document – I googled a bit and put together Part 1 of my libxml HTML tutorial.
4. Write a Wrapper – For Part 2 of my libxml HTML tutorial I wrote an Objective-C wrapper for libxml2.
5. Astonishment – The feeling you get when you find that you begin to understand the C-code needed
6. Benchmark – I removed all string building code and compared the raw parsing performance of both approaches
7. Transform the Pasta – Moved the code for building the attributed string into an aptly named class and have this driven from events generated by the new HTML parser.

The final step I called like this because if you break up Spaghetti code into several logical pieces and then layer these into several layers that’s a different kind of Pasta, that’s called Lasagne.

For step 6, the comparison I moved the NSScanner parsing loop into its own class and directly compared the running time on my iPhone 4S resulting in this tweet:

War&Peace HTML (3.4 MB), NSScanner: 4.264s, libxml2: 1.398s = 3x as fast on single thread, plus latter fixes HTML structure

At this stage it was clear to me that I need no extra self-convincing. So I went to work in a branch of the project. Most of the work was simply copy/pasting the attributed string building code into the right place, tag start, tag end or the characters found event method. This also allowed for omitting some workarounds that where needed to deal with non-well-formed HTML.

The second big BIG advantage of libxml2 is that its HTML parser fixes up the structure for you and also adds a missing html and body tag so that you end up with a perfect structure. It even adds a </br> right after a <br>. Even though this is completely unnecessary it still makes the HTML look like perfect XML. Much nicer to work with.

While I was doing the migrating new issues and pull requests with fixes stated to come in putting me a bit under stress because I needed to include the fixes in both branches. Which is why I decided to merge the branch back into master at the earliest possible time.

The initWithHTML method has shrunken to a much more manageable size:

- (id)initWithHTML:(NSData *)data options:(NSDictionary *)options documentAttributes:(NSDictionary **)dict
{
	// only with valid data
	if (![data length])
	{
 
		return nil;
	}
 
	DTHTMLAttributedStringBuilder	*stringBuilder = [[DTHTMLAttributedStringBuilder alloc] initWithHTML:data options:options documentAttributes:dict];
 
	[stringBuilder buildString];
 
	return [stringBuilder generatedAttributedString];
}

Benchmark after the Merge

And of course I did some further benchmarking on my iPhone 4S to compare the resulting speed increase. All tests where done by simply adding two NSLogs at the beginning and end of the initWithHTML method and subtracting the timestamps. Note that this is just the time for building the attributed string and does not include the layouting or drawing.

Demo HTML Snippet 

NSScanner: 50 ms

libxml2: 43 ms

War&Peace ePub HTML 

NSScanner: 10.968 sec

libxml2: 8.796 sec

This is an overall parsing and string building speed increase of between 14% and  22%.

You might now ask what happened to the 60% increase we saw from just comparing the parsers. The string building itself is still all happening on the same thread as the parsing and by itself has many opportunities for optimization.

For one thing the next thing I’d like to do is to put the string building operations onto its own GCD background queue. This way the events coming in from libxml would hand off to a queue running on a different thread and would immediately return to parsing. This could easily double the overall scanning performance because you can now make use of the two CPU cores present in the more modern iOS devices.

And there are obviously many more optimizations that now are feasible to do because you no longer face a daunting monster spaghetti but instead stand a chance of understanding what is happing in DTHTMLAttributedStringBuilder.

Conclusion

The main three advantages of libxml2 over NSScanner are: Performance, HTML non-wellformedness-resilience, simpler code through event-based handling of HTML.

This merge now makes it possible again for interested developers to contribute optimizations and new features because the code has become so much simpler to read and understand.

Here are my mental notes for today’s earnings call…

Peter Oppenheimer, CFO

“Highest Quarter Earnings and Revenue”, ”All Time Highs”

3-4 Weeks Mac channel inventory

MAS: 100 Mio Downloads in less than a year

128% YOY grown in iPhone, “tremendous popularity of iPhone 4S”. Target range 4-6 weeks of iPhone channel inventory

available in over 90 country

iPads increase 111%, accessories grew 99%. Within 4-6 weeks of iPad channel inventory.

With iBook Author anyone can create “iBook Textbooks, Cookbooks, picture books and more”

> 85 Million customers on iCloud

Half of Macs sold in stores are to New Mac users

“Easy Pay” checkout process key in handling the amount of buyers in store

They are actively discussing on what do do with their cash balance, but nothing to announce yet.

Extremely enthusiastic about their product pipeline.

Tim Cook, CEO – Q&A

iPhone grows at 3 times of the mobile phone market

Sold more iPhones than they expected, so called “short in channel inventory”. Caught up in some countries, but not all.

Component environment is and will stay favorable, except for hard disks (Thailand). Did get better prices for displays, NANDs, DRAM last quarter.

iPad: hugh opportunity for Apple, tablet market will be larger than PC market. “Limited function and eReaders” are a different category and thus not a thread. Who wants an iPads gets one. They’ll “continue to innovate like crazy in this area.”

Team at Apple “so incredible”. “We feel very good about where we are”

5 Reasons why this quarter will be earning less than last

1. Dec quarter had a 14th week = 1/14th of the Revenue, DOH
2. A week missing in March Quarter that is usually going well
3. Last year they produced more iPhones
4. “significant pent up demand for 4S”
5. Strengthening of Dollar against the Euro

About acquisitions: Apple generally believes in totally integrating acquired companies into the mothership. “You will be assimilated, resistance is futile” (well at least that’s what I thought to myself)

“There was not an obvious aspect on the numbers” regarding if the Kindle Fire had any positive or negative effect on iPad sales. iPad cannibalizes the Mac, but much more cannibalizes Windows PCs.

Mac has outgrown market for 20 quarters in a row. Seems like all of the data make it a very close race against Android, iPad is way ahead. Though it’s hard to get any “crisp numbers” on Android handset shipments.

“iOS is doing extremely well”. Prefers to ignore the “other horses” and focus on innovating.

And when asked about larger screens and LTE we got usual response.

The parsing of HTML necessary for my DTCoreText open source project is done entirely with the sophisticated use of NSScanner. But it has been long on my list to rewrite all of this parsing with a the industry standard libxml2 which comes preinstalled on all iOS devices. Not only is this potentially much faster in dealing with large chunks of HTML. It probably is also more intelligent in correcting structural errors of the HTML code you throw at it.

In part 1 of this series I showed you how to add the libxml2 library to your project and explained the basic concepts of using libxml2 for parsing any odd HTML snippet into a DOM. In this here part 2 we will create an Objective-C based wrapper around libml2 so that we can use it just like NSXMLParser, only for HTML.

There are basically two methods – approaches – to handling XML/HTML: DOM or SAX. DOM (short for Document Object Model) refers to building a tree of nodes and leaves which you then can query for information based on their path (with XPath). SAX (short for Simple API for XML) basically walks through the the document and sends you events as it discovers things. For example the begin of an element, the end of it, when a comment was found, etc.

NSXMLParser is a SAX parser for well-formed XML. That means you can use it only on XML, it cannot deal with the nature of HTML where some elements might not require that you close them. But if you look at the delegate protocol and compare it to the SAX interface of libxml2 you find an eerie resemblance. It might as well be that NSXMLParser is just a wrapper around the XML SAX interface of libxml2.

As we have previously learned we know that libxml2 also possesses a HTML parser which reuses most of the same internal data structures as its XML cousin. So we shall endeavor to create DTHTMLParser as a SAX parser for HTML.

Don’t worry about piecing together all the source code from this article, I will put that online in my DTFoundation project. Instead I want you to follow my lead and understand the pieces.

The libxml2 SAX Interface

The SAX interface works such that you register a handler function – in C – for each SAX event that you are interested in. This is done via a C-structure of type htmlSAXHandler which is nothing more than an aggregate of multiple C-function pointers.

In the libxml headers you often see some “html…” data structure typedef’d into one with “xml” as prefix. The reason for this is that it reuses whatever it can from the XML parser. But this also adds confusion – as I found through my experimenting – because some of the events that make sense for parsing XML don’t fire for HTML, like for example the one that reports CDATA elements. HTML does not have those, but the function pointer is still present in the htmlSAXHandler struct.

By CMD+Clicking on a type we can swim upstream to find the original definition of htmlSAXHandler:

htmlSAXHandler = xmlSAXHandler = _xmlSAXHandler =

struct _xmlSAXHandler {
    internalSubsetSAXFunc internalSubset;
    isStandaloneSAXFunc isStandalone;
    hasInternalSubsetSAXFunc hasInternalSubset;
    hasExternalSubsetSAXFunc hasExternalSubset;
    resolveEntitySAXFunc resolveEntity;
    getEntitySAXFunc getEntity;
    entityDeclSAXFunc entityDecl;
    notationDeclSAXFunc notationDecl;
    attributeDeclSAXFunc attributeDecl;
    elementDeclSAXFunc elementDecl;
    unparsedEntityDeclSAXFunc unparsedEntityDecl;
    setDocumentLocatorSAXFunc setDocumentLocator;
    startDocumentSAXFunc startDocument;
    endDocumentSAXFunc endDocument;
    startElementSAXFunc startElement;
    endElementSAXFunc endElement;
    referenceSAXFunc reference;
    charactersSAXFunc characters;
    ignorableWhitespaceSAXFunc ignorableWhitespace;
    processingInstructionSAXFunc processingInstruction;
    commentSAXFunc comment;
    warningSAXFunc warning;
    errorSAXFunc error;
    fatalErrorSAXFunc fatalError; /* unused error() get all the errors */
    getParameterEntitySAXFunc getParameterEntity;
    cdataBlockSAXFunc cdataBlock;
    externalSubsetSAXFunc externalSubset;
    unsigned int initialized;
    /* The following fields are extensions available only on version 2 */
    void *_private;
    startElementNsSAX2Func startElementNs;
    endElementNsSAX2Func endElementNs;
    xmlStructuredErrorFunc serror;
};

All these function pointer types are aptly named SomethingFunc and we’ll get to how these need to look like in a minute. The problem for us to solve how to bridge from our Objective-C paradigm (using self, properties, IVARs) into the C-function paradigm (no access to our parser object).

This problem is compounded by the fact that we are building this with ARC so that we don’t have to deal with memory management any more. Our resulting class will run on iOS 4 and above. Somehow we need to pass enough information to each event-function so that we can communicate with the parser object.

There are two reasonable possibilities: either we pass a reference to the parser instance or we pass a pointer to a struct that contains multiple values that we might need. I decided for the first because it seems cleaner to me.

libxml2 provides a concept of a user_data pointer that is passed to the event-functions together with other relevant information. We can use (abuse?) this for passing a reference to self. ARC requires this to be be cast to (__bridge void *) which in plain English means: “please treat this as just some typeless memory pointer and also don’t worry about retaining or releasing. I’ll take care of that, don’t panic.”

Like any self-respecting class we’ll start out with a good block of IVAR definitions and and init method.

@implementation DTHTMLParser
{
    htmlSAXHandler _handler;
 
    NSStringEncoding _encoding;
    NSData *_data;
 
    __unsafe_unretained id <DTHTMLParserDelegate> _delegate;
    htmlParserCtxtPtr _parserContext;
}

Notice that I’m defining the IVARs in the implementation, not the interface (in the header). This is possible as of using the new Apple LLVM compiler and provides for a nice way to hide the IVARs that we don’t want to be visible.

- (id)initWithData:(NSData *)data encoding:(NSStringEncoding)encoding
{
	self = [super init];
	if (self)
	{
		_data = data;
		_encoding = encoding;
 
		xmlSAX2InitHtmlDefaultSAXHandler(&_handler);
	}
 
	return self;
}

We save the data and the encoding into our IVARs, _data being a strong reference to it. That means that it retains it during the lifetime of our parser and will only release it when the parser goes away.

We also initialize the _handler structure via the xmlSAX2InitHtmlDefaultSAXHandler function. This sets up some initial internals and also chooses the SAX2 method for our work. There’s also an older SAX1 method, but we obviously prefer the newer one. This setup has to happen inside the init method because right after such an instance of a DTHTMLParser is create we want to be able to set the delegate which should take care of wiring up the needed functions.

Each event-function should have a corresponding delegate method in our DTHTMLParserDelegate protocol, but to avoid unnecessary function-calling we only add the functions to the handler for which the delegate actually implements the function.

#pragma mark Properties
 
- (void)setDelegate:(__unsafe_unretained id<DTHTMLParserDelegate>)delegate;
{
	if (delegate != _delegate)
	{
		_delegate = delegate;
 
		if ([_delegate respondsToSelector:@selector(parserDidStartDocument:)])
		{
			_handler.startDocument = _startDocument;
		}
		else
		{
			_handler.startDocument = NULL;
		}
 
		if ([_delegate respondsToSelector:@selector(parserDidEndDocument:)])
		{
			_handler.endDocument = _endDocument;
		}
		else
		{
			_handler.endDocument = NULL;
		}
// ...

This way when you set the delegate it is inspected one by one whether it responds to certain methods of the protocol. And whenever it does the corresponding function pointer is set in the _handler IVAR.

At the top of the file we need to define these functions. If we do just that then the compiler will warn us for all of these implementations that there was “no previous prototype”. In C a you call the first line of a C-function it’s prototype. So we just have an extra block for these.

#pragma mark Event function prototypes
 
void _startDocument(void *context);
void _endDocument(void *context);
void _startElement(void *context, const xmlChar *name,const xmlChar **atts);
void _endElement(void *context, const xmlChar *name);
void _characters(void *context, const xmlChar *ch, int len);
void _comment(void *context, const xmlChar *value);
void _error(void *context, const char *msg, ...);

Wherever you see a context parameter this will be the user_data pointer we will provide. And for example the _startElement function gets a name and an array of attributes as well. The xmlChar type is just a char. This is actually always provided as UTF8-encoded string regardless of what the data is encoded as.

Let’s look first at an easy event-function and then at a more complex one.

void _startDocument(void *context)
{
    DTHTMLParser *myself = (__bridge DTHTMLParser *)context;
 
    [myself.delegate parserDidStartDocument:myself];
}

You see how I cast the void pointer back into a pointer of type DTHTMLParser. I have to name the variable different than “self” because that is a reserved word. Then I can simply call the delegate method that corresponds to this event function.

By the way, these are the delegate methods we want to implement:

@class DTHTMLParser;
 
@protocol DTHTMLParserDelegate <NSObject>
 
@optional
- (void)parserDidStartDocument:(DTHTMLParser *)parser;
- (void)parserDidEndDocument:(DTHTMLParser *)parser;
- (void)parser:(DTHTMLParser *)parser didStartElement:(NSString *)elementName attributes:(NSDictionary *)attributeDict;
- (void)parser:(DTHTMLParser *)parser didEndElement:(NSString *)elementName;
- (void)parser:(DTHTMLParser *)parser foundCharacters:(NSString *)string;
- (void)parser:(DTHTMLParser *)parser foundComment:(NSString *)comment;
- (void)parser:(DTHTMLParser *)parser parseErrorOccurred:(NSError *)parseError;
 
@end

The @class is necessary before the protocol definition because a good delegate protocol also passes a reference to the caller. This also enables the casual reader of your code to know at a glance that this method belongs to a protocol connected to this class.

Now for something more complex, let’s look at didStartElement. The complexity of this method results from the arts parameter being alternating key, value, key, value etc.

void _startElement(void *context, const xmlChar *name,const xmlChar **atts)
{
	DTHTMLParser *myself = (__bridge DTHTMLParser *)context;
 
	NSString *nameStr = [NSString stringWithUTF8String:(char *)name];
 
	NSMutableDictionary *attributes = nil;
 
	if (atts)
	{
		NSString *key = nil;
		NSString *value = nil;
 
		attributes = [[NSMutableDictionary alloc] init];
 
		int i=0;
		while (1)
		{
			char *att = (char *)atts[i++];
 
			if (!key)
			{
				if (!att)
				{
					// we're done
					break;
				}
 
				key = [NSString stringWithUTF8String:att];
			}
			else
			{
				if (att)
				{
					value = [NSString stringWithUTF8String:att];
				}
				else
				{
					// solo attribute
					value = key;
				}
 
				[attributes setObject:value forKey:key];
 
				value = nil;
				key = nil;
			}
		}
	}
 
	[myself.delegate parser:myself didStartElement:nameStr attributes:attributes];
}

The while loop walks through the attributes and and when a NULL is encountered where a key would be then it breaks out of the loop. It is also necessary to deal with a situation where there is only an attribute name, but no value because HTML also allows that. Theoretically one attribute could also be occurring multiple times, but we ignore this. For later convenience we want the key/values for the attributes in an NSDictionary.

The final piece is how the parsing actually is started. For this we have a parse method, just like NSXMLParser.

- (BOOL)parse
{
	void *dataBytes = (char *)[_data bytes];
	int dataSize = [_data length];
 
	// detect encoding if necessary
	xmlCharEncoding charEnc = 0;
 
	if (!_encoding)
	{
		charEnc = xmlDetectCharEncoding(dataBytes, dataSize);
	}
	else
	{
		// convert the encoding
		// TODO: proper mapping from _encoding to xmlCharEncoding
		CFStringEncoding cfenc = CFStringConvertNSStringEncodingToEncoding(_encoding);
		CFStringRef cfencstr = CFStringConvertEncodingToIANACharSetName(cfenc);
		const char *enc = CFStringGetCStringPtr(cfencstr, 0);
 
		charEnc = xmlParseCharEncoding(enc);
	}
 
	// create a parse context
	_parserContext = htmlCreatePushParserCtxt(&_handler, (__bridge void *)self, 
		dataBytes, dataSize, NULL, charEnc);
 
	// set some options
	htmlCtxtUseOptions(_parserContext, HTML_PARSE_RECOVER | HTML_PARSE_NONET |
		HTML_PARSE_COMPACT);
 
	// parse!
	int result = htmlParseDocument(_parserContext);
 
	return (result==0);
}

There is a xmlDetectCharEncoding function in libxml to try and detect the encoding of the data which we invoke if the passed encoding is 0. The way to convert the NSStringEncoding into the xmlCharEncoding is only work shift, that needs some improving, possibly by simply having a big switch statement. Maybe later.

There are multiple ways how you can parse a document, some of which take individual parameters, some take a html parsing context. I settled on using the one for the push interface because there the htmlParseDocument function does not return a pointer to an html document tree (DOM), but instead a result that is 0 for success or -1 for error.

It is my understanding that this variant does not actually build the entire tree but keeps just enough state information to track the current hierarchy level inside the HTML tree. The push parser variant also has the ability to parse chunks of HTML data as they become available, e.g. while downloading. But exploring that is outside our scope for now.

Conclusion

Once your eyes have gotten used to the coding style of libxml2 you can start to appreciate the plethora of functions that it provides. You can also peruse the official reference manual online.

Finally I invite your collaboration on using and helping to polish DTHTMLParser if you have any need for parsing HTML text from within your iOS apps. Look for it on the GitHub DTFoundation project as soon as I can add a bit of AppleDoc-style documentation to it.

In my previous article about Sub-Projects in Xcode I showed you how you can have an Xcode project as a dependency. The advantage of this is that you can update your library from within the same project and debug into the sub-project’s code.

One thing that was not immediately obvious was how you deal with the problem that Xcode cannot find your sub-project’s library headers. Hence this writeup.

A header file generally informs the compiler what c-functions, global variables, Object-C methods and classes exist in an object file. All these are generally called “symbols”.

Build Process Review

The build process consists of these three steps:

1. Preprocessor – this takes care of including/importing header files into your source code, essentially pasting the content of the specified file right into your source code for building. This step also replaces all instances of #defined values with what you defined them to be.
2. Compiler – this goes through all the .m files and builds an object file with .o extension for each. Each item that is not coming from the same .m file is marked as an external symbol.
3. Linker – this merges all the object files into one big binary while at the same time linking the symbols with the actual implementations.

Header files do not actually contain executable code, but instead they tell the compiler which names and functions it should accept. It is the linker then that resolves these references. In short: If you want to use something which is implemented outside the current .m file than you need the appropriate import for that.

There are two kinds of imports:

• <path/header.h> with angle brackets – these are meant for system or “global” includes.
• “path/header.h” with double quotes – these are meant for items that are limited in scope to your current project

You have probably used both already, the one with the angle brackets for including headers that Apple provided. The one with the double quotes for accessing your own classes. Paths are usually relative paths because everybody has a different folder structure on his hard disk.

Get the Headers into Your App

The easiest way to get the headers into your project is to just add them. But you should generally refrain from duplicating your code for convenience, because then you have to perform an update on every copy of the file. A variation of this theme would be to not add the file itself, but just a reference. But this approach would also make your project dependent on your folder structure. Don’t do it, just don’t.

If you have followed the Sub-Projects guide you are already perfectly set up to get to the header while still keeping the project self-contained and independent from the file system layout. You only need to tell the outer Xcode project – probably your app that includes sub-projects for some static libraries – where it can find the necessary headers from the sub-project. Again you have multiple options: you can point it to the source code folder where the .h files are located. Or you can point it to the build output folder. Both are equally viable but if you work with static universal frameworks – as I do – then you can only use option two without losing your mind.

On my projects that contain reusable code I usually have multiple targets. One for a universal static framework – which is very convenient to add to a project because it automatically sets up the library and header paths for you when you drag it into your project. I have a second target for a static library because this allows for using the project as a sub-project and have the outer project link to the static library product. Because the framework conveniently bundles together multiple .h files you usually have the general name of the framework as path, i.e. you find <MobFox/MobFox.h> as the import of choice. For for example an Apple framework <Foundation/Foundation.h>.

Contrasting the MobFox/ path the actual location of the header file is PROJECT_ROOT/Core, so no MobFox path component there. This is the reason why we cannot point the containing app to the source folder because there is no header file MobFox.h inside a folder named MobFox. So via approach number 1 we would never find the header, we want to take door number two which is to point Xcode to the static library build output folder. But for this to work some setup is necessary.

Specifying Public Header Output Location

After building the static library Xcode will also copy certain headers to the output folder. A header can be public, project or private. Public means that it will be distributed together with the library. Project and Private won’t. You can set this for every header file individually. Headers for objects that you want to be able to see from your app should be Public.

If you now build the static library and look at the build log you will see that at the very end the public headers are copied somewhere:

We are astonished to find a path ending in Build/Products/Debug-iphoneos/usr/local/include for the two public header files. The reason for this being the default setting is that traditionally people would build binary files for the system they are working on. But that’s not what we want. So we go into the build settings for the static library target and adjust this. Don’t put this entry into either the Debug or Release lines, but into the line above these so that it may override both.

When we now build we see the output go to Build/Products/Debug-iphoneos/MobFox – as we want it. It is this extra MobFox path component that enables us to use the <MobFox/MobFox.h> style of including later on.

Now back to our app that is including this sub-project’s static library. There we now need to tell the compiler where it can find the above folder. It so happens that when building Xcode usually uses the same output folder for all products it needs to build, namely whatever is in the $(CONFIGURATION_BUILD_DIR) variable at the time. This in turn is set by default to $(BUILD_DIR)/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME).

The final piece of the puzzle to know is where to put that. This depends on the include style that I explained above. Double quoted includes need the path in “User Header Search Paths”. Angle-bracketed includes need the regular “Header Search Paths”. We are forced to use the latter due to the framework. We don’t need to check the Recursive box because at the given location there is a MobFox folder.

The <Multiple values> might cause you to pause for a second, but remember what I told you above. The configuration (Debug/Release) is part of this path and so even though you put the same variable in there, the actual result is a different one. Note that these values are bold because the target settings override the settings specified by the project. You can revert to the project settings by selecting the line and hitting CMD+Backspace.

There is an alternate option to put this path in the “User Header Search Paths” instead, but then you need to set “Always Search User Paths” to Yes. This will cause the compiler to also search the user header paths for angle bracket includes. Though personally I prefer the other option that needs less settings.

The other steps for building are explained in the Sub-Projects posts. If you carry them out (link with target), you will find that now your app finds the headers and successfully builds.

Conclusion

If Xcode complains that it cannot find a header then you should make sure that it is indeed in a place where it can find it. There is an abundance of default settings that may seem daunting, but if you know what to look for you can quickly find why your header cannot be found.

I wrote this article because I myself was stumped by not being able to explain why a project couldn’t find the MobFox header. So another learning of this episode is that you should not assume that the owner of your sub-project has set everything up correctly. In my case I too was missing the custom header output path setting. Errare humanum est.

Besides fixing several bugs reported by users this release focusses on implementing a new custom-built super-charged strings scanner.

Changes

• FIXED: missing strings file no longer aborts import
• FIXED: added – Button for deleting tokens
• FIXED: deletion of wrong row
• FIXED: over-escaping of \ on writing strings files
• FIXED: wrong token name updated on enter key
• FIXED: search filter not reset on adding a new token
• FIXED: Previous/Next buttons in wizard cease to function
• NEW: replaced genstrings with high-speed custom implementation (>20x speed improvement)
• NEW: support custom localization macro prefix

This new version does no longer support 32-bit because the new strings scanner was implemented using ARC. We hope that this does not inconvenience any of our existing uses. But then again, serious developers are on a 64 bit machine anyway since Snow Leopard…

The new version has been submitted to Apple today.

Over the past week I’ve been working on DTLocalizableStringScanner or in short genstrings2. The original genstrings dates back to the NeXtStep days. You know how it is “never change a winning team” BUT “the good is the enemy of the great”, because if something kind of works, why change it?

Besides of the other problems I’ve alluded to in my previous article genstrings is very slow. Internally it is written in Objective-C as you can tell from the occasional stack trace when it crashes again. But it was created in a time when Macs did only have single CPU cores and when we did not have the awesome LLVM with ARC, GCD and multi-threading.

At the time of this writing we implemented genstrings2 twice: once based extensively on NSScanner which I became intimately acquainted with when working on DTCoreText. The other is a rewrite of the rewrite totally ditching NSScanner and doing all scanning directly on the individual characters.

My feeling is that the lower-level scanning will be orders of magnitude faster and at the same time it is way more robust because for non-literal macro parameters it knows how to deal with pairs of parenthesis and commas contained in strings. But this just begs to be benchmarked.

In order to have a sufficient amount of code to scan through I downloaded the latest source bundle for Adium. Getting a time for the individual versions is easy with the time command:

time genstrings -q -s AILocalizedString ~/Desktop/adium-1.4.4/Source/*.m
 
real     0m7.872s
user     0m6.848s
sys      0m0.053s

When benchmarking you also need to mention the type of bench you are marking. These numbers come from my 2010 MacBook Air which is somewhat skewed because the disk IO times are possibly faster than usual. But we are mostly interested in the parsing efficiency. Here the benchmark machine has 2 CPU cores (1.6 GHz Intel Core 2 Duo).

You can see in activity monitor that original genstrings is single-threaded. It uses up to 100% of a single core and shows to have only 1 thread.

The version of genstrings2 using NSScanner clocked in much better. 1.7 seconds down from 7.9 seconds, 5 times as fast.

real     0m1.663s
user     0m2.950s
sys      0m0.070s

Now for the turbo version that does all the scanning without NSScanner. Instead it is working with the individual unichar characters that make up the strings. On iOS these are 16 bit because internally all NSStrings are using UTF16.

real     0m2.167s
user     0m3.973s
sys      0m0.074s

Whoa! Hold your horses, why is that slower than my version? My first thought was that maybe NSScanner really is very efficient. But once I fired up Instruments I found this, lots of CFRelease calls were bogging down the turbo scanner.

Looking at the sample distribution in _scanMacro revealed the culprit.

From this it was quite obvious that the creation of the mutable array to hold the scanned macro parameters was taking most of the CPU time in this area of the code. Turns out this was allocating and releasing this parameter array way more than necessary. We only need to created that once we are sure that this is actually one of the macros we are looking for, not some random word that happens to be made up of macro name characters.

So moving the array creation a few lines down and changing it to an alloc/initWithCapacity (we know that we probably will have less than 10 parameters) yielded this result in the benchmark:

real     0m1.385s
user     0m2.045s
sys      0m0.081s

Looking at Instruments again we see even more potential for optimization. If a line takes a quarter of the CPU time spent in a method then you have to ask yourself if there is a way to get rid of this work. Now this validMacroCharacters is already supposed to be a lazy initialization, so how to optimize this?

While it is lazy there is still quite some overhead to do the method call. On the first time around the character set is initialized and on subsequent calls the method returns the initialized IVAR. We can easily get around this by moving the initialization of the IVAR to the class init and reference it directly.

Normally this method call plus if plus returning the IVAR would not be a problem, but _scanMacro is called thousands of times and so – as proven by Instruments – this is an easy optimization. The benchmark improvement is very impressive:

real	0m1.065s
user	0m1.839s
sys	0m0.059s

At this stage the turbo scanner (with two changes) is already almost twice as fast as the NSScanner-version and 7 times as fast as original genstrings. BUT three is a charm and it so happens that I had a stroke of genius.

My train of thought went: if I can defer CPU-intensive work to the latest moment possible and if I can reduce the amount of times when it is necessary, what other method could I come up with to skip out of _scanMacro early? After the previous optimizations most CPU time was distributed between creating an NSString object and the containsObject of the macro dictionary. Two items that cannot be easily optimized further.

Upon closer inspection I found that far too many times words were found that were made up of characters in the macro words. This resulted in these two statements being called way more often than necessary, even for words that are not plausible macro names because they are too short. On the NSScanner version I had had the optimization to ignore short commands like “if” and “do” because these would look like macro characters but are in reality reserved words. But this did not yield and advantage.

The stroke of genius was to find the lengths of the shortest and longest macro names (also in the init) and then have a simple if comparison to return from _scanMacro if we character sequence is far too short for one of our macro names. The results blew me away.

real	0m0.442s
user	0m0.691s
sys	0m0.044s

Now we have truly reached the point of impossible further optimization. Looking in Instruments the CPU times are now much wider spread, only characterIsMember cumulates more CPU times then the other lines, but this method cannot be optimized further since that would mean writing a replacement for NSCharacterSet.

Conclusion

Let’s summarize the lessons learned:

• Good C-code easily outperforms NSScanner
• Defer creating objects to the latest possible moment
• Avoid lazy instantiations if they are called in tight loops that cause them to be called too often
• Try to find simple mathematical or logical abort conditions (quickly processed) to avoid CPU-intense work where possible

Granted I could spend more effort on working out the performance problems the NSScanner version has, but I doubt that I could get anywhere near the double turbo version. The final results thus are:

• genstrings: 7.9 sec
• genstrings (NSScanner): 1.7 sec
• genstrings (optimized turbo): 0.4 sec

The optimized turbo version is 20 times as fast as the original, let’s be content with that.

LEGO’s Life of George game brings together 5 worlds like nothing else before:

• 8-Bit Design
• iPhone
• LEGO bricks
• manual interaction, a “touch interface” if you will
• computer vision for gamification

Let’s have a quick look at this fun pastime that is all the rage.

The LEGO Life of George app is a free download on the app store. To play you need the box which contains some additional ingredients you need to play. You get 6×1, 8×2, 4×3, 4×4 and 2×6 pieces in red, green, blue, black, yellow, white. Any self-respecting LEGOist probably has these at home. Then there is a piece of cardboard that has a grid of dots which aide in recognizing the shapes you put together. Plus there is a very brief Getting Started guide and a sticker showing George in all his 8-bit glory.

The product is smartly priced at a level where you probably prefer the convenience of having it all in a box over pirating the pieces from your kids and using a scan of the mat. Also this level of innovativeness needs to be rewarded, so I purchased the original product at my Apple Store. And so should you: if not for yourself then for all the LEGO+iPhone fans around you.

You select a model from initially 130 models and your goal is to complete these in as little time as possible. So you select a model, build that from the included pieces, drop it onto the grid, tap the cam button on the iPhone app which displays the model and countdown. Then you position the iPhone such that it sees the entire grid. It does not matter where on the grid you place the finished model.

When playing you should make sure that you have sufficient space to lay out the pieces for faster access. Also the bricks have a tendency to fall to the floor if permitted. The models are rated in 12 difficulty levels.

The most fun, but also the biggest weakness of this game is Multiplayer. There is no support for Game Center and Online competition. Instead you have to make due with pass’n'play mode. Players alternate building different models, because it would be useless to be building exactly the same model if the person before you has already found all the pieces. Though there seems to be a balancing glitch: On one of my multi-player games I was getting a very difficult model while my opponent got something extremely simple. Unfair!

The other problem is that on several occasions the model was not recognized properly causing no points to be awarded. Instead you just get a display with tips to improve recognition, like tilting to avoid shine or making sure that the entire mat is visible. But no option to retake the picture. Annoying!

LEGO plans to sell us additional models for certain holidays, the first one being the Christmas pack for $2 which contains “3 new levels of Christmas content with 30 new Challenges and 45 models”. Don’t ask me how this works out mathematically and what the difference between challenge and model is. That the price is still $2 even after the turn of the year reminds us that Apple still does not permit In-App-Purchases for free.

Finally – what might be even be the coolest feature of all – you also get to make your own models. I tried making my Cocoanetics logo here:

The image recognition software tries to match the contained pieces with what it “sees” on the mat. But the pieces need to be upright. We build a ciqua logo, but with the pips pointing to the side and that was not properly recognized.  The software cannot differentiate between two adjacent pieces of the same color. If you please a 2 and a 1 next to each other, it is recognized as a 3. So far the models the app tries out all the variations that are possible and seems to always try to take the longest bricks possible.

Their low-cost technology circumvents the problem you have with normal computer vision of having to adjust for a wide variety of image quality by simplifying the shapes it has to recognize, little squares in primary colors, and by using this grid mat for calibration. It’s is orders of magnitude more simple to recognize and align a grid of primary color blocks versus having to do edge and shape detection. This is an smart approach makes the technology robust enough for use by everybody. If you are used to buying LEGO bricks then you also have no issue with using this cardboard thing. So this partnership turns out to be a very nice fit between LEGO and Eye Cue.

LEGO is smart to exploring this unknown territory between digital and physical, between virtual and real. Normal apps which are digital offer much less satisfying value to the player than LEGO can.

If what I am hearing is any indication then LEGO has a hit. What’s next? Maybe an app to remote control LEGO robots?

Actually I am on vacation, but I couldn’t help myself using the breather to work on a little hobby project. This I shall reveal to you today!

If you localize your apps (iOS or Mac regardless) you are probably used to working with genstrings. Probably painfully working, which is why we built the Mac app Linguan which remote-controls genstrings and merges the results with your previously existing tokens.

But genstrings has several big problems.

The most pressing problem of using genstrings from Linguan is that it is very picky about the syntax of the string macros. If you have a parameter for the macro that is not a literal @”string” then it will crash. Several Linguan users have been stumped by this behavior because this makes it impossible for Linguan to scan their source code.

The second problem is that we don’t have access to genstrings source code, and so we neither cannot know what this tool really does, nor can we modernize it to make use of modern multi-processor systems. genstrings is single-threaded.

The third problem is that before now the only way how we could call genstrings from within Linguan is basically open an invisible shell and then read the output files from some hidden temporary output folder. You get genstrings with Xcode, but what if you don’t want to install Xcode on the machine you want to use with Linguan? Having to spawn a process for each source file individually has also proven to be impracticable because instead of a 0.3 seconds this would take 12 seconds for scanning just the Linguan source files.

If that is any indication then the genstrings man page has not been updated since May 7, 2007. Any substantial improvements to the tool are probably even older than this, somebody from Apple told me that this goes back as long as the NeXt days.

My goal was to reverse-engineer the functionality of genstrings, put that into a static library and thus be able to bring native high-performance string scanning to Linguan without the need for genstrings to be present any more.

My latest Open Source project DTLocalizableStringScanner (on GitHub) contains two targets: one for a static library which we can use for scanning the source files. The other is a re-implementation of a command line tool which can take the place of genstrings.

genstrings2 is built around NSScanner for the scanning and GCD queues for multi-threading. There might be faster ways to scan for the macros but there is no reason to do that at the expense of the code being easy to read. I’d rather fill up a Grand Central Dispatch queue with one block per source file and thus make full use of multiple CPU cores. It works on as many files in parallel as GCD permits.

You can help hone this into a worthy successor for genstrings by testing it on your own source code and telling me about edge cases that it does not handle properly. For one thing it should ignore invalid macros instead of crashing like the original, that’s one immediate advantage that you get from symbolic-linking genstrings2 to be used instead of genstrings.

There is one (undocumented) behavior that is not implemented yet. genstrings apparently expands tokens containing lists in the format [one, two, three] into three lines with [one], [two] and [three]. People would use this to localizes NSPredicateEditor. But that’s an easy exercise that I will do in the next few days.

If there are no show stoppers then DTLocalizableStringScanner will replace genstrings already in Linguan 1.0.3.

Imagine me getting a mani-pedi during the Christmas holidays. No, really! We have a lady come to our house regularly and doing all willing feet.

Somehow our casual conversion came to the iPhone. Now imagine this healthcare professional telling me:

“For me the iPhone is no option because it does not work with the gloves we use”.

One does not have to say THAT to me twice. This myth was begging to be busted.

First thing I tried out was one of the original Latex gloves. There was almost no difference in the ability of my iPhone and iPad detecting touches.

Can you imagine how belief-shattering this was when I showed it off to my witnesses? ”Oh my, then I had been told a lie!”

When I mentioned this finding on Twitter one of the responses was:

They’re probably thin enough. Regular gloves don’t work, unless they’re specially prepared.

Thicker. Hm, where do I get thicker gloves? Ah, my wife got these cleaning gloves made from rubber. Those only worked like 80%, you had to push slightly more than usual and also their rubbery surface made sliding over the display a bit awkward. But it still worked well enough for emergencies.

So thickness does not play a role either as this step 2 in our experiment showed.

John Blanchard told me:

“I recently noticed I can use my iPhone through my motorcycle gloves. Didn’t expect that!”

And here is picture proof of him using his iPhone with his real leather Stryker Gloves. They don’t even mention “iPhone-compatible” in the product description. Maybe they should!

I asked for other people to contribute photos of them using their iPhones with gloves. Hermano Queiroz was the second person to present photo proof of his gloves working with his iPhone:

Haha, very funny Hermano! We all know that you’re cheating on this photo. Clearly these are your wife’s fingers, not yours.

But Hermano also had this picture proof. He need 3 tries for that. On the first you could only see the screen showing my original tweet, but not the material of the gloves. The second only showed a blurry screen. The HDR setting finally got the best of both worlds.

Finally the coolest entry in the “show your gloves contest” did not involve gloves at all. Timothee Boucher mentioned:

I’m sure it would work with a Ziplock™ bag. I used that to still listen to and control Pandora while painting.

It just so happens that I hav such a bag at home, who doesn’t? I have been using it to waterproof my iPhone while walking the dog while it rains. In Europe these are sold by Toppits.

Here you see me aceing the final test:

If we learned anything from this exercise then it is that our iOS devices are even more amazing than we previously realized.

It is true that most material that are used for gloves won’t work. I tested wool and rough leather to no avail. But smooth leather has been proven to work on John’s gloves. Apparently there are a couple of elusive factors at work that are more complex than simply material or thickness.

This post would not be complete if I didn’t mention that – of course! – there are winter gloves with specially coated finger tips. These guys in Michigan have compiled an extensive list: Best iPhone, iPad & Android Touchscreen Gloves.

People just love to use their iDevices with gloves.

Sometimes you want to tell somebody how you fixed a problem in their code, but for some reason the code is not on github so you cannot send them a pull request. If you felt really smart then you might put the changes you made into an e-mail, like “in file1.m:102 you change it to x, in file2.m:54 you make change y”.

Thought this doesn’t really help the developer you are trying to help. Even when following your change instructions to the letter it is a tedious and error-prone method of applying your changes.

Thanks to M. Douglas McIlroy, Adjunct Professor at Dartmouth Colleague – one of the early Unix pioneers – there is a better way. He invented the form of diff that we are using today to get the Difference between two versions of a file. And Larry Wall who invented the patch command which can take a diff file and effectively apply it to files.

Honestly I was afraid of using patch before, never having played with it. What you don’t know you fear…. but since you are reading this article I now finally did, so that I can explain its usage.

Making a Patch

Let’s try with a simple text file first.

file1.txt

Here's some text.
And a second line.
And a third line.
A fourth line.

file1_changed.txt

Here's some text.
And a third line.
A fourth line changed.

To get the difference between these files we just diff them:

diff file1.txt file1_changed.txt

This produces this output:

2d1
< And a second line.
4c3
< A fourth line. --- > A fourth line changed.

We see that diff features each changed block with a sort of address. We removed line two, so we see that the d in 2d1 probably means deleted. And the c in 4c3 probably means changed. If a line was changed you first see the original line prefixed by a less-than, then three dashes and then the changed line prefixed by a greater-than. diff also has a -y flag that puts the changes side-by-side, which more clearly visualizes the changes.

This kind of output is called the “normal” format. To be able to use the diff for patching this is missing some vital information, namely which file the changes belong to. For this purpose you have to use the “unified” format which you get with the -u flag. We also pipe the output into a new file.

diff -u file1.txt file1_changed.txt > patch.diff

This results in the patch file to have this content:

--- file1.txt	2011-12-25 12:17:30.000000000 +0100
+++ file1_changed.txt	2011-12-25 12:01:30.000000000 +0100
@@ -1,4 +1,3 @@
 Here's some text.
-And a second line.
 And a third line.
-A fourth line.
+A fourth line changed.

You can see that has information about which files are involved. Also you have minus and plus prefixes. A minus with a plus following means that the line was changed. Just a minus means that the line was only present in the original, i.e. deleted. Just a plus means that line was only present in the modified version, i.e. inserted.

Let’s Patch

Appling the patch we just created is very simple. If the current working directory contains both the patch and the to-be-patched file all we need is:

patch < patch.diff

You get an output telling you while files where touched. As soon as you have applied a patch, calling the patch command again with the same patch will get you a message about this giving you the option to ignore individual “hunks” which is what these blocks of changes are called.

Multiple Files

For multiple files the process is only slightly more complex. Say you have a a folder “original” and a folder “changed” and the latter contains multiple changed versions of the files in the former.

If you call diff passing a folder name instead of a file name then this causes diff to compare file1 in one folder against file1 in the second folder, exactly what we need. There are a couple of additional flags for diff that are of use in this case:

• -r to recurse through all subfolders of the passed folders
• -u to get the “unified format” output
• -p prints the c-function a change was contained in
• -N treats absent files as empty

So we go into the folder where our “original” and “changed” folders are located in and do:

diff -rupN original changed > original.patch

With this patch file it is easiest if you just move into the root of the original and copy the patch there. Then you can apply it without any additional parameters. Even though the diff contains the original and changed path prefixes it figures out that you must have meant the file contained in the current working directory.

cd original
cp ../original.patch .
patch < original.patch

There is one parameter of the patch command that you might need if the paths mentioned in the diff refer to a different folder structure than you have on your own machine. The man page of patch has this to say about -p:

-pnum or –strip=num
Strip the smallest prefix containing num leading slashes from each file name found in the patch file. A sequence of one or more adjacent slashes is counted as a single slash. This controls how file names found in the patch file are treated, in case you keep your files in a different directory than the person who sent out the patch. For example, supposing the file name in the patch file was

/u/howard/src/blurfl/blurfl.c

setting -p0 gives the entire file name unmodified, -p1 gives

u/howard/src/blurfl/blurfl.c

without the leading slash, -p4 gives

blurfl/blurfl.c

and not specifying -p at all just gives you blurfl.c. Whatever you end up with is looked for either in the current directory, or the directory specified by the -d option.

When I saw someone use patch for the first time he totally confused me by claiming that -p is short for the “patch level” which – as you can see above – is utter nonsense. It is probably short for prefix and the long form of it –strip actually means something entirely different.

It is essentially the number of slashes that are to be ignored, including the part of the path to the left of each ignored slash. A strip level of 0 does not modify the paths at all. A level of 1 ignores the path up to and including the first slash. Omitting the p parameter causes patch to ignore the complete paths and only look at the file names.

If a patch without specified strip level doesn’t work then you will have to take a look at the patch file and figure out how much of the paths you need to ignore so that these paths fit with the situation on your own file system. In our example above (with the original and changed folders) we could have used -p1 to have patch ignore the relative paths, but it seems to do that by itself.

Creating Patches from Repositories

Above we discussed one way to create diffs that can be used for patches by having two versions of your files in two folders and diff these two. This typically suites those people who have not yet warmed to the use of source code management systems (SCM) like git or Subversion.

In an SCM you don’t have multiple copies of each file, instead you have only one current version of the source code in the project directory and all previous versions are somehow hidden from view, but can be retrieved. In git this versions are called “commits” in Subversion “revisions”.

For the sake of trying it out, let’s create an empty Xcode project with a git repo. On the dialog where you specify the folder for the project you can set a checkbox to have Xcode also put it under version control.

It is probably an oversight in Xcode but if you create the project like this the first commit is being done for you with comment “Initial Commit”, however there are changes to the project file adding a “lastKnownFileType” to the project.pbxproj – which is contained in the xcodeproj bundle – that causes the project file to show as modified if you do “git status”.

To get to a clean starting state for the project I go to the folder where I created the project and do an additional commit.

cd ~/scmtest
git status
git commit -a
git log

I have vi set up as my editor so on the file that opens from the commit I enter some comment and then ESC : w q to save and finish the commit. The git log now shows no uncommitted changes, two commits and two untracked files containing some user workspace settings which you probably don’t need or want in your repo.

Now let’s modify some code. For this test I just added an NSLog into the app delegate file. What’s really interesting now is how to create a patch that contains the changes from our second commit to the current modified state of our working directory. Simple, really:

git diff --no-prefix > diff.patch

We need the option to omit the prefix because otherwise git diff adds an “a” and a “b” prefix in the paths which never correspond to actual paths. Of course you can also apply patches that still contain the prefixes if you use the -p1 strip level.

Our patch now looks like this:

diff --git scmtest/DTAppDelegate.m scmtest/DTAppDelegate.m
index 7730864..795fa6d 100644
--- scmtest/DTAppDelegate.m
+++ scmtest/DTAppDelegate.m
@@ -20,6 +20,8 @@

 - (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
 {
+    NSLog(@"Hello World");
+
     self.window = [[[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]] autorelease];
     // Override point for customization after application launch.
     self.window.backgroundColor = [UIColor whiteColor];

You can see the inserted lines prefixed with plus, also there are a couple of lines of code before and after the insertion so that patch can find the place even when there have been other modifications to the file that changed the line numbers.

A simple git diff always contains all modifications made on the latest commit. To test this patch we discard our changes and see that we ended up on the second commit.

git reset --hard
git log

Then we apply the patch, we need a strip level of zero so that the relative paths are used as they are.

patch -p0 < diff.patch
git status

After this patch we see that the changes appeared again.

You can also create a patch between any two commits by specifying the commit identifiers as parameters. You can get the identifiers from git log.

git log
git diff 92cef602e685970e6d4316cd383217d39d169b89 ec24c6d3435a2ae9ddaabd76f958561e86f9f02e > b.patch

Finally let’s also see how the same can be gotten from an SVN repo. Same technique:

svn status
svn diff > diff.patch

Between two specific revisions the same is possible. Subversion numbers revisions incrementally instead of with identifiers as git does. So the command for the changes from a specific version to the next is this:

svn diff -r 126:127 > c.patch

Contrary to git the diff command for svn does not prefix the file paths, instead the revision info is contained in brackets after the files which is ignored by patch.

Conclusion

Knowing how to create and apply patches is a great skill to have as a programmer. So the next time you find that you want to precisely communicate what changes you made in somebody else’s code you should send them a patch instead of an email containing line numbers and code fragments.

And if they don’t know what to do with that, also send them a link to this here tutorial.

PS: AlBlue points out that on git your can also directly send patches via e-mail. Those even include your commit messages and identity.

Robert Meraner asks:

“Do you have any idea why this code works in iOS 4.3, but no longer under iOS 5? Googling it seems to turn up some ideas, but no immediate explanation.”

This is the code Robert refers to:

NSString *currentElementValue = @"01.12.2011 09:35:13 CET";
NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
[dateFormatter setDateFormat:@"dd.MM.yyyy HH:mm:ss zzz"];
NSDate *date = [dateFormatter dateFromString:currentElementValue];

This got me confounded initially as well, but thanks to Cédric Luthi we got an official answer to this riddle: works as intended!

Trying out the above mentioned code you indeed find that date is correctly parsed on the 4.3 iPhone Simulator. If you simulate it on the 5.0 iPhone Simulator then you get a mysterious nil.

NSDateFormatter has an NSLocale property to specify how text is output or – if you use it for parsing – is to be interpreted. So my first instinct was to try different locales because of the CET time zone abbreviation.

Lo and behold, using en_GB did the trick.

NSLocale *enLocale = [[NSLocale alloc] initWithLocaleIdentifier:@"en-GB"];
dateFormatter.locale = enLocale;
[enLocale release];

I also checked which locale was used if you didn’t set the property and found that it defaulted to en-US. So apparently there was a change for this locale in iOS 5.

Cédric had exactly the same problem and raised it with Apple. He was so kind as to share his Radar as well as the detailed response he got from an Apple engineer (emphasis mine):

Engineering has determined that this issue behaves as intended based on the following information:

This is an intentional change in iOS 5. The issue is this: With the short formats as specified by z (=zzz) or v (=vvv), there can be a lot of ambiguity. For example, “ET” for Eastern Time” could apply to different time zones in many different regions. To improve formatting and parsing reliability, the short forms are only used in a locale if the “cu” (commonly used) flag is set for the locale. Otherwise, only the long forms are used (for both formatting and parsing). This is a change in open-source CLDR 2.0 / ICU 4.8, which is the basis for the ICU in iOS 5, which in turn is the basis of NSDateFormatter behavior.

For the “en” locale (= “en_US”), the cu flag is set for metazones such as Alaska, America_Central, America_Eastern, America_Mountain, America_Pacific, Atlantic, Hawaii_Aleutian, and GMT. It is not set for Europe_Central.

However, for the “en_GB” locale, the cu flag is set for Europe_Central.

So, a formatter set for short timezone style “z” or “zzz” and locale “en” or “en_US” will not parse “CEST” or “CET”, but if the locale is instead set to “en_GB” it will parse those. The “GMT” style will be parsed by all.

If the formatter is set for the long timezone style “zzzz”, and the locale is any of “en”, “en_US”, or “en_GB”, then any of the following will be parsed, because they are unambiguous:

“Pacific Daylight Time” “Central European Summer Time” “Central European Time”

The mentioned ICU change (International Components for Unicode) is quite a mouthful, so we can be thankful that Apple is on top of what these Open Source guys are deciding and their engineers are merging all these locale fixes into our favorite operating system.

In this special case we learn that it is not wise to use time zone abbreviations in server-generated XML. Better to have all dates use UTC with time zone “Z”. Alternatively you could use the UTC offset. This is never ambiguous.

If you have no say in what comes from the server then the officially recommended locale that understands CET and CEST would be en_GB. You should always use the same locale to decode a date string as was used to encode it. The necessity for this would also be present if month names are spelled out or abbreviated in the source string.

In short: Thou shalt not rely on the default locale to understand your date strings.

Apple owns patent US5946647 and slapped that around HTC’s team of lawyers successfully. Apple has previously sued HTC over them infringing 4 of their patents seeking to block sales of several Android-based mobile phones that HTC is making. On December 19th the International Trade Commission published their final determination as to the validity of two claims in this patent while finding no infringement on the others.

This result is interesting for us iOS developers for several reasons. For one it shows that something we are taking for granted was actually patented by Apple. They filed it on February 1st, 1996 and the patent was granted 3 years later on August 31, 1999. Usually patents are very elusive and it is generally hard to show how a device or operating system really infringes upon them. But in this rare specimen Apple has the rights on something that you probably see every day.

The second thing that fascinates me that in all the legalese that you find in the patent and ruling documents you can actually begin to see the outline of this mentioned technology if you know what you are looking for and know where to look. The final determination specifically mentions claims 1 and 8 of said patent. So let’s first have a look at what the patent is about and then share the experience of actually seeing that Apple is right in asserting their ownership.

Claim 1 is:

1. A computer-based system for detecting structures in data and performing actions on detected structures, comprising:

• an input device for receiving data;
• an output device for presenting the data;
• a memory storing information including program routines including
• an analyzer server for detecting structures in the data, and for linking actions to the detected structures;
• a user interface enabling the selection of a detected structure and a linked action; and
• an action processor for performing the selected action linked to the selected structure; and
• a processing unit coupled to the input device, the output device, and the memory for controlling the execution of the program routines.

Got that so far? There is some data that has has some structures in it and somehow actions are linked to that. Don’t worry just yet, it will be clear soon.

Claim 8 is much simpler:

8. The system recited in claim 1, wherein the user interface highlights detected structures.

Now the bell should be ringing! Let me visualize it for you:

Here the data is the plain text of the email, and the detected structures are an appointment (formulated in natural language) and a domain name. Both had been detected by iOS (a “computer-based system”) and turned into clickable links. Or in legalese: The “user interface highlighted these detected structures”.

NSDataDetector (available as of iOS 4) currently knows the following types, but you can add your own if you can put it into a regular expression as NSDataDetector is based on NSRegularExpression.

• NSTextCheckingTypeDate
• NSTextCheckingTypeAddress
• NSTextCheckingTypeLink
• NSTextCheckingTypePhoneNumber
• NSTextCheckingTypeTransitInformation

It’s very nice of Apple to provide this intelligent way of finding “structures in data”.

Did you have your DOH! moment yet?

Nowadays we are used to email and web pages working this way. Isn’t it extremely obvious? Users have come to expect a domain name to be clickable. But my iPhone was the first device that also turned phone numbers and dates into hyperlinks. The true feat of imagination is that Mr. Miller (and his 3 friends) thought of this 16 years ago, in the year that Windows 95 came out. It was sufficiently NON-obvious back then so that the patent was granted.

This patent being used to win the case against HTC might only be the first step. HTC claims to have a workaround they are scrambling to implement, but this workaround can only be to remove the auto-hyperlinking in Android apps, browsers and email clients. If Apple truly would want to be evil then they could now use this patent against any other product doing this broad sort of link detection: Microsoft Outlook, Gmail, even all sorts of editing apps that are modifying the text you input for your convenience, even blog engines like WordPress. There is a UI and there is code that detects and highlights. Bam!

When I shared my initial reaction to this online, people generally responded along the lines of “patents are bad, they stifle innovation”. And generally I do agree. I would abolish software patents in the blink of an eye if I could. But since they are a fact of current US business – that we cannot do anything about – I consider myself lucky to be focussing on Apple’s platform. Patents aside, Apple products generally DO win on quality. But it is also reassuring to know that Apple wields some patents that they are not afraid to use to protect the platform we depend on for our livelyhood.

Speaking of the general brokenness of the US patent system the lesson might be that often patents are granted that are not obvious at the current technical level we are at. But they might be commonplace 15 years from now. There needs to be either an expiration date or a re-evaluation process that un-grants patents that have outlived their uniqueness. Though that might still prolong the suffering of a dying patient. The patentability of UX needs to be abolished altogether.

Joseph Collins asks:

How do you decipher a bit mask from an argument which logically OR’d multiple values together? Enum uses bit shifting.

This question came to me while looking at UIView’s header file and wondering how Apple handles the animation options bitmask.

If you have several modes of something then usually you get by with an emum. But if you can combine several flags in s single value then you have to do this by means of bit masks. Let’s explore these today.

An enum is basically just a number, an integer. But you are telling the compiler that the individual values are legal values. Internally the first value will be 0, the second will be 1 and so on. If you want the first value to be a different value you can specify this and you can even specify a different number for each value. If you don’t, then they are just numbered sequentially.

Here’s an example from UIView.h:

typedef enum {
    UIViewAnimationCurveEaseInOut,         // slow at beginning and end
    UIViewAnimationCurveEaseIn,            // slow at beginning
    UIViewAnimationCurveEaseOut,           // slow at end
    UIViewAnimationCurveLinear
} UIViewAnimationCurve;

This defines UIViewAnimationCurveEaseInOut as 0 and the other values as 1, 2 and 3 using the automatic numbering. Often a developer chooses the default value to be the first because then a property using this type is also automatically initialized with the first value, aka 0.

From the same header file we can glean another example. Still defined as an enum, but using a bit mask:

enum {
    UIViewAutoresizingNone                 = 0,
    UIViewAutoresizingFlexibleLeftMargin   = 1 << 0,
    UIViewAutoresizingFlexibleWidth        = 1 << 1,
    UIViewAutoresizingFlexibleRightMargin  = 1 << 2,
    UIViewAutoresizingFlexibleTopMargin    = 1 << 3,
    UIViewAutoresizingFlexibleHeight       = 1 << 4,
    UIViewAutoresizingFlexibleBottomMargin = 1 << 5
};
typedef NSUInteger UIViewAutoresizing;

This is actually doing what I told you above, setting individual values for the numbers of the enum. The first value is 0, the second is a 1 bit shifted to the left by 0 positions, i.e. 1. The third value is a 1 bit shifted to the left by 1 positions, i.e. 2. The fourth is a 1 bit shifted to the left by 2 positions, i.e. 4. Get it it? They could have written out the numbers like this, but a geeky engineer preferred to write the values as shifts, number << bits_to_shift.

There's another interesting fact to be seen there. Usually you would typedef an enum and call that type something of your own. Then you would use this type for defining your properties. Apple actually does not give the enum (typedef keyword omitted) a name. But actually they define a type UIViewAutoresizing to be an unsigned integer. Isn't that a bit weird because it breaks the compile-time type checking?

To understand working with bits we need to review a couple of operators in the C language.

• logical OR: a || b
• logical NOT: !a
• logical AND: a && b
• bitwise OR: a | v
• bitwise NOT: ~a
• bitwise AND:  a & b
• bitwise shift left: a << b
• bitwise shift right:  a >> b

One thing that tends to confuse people is the distinction between “logical” and “bitwise”.  You probably have used the logical operators quite a bit in if statements. Think of the bitwise cousins of essentially the same but not just for one and zero, but for all bits of a number. A logical NOT of 5 is 0, but a bitwise NOT of 5 (bin 101) is 2 (bin 010).

To put it in simpler terms, a NOT flips all ones to zeros and vice versa. The AND and OR are twice the character for logical and once for bitwise. OR means that one of the bits needs to be one for the result to also be one. AND means that both need to be one for the result to be one. Remember that.

To combine several values from a bit mask enum into one, you simply chain them together with the pipe symbol, which is the bitwise OR operator. Mathematically a bitwise OR is the same as an addition of the values, as long as they don’t overlap.

view.autoresizingMask = UIViewAutoresizingFlexibleWidth|UIViewAutoresizingFlexibleHeight;

Now if you want to use a bit mask in your own code would involve the above mentioned type definitions as well as an if to check if certain bits of this mask are set. The simplest method I know is to basically make use of the logical AND. If the result is not equal to 0 – which it is if the bit was set – then we know that this bit was set.

if (view.autoresizingMask & UIViewAutoresizingFlexibleWidth)
{
   // bit was set
}

You could also compare the result to != 0, greater than 0 or equal to the value you are checking for but personally I think that this version is the most readable. Let me repeat that bit masks only work if you have non-overlapping numbers in the enum. Each flag needs to be a distinct bit in the number, otherwise you end up with unexpected result. Adding values to it that are not single bits would set multiple flags at the same time.

And then there’s the “pro” option. If you need to have more than a single bit on or off, but actually want to have an enum in and enum, there is also a way to do that as you can see in UIViewAnimationOptions. This is really weird to mentally parse if you are seeing that for the first time.

enum {
    UIViewAnimationOptionLayoutSubviews            = 1 <<  0,
    UIViewAnimationOptionAllowUserInteraction      = 1 <<  1, // turn on user interaction while animating
    UIViewAnimationOptionBeginFromCurrentState     = 1 <<  2, // start all views from current value, not initial value
    UIViewAnimationOptionRepeat                    = 1 <<  3, // repeat animation indefinitely
    UIViewAnimationOptionAutoreverse               = 1 <<  4, // if repeat, run animation back and forth
    UIViewAnimationOptionOverrideInheritedDuration = 1 <<  5, // ignore nested duration
    UIViewAnimationOptionOverrideInheritedCurve    = 1 <<  6, // ignore nested curve
    UIViewAnimationOptionAllowAnimatedContent      = 1 <<  7, // animate contents (applies to transitions only)
    UIViewAnimationOptionShowHideTransitionViews   = 1 <<  8, // flip to/from hidden state instead of adding/removing
 
    UIViewAnimationOptionCurveEaseInOut            = 0 << 16, // default
    UIViewAnimationOptionCurveEaseIn               = 1 << 16,
    UIViewAnimationOptionCurveEaseOut              = 2 << 16,
    UIViewAnimationOptionCurveLinear               = 3 << 16,
 
    UIViewAnimationOptionTransitionNone            = 0 << 20, // default
    UIViewAnimationOptionTransitionFlipFromLeft    = 1 << 20,
    UIViewAnimationOptionTransitionFlipFromRight   = 2 << 20,
    UIViewAnimationOptionTransitionCurlUp          = 3 << 20,
    UIViewAnimationOptionTransitionCurlDown        = 4 << 20,
    UIViewAnimationOptionTransitionCrossDissolve   = 5 << 20,
    UIViewAnimationOptionTransitionFlipFromTop     = 6 << 20,
    UIViewAnimationOptionTransitionFlipFromBottom  = 7 << 20,
};
typedef NSUInteger UIViewAnimationOptions;

I’m not dyslexic, but initially I thought “WTF are they doing shifting the number 20 (bin 10100) to the left?”. So you’re forgiven if you have the same reaction.

The first block is as we discussed before. The second and third block are actually more than single bits. You need two bits to represent the values 0 through 3. And three bits for 0 through 7. So here Apple is simply using the values and shifts them into place, 16 or 20 bits respectively. The advantage here is that a single animation options bit mask offers 32 bits that can be used. Packing the transition and the curve values into this still leaves with with a single integer to pass around.

Inside the UIKit code you probably have something like this, which shifts the values back to the right edge and then masks it and then compares.

NSUInteger transition = (animationOption >> 20) & 7;
NSUInteger curve = (animationOption >> 16) & 3;

The bitwise AND serves to mask out other set bits. Though if you are bit shifting the leftmost slot to the right, then the “incoming bits” are all zeroed anyway. But it’s probably a smart idea to make this masking a habit.

When creating your own bit masks I suggest that you name them in a similar way prefixing all enum values the same. This causes them to appear together in autocompletion and groups them logically.

Did you know you can actually also specify binary and hexadecimal values directly in your code? For hex numbers you add prefix 0x for binary prefix 0b. I heard that some compilers don’t know the binary prefix, but ours (LLVM) does. So if you EVER find yourself needing that you can use it. (You probably never will.)

NSUInteger b = 0b101; // binary
NSUInteger h = 0xAA; // hex

If you only take one thing away from this article then it should be the distinction between “logical” and “bitwise”. Because then you would also know what is wrong in the original question. Can you spot it?

Image “bitmask {phantom of the opera}” by S.D. Hilderbrand

Today BinPress announced the winners of the component contest. I had the pleasure of being one of the judges.

BinPress had custom-built an internal microsite for us judges where we could download the source code and fill in a form with our judgements. That allowed me to go into the components I was assigned to and really dig into the implementation details. Sorry, but I need to be wagging my finger here, all of the ones I saw had terrible form, little to no code style, the project setup generally was a mess and documentation non-existent.

But nevertheless a ranking was possible – especially because there where other equally important judging factors besides code quality. And so the shining winners are …

• Third place goes to RadioTunes SDK for iOS which is basically a wrapper for FFmpeg to decode various formats of Internet Radio streams.
• Second place is won by AsyncMVP which I have no idea what it does. Neither do I care because it is for Android.
• First prize was iCarousel which gives us multiple forms of carussels, including cylindrical, flat and “CoverFlow” style carousels.

The grand prize winner combines amazing functionality with exceptional code style. He gets $1500 cash plus $10000 worth of other goodies.

So one way or the other participating in this contest was definitely worth it. For the winners because of the additional business their components are seeing now and the tangible prize money they are raking in. For BinPress because of the publicity and for me as a judge because I got to see code that other developers are selling. And the latter strengthens my own resolve to renew my efforts to write nicer code.