Here’s another maintenance release fixing a crashing bug in the About section.

Changes

• Fixed: Crash on trying to open usage instructions
• Added: More detailed explanation of accuracy and filter settings

This update, like the one before it, goes out for the paid version of GeoCorder first. If the issues are resolved once this comes into the store then we’ll update the ad-sponsored version, too.

You might have noticed – if you follow the DTCoreText project on GitHub – that I made many changes on this Open Source project. The most recent change was that somebody donated a CocoaPods spec for the project and thus forced me to give it a version number. The reason being that pods are usually pointing to a specific tag in a GitHub repository. This way people using the project via CocoaPods can be certain that they are getting a stable version.

So I stumbled into this, but when trying to think of a good version number I could only come up with “1.0.0″. DTCoreText has matured sufficiently to call it that. Hey, earlier versions made it into quite a few popular apps, including Float. There are tons of performance improvements, additional features and most importantly the parser has been replaced with libxml2. This makes it both faster and able to deal with any kind of HTML you throw at it.

DTCoreText has two parts: first it creates NSAttributedString instances from HTML, second it displays these properly. CATextLayer would be able to take attributed strings, but it ignores paragraph attributes and cannot draw images. DTCoreText  has an ingenious mechanism where you can supply your own UIViews for each attachment.

Attachments can be generic <object>s, <video>s or even <img>. DTCoreText takes care of the drawing and leaving some space and then tells you precisely the size you need to provide for your image views. Of course, if you don’t need any special treatments then images can also be drawn together with the text.

What is the connection between DTCoreText and DTRichTextEditor?

To make up a rich text editor you need multiple parts which are all in separate projects.

• To render the display: DTCoreText
• To accept text input, provide selection and show a loupe: DTRichTextEditor (which includes DTLoupeView)
• To copy and paste HTML: DTWebArchive

DTRichTextEditor is being sold for €500 and includes DTLoupeView priced at €150. You can also purchase the Loupe separately if you don’t want to full text editing, but just the magnifier.

Several people have asked me about the exact terms of these Non-Attribution Licenses. I introduced these as a way for developers to sponsor the ongoing development in exchange for not having to mention Cocoanetics in the app. You get a clear conscience and the good feeling to have supported something that saved you hours and hours of work. You get an invoice for your accountant so you can even expense this payment.

But I have no way of checking if you do adhere to the terms, or if you use the components without attributing them properly. Hey, it is YOUR KARMA. If my Open Source projects save you a single hour of development time, then the investment in this license should be easily worth it.

Of course I would be over the moon if somebody told me that he wanted to purchase 12 such licenses for 12 of his clients. Hey, do what you feel that is fair, ok?

If you do the math you’ll find that this is indeed a bargain:

DTCoreText … €75
DTWebArchive … €75
DTLoupeView … €150
DTRichTextEditor … €200

Something that you cannot get elsewhere … €500

Actually that gets me thinking, I should maybe increase the prices. I just convinced myself that this is all way too cheap…

DTRichTextEditor Updated, Too

Now with DTCoreText gone gold I also updated DTRichTextEditor to include all these great advances. The project too comes with a Demo app that shows how to interact with the component as well as demonstrates how to integrate the editor. The above mentioned components are sub-projects of this with the two Open Source ones being a clone of the GitHub repo on my Subversion server.

Technically the demo (or your own app for that matter) only needs to link in the libDTRichTextEditor static library and copy the DTLoupe.bundle (containing the loupe images) into the app bundle. If you inspect the Demo settings you see how this is done.

It is generally wise to add the items you need to the dependencies so that Xcode will know to refresh these if they are modified. In the linked libraries you see a heap of Apple frameworks and that we link in libxml2 as well as the static libDTRichTextEditor which aggregates the objects from the other three projects into one convenient library.

HINT: There is one (non-obvious) maneuver you have to know to be able to get a resource bundle created in a sub-project in the list of bundle resources. You need to expand the sub-project so that you can see the products group (yellow folder). Then choose the project root to see the targets. Finally drag the bundle product from the sub-project into the “Copy Bundle Resources”. Xcode is smart enough to know where to copy the product from.

With the above setup Xcode will build all necessary parts for the one or two necessary architectures and merge it all together nicely. For further tips regarding what you need to know when using Sub-Projects refer to my earlier article on Sub-Projects and Header Magic.

Conclusion

I happen to like the concept of self-contained pieces very much and what I learned over the past few weeks of how to do it right enabled me to use them in a large scale component as DTRichTextEditor, bringing together multiple bits and pieces.

Of course development will go on, mostly on the Open Source elements. And there are several items still missing that I am guilty of procrastinating. I want  to provide a time-limited binary version of the framework for testing and evaluation. There are many items on the to do list for DTCoreText itself. I need a better quick start guide to do justice to the dozens of users who are funding the development efforts. And there really should be some AppleDoc-style documentation.

While I do like writing, I don’t like writing documentation as much. So I’ll do that when I get around to it…

We are lobbying since 2009 to get Apple to publish a proper API for downloading all kinds of reports.

The first reaction we got was prohibition of ITC scraping. The second reaction was that Apple created the Mobile ITC app which unfortunately lacks any kind of possibility to get the reports out or get monetary amounts. The third reaction was a half-harted publishing of a Java class that is able to download daily and weekly sales reports.

This changes today, at least if you are like me and feel uneasy to use Java for downloading reports.

The DTITCReportDownloader project is a complete rewrite of the Autoingest Java class in proper Objective-C. This way we iOS and Mac developers can at least download these two kinds of reports without having to have a JVM installed.

Please support our cause by duping Radar rdar://6807195 which is still lacking any kind of response. I was told back in 2009 that “if enough developers wanted it” Apple would finally give us the API we are wishing for.

On GitHub: https://github.com/Cocoanetics/DTITCReportDownloader

We’re rebooting the Cocoanetics Podcast. The last episode had been over a year ago.

Happy New … February

Please comment, tweet or mail your suggestions on topics that are of interested to iOS (and by extension Cocoa developers in general).

The new title song “Cocoa Cookin’” was composed and produced by Jamie Harper.

I finally got around to report an annoying bug in CoreText that has been bugging us in DTCoreText until I wrote a method to correct line origins as a workaround. rdar://10810114

The annoying thing about this bug is that it adds visual noise to otherwise pristinely rendered text. Especially on larger font sizes you see that additional space appears before each CTLine that ends with a paragraph break (\n).

UPDATE: This is a duplicate of rdar://9931615.

CoreText Line Spacing Bug

Summary

CoreText inserts too much space before any line that ends with a \n. This extra space depends on the font and font size. On large print this causes visual noise by not being uniform.

Steps to Reproduce

Create a CTFrame from a CTFrameSetter with a string that is long enough to wrap and that contains paragraph breaks. Use a non-UI font, like for example AriaMT.

Expected Results

Line origins should be spaced by exactly the same distance for identical text and identical attributes.

Actual Results

Each line that ends with a paragraph break is shifted down. With the system UI font, size 54 baselines are spaced exactly 64 pixels apart. With ArialMT, size 54, baseline spacing differs between 62 and 65.

Regression

This has been a bug since before iOS 4.3.

Notes

This does not occur with all fonts, Using a system font the spacing is precisely correct. I have attached a project to demonstrate the issue. See TextView.m.

It appears that the text metrics for an (invisible) paragraph glyph are miscalculated. Since the glyph is not visible you’d expect neither and ascender nor descender value. But instead the descender is too large. If you walk through the entire line and get the maximum ascenders and descenders the value is correct if you omit the \n in this calculation.

In short: A trailing \n messes up the font metrics for the entire CTLine.

Attachment: CoreTextLineOrigins Demo Project

When I moved the default CSS rules into a separate file I was facing the old dilemma: how can I embed this in the static library but still be able to easily add contents to it via Xcode. I previously explained how you can turn any file into a c-Byte-array. But this still required manual work.

I did a bit of researching and found that on regular Linux systems people seem to have a tool named objcopy which can copy files as their are into an object file (.o) which can be linked together to the final binary by the linker. But unfortunately this tool does not come with Xcode. So it is out of the question because I want everybody to be able to build DTCoreText.

Xcode Build Rules come to the rescue. They can automate any kind of preprocessing you like and it turns out they are an easy solution for this very problem.

There are essentially three kinds of source file that Xcode knows how to deal with (greatly simplified):

• c, c++ or Objective-C source code, or any code that can be somehow compiled
• header files
• other files

“Other files”, if you add them to a target, are treated as resources that simply get copied into your app bundle.

And now I’ll teach you how you can make any kind of file compilable. Be it some textures that you are compressing for building your game, or be it some resource that you want to embed into your binary.

Make a Build Rule

When Xcode is asked to compile a file it has a set of rules how it does that. So what we are going to do is to add our own rule how a css file is turned into a .c file. And for .c files are something that Xcode has a built-in rule for. So it can happen that you have two or more daisy chained rules that hand off their results to each other.

It is somewhat inconvenient that you cannot define rules globally. Instead you have to repeat your custom rules for each target.

To add your own rule, go into the projects info where you see your targets and on the target you want the rule for add it like shown.

This rule matches all files that end with .css (asterisk is a wildcard) and executes a custom script:

cd "$INPUT_FILE_DIR"  # move into file dir, otherwise xxd takes the full path for the symbol
/usr/bin/xxd -i "$INPUT_FILE_NAME" "$DERIVED_SOURCES_DIR/$INPUT_FILE_BASE.css.c" # builds a c file with a hex array

I’m moving into the folder of the input file first because xxd will always take the full passed file path and turn it into the name of the c-array. This way it will just be named default_c and the length will be in default_c_len.

The above rule is all it takes to teach Xcode how to turn a .css file into a .c file. Now the next step is to make sure that we are actually compiling the default.css file as opposed to copying it together with the other resources.

Compile not Copy

You can see that default.css is compiled by having it in the Compile Sources section of the Build Phases.

If you build the project you see two hints about what happens now. Towards the top of the build log you see the preprocessing of the file. A little further down you see how the intermediate c file is being compiled.

Further down the library tool puts all object files into the library archive. If you don’t believe me, you can employ the nm tool to inspect the symbols in the final product.

nm DemoApp | grep default_c
00045474 D _default_css
00045fe0 D _default_css_len

These are the two symbols from the default.css.c file that made it into the final product.

This guide wouldn’t be complete if I didn’t show you how to get to these symbols.

Accessing the Objectified File

I told you above that xxd uses the passed file name to make two variables, one for the c-style array, one for the length. Only change is that it has to turn the file name characters into a legal symbol, so dots are turned into underscores.

In DTCSSStylesheet.m I am accessing this array like so. I define these two variables as external which prompts the linker to insert their correct address.

+ (DTCSSStylesheet *)defaultStyleSheet
// external symbols generated via custom build rule and xxd
extern unsigned char default_css[];
extern unsigned int default_css_len;
}

And to get the string out of them is just as easy:

+ (DTCSSStylesheet *)defaultStyleSheet
{
   // get the data from the external symbol
   NSData *data = [NSData dataWithBytes:default_css length:default_css_len];
   NSString *cssString = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
 
   return [[DTCSSStylesheet alloc] initWithStyleBlock:cssString];
}

This works because I know that all my source files are UTF8 encoded, i.e. one byte per (normal) character. It is quite handy to have the default_css_len integer telling us the length of the array, because in c an array is just a pointer to the first byte.

Conclusion

If you know how to employ custom build rules to your advantage many tasks that you would have previously run external scripts for now become part of the build process. What you can do with this knowledge is only limited by your imagination … and your scripting skills.

But the general advantage of build rules versus external programs is that they are part of the project. So if they use standard commands they are portable to other developer’s machines. Perfect for Open Source projects.

This is a maintenance update. There was a bug in DTAboutViewController which caused a crash under iOS 5.

Change

• Fixed: Crash on iOS 5 when opening About Section

The update has been submitted to Apple for approval.

Update Feb 8th: Ready for Sale. It should be available shortly.

My iOS-related Podcast started out as “Dr. Touch Podcast” and later we rebranded ourselves to the more unique “Cocoanetics Podcast”. Though the topics always revolved around iOS development, Apple in general ans the people involved in our profession.

After an extended hiatus planning has begun to revive the podcast. I like to invite your comments, wishes and ideas as to what you would like to see – or more precisely hear – in a podcast that focusses on your daily bread and butter.

This impetus actually came from a composer by the name of Jamie Harper. A couple of weeks ago he contacted me offering his services. I thought to myself “well if I get new theme music for the podcast then I’ll relaunch it”.

I was kind of thinking that I could probably either not afford the composition or that this would take a long time to get done. Often you are dealing with people who vocally offer some great services only to disappoint you and never deliver.

But Mr. Harper delivered and he delivered very well. Not only did he compose a very nice intro jingle, but also made me a shorter and longer outro. And he did that at an exceptionally affordable rate. You’ll hear this music on the next show that I shall record.

Speaking of which … In the past I would always to a summary of iOS development news focussing on the kinds of news that would only half-heartedly be covered by the mainstream media which focusses on Apple products and the consumer end.

Sometimes I had the opportunity to interview industry leaders like Ortwin Gentz from Futuretap. Those podcast episodes where always good fun.

Finally I did speculate quite a bit on the future direction that Apple hints at with their actions. Crystal ball looking if you will. I like to talk and voice my opinion, that is one half why I want to do a podcast (because the established networks keep ignoring me).

The other reason for “why now?” is that there really is no dedicated iOS development news podcast. If you are interested in how great Marco Arment or his coffee is, then there’s a show for you. But nothing really covers stuff that I as a pro developer find interesting.

So here’s your chance to help shape the future of the Cocoanetics podcast. What are you interested in? What is niche enough to not be covered by others?

Also, let me know if you want to be on the show and get some free advertising for your apps and services. You get airtime if you let me ask you some in depth questions in an interview.

Your comments welcome, either via the public comments or if you don’t want anybody to know just drop me a mail.

There’s a category of developers – probably coming from other Linux-based desktops – who are feeling at home in the console (aka terminal). Especially when working with SCM systems like Git or Subversion you’re often faster committing or updating doing that via short commands as compared to Xcode.

Unfortunately Apple did not think to make the console a nice thing to look at from the get go. Let me show you quickly how you can geek up your terminal as well. Your colleagues will be impressed.

It’s really simple. You need to add these lines to your .profile which resides in your user home directory. If the file does not exist, create it.

export CLICOLOR=1
export LSCOLORS=Gxfxcxdxbxegedabagacad

The first line activates color support for the CLI (command line interface), the second line specifies which colors to use for certain kinds of things. When this is set every new console window will emit special ANSI sequences to set the colors.

You also need to enable these colors for your terminal. Make sure you have set your terminal emulation to xterm-color in Preferences – Advanced in the Terminal app. While you are here you can also set up some other things for how the Terminal should be styled. I prefer the “Pro” style which has a tinted glass background, sort of like a holographic display which you can see through.

The above settings will make your terminal look like this:

If these colors are not your style, don’t fret, Geoff Greer has build a cool online generator to piece together the LSCOLORS code. Just make your own color finger print and paste it into your .profile.

There’s more! Git also sports several color options:

git config color.branch auto
git config color.diff auto
git config color.interactive auto
git config color.status auto

… or you use the catch-all setting which includes all of the above:

git config color.ui true

This adds splashes of colors here and there, for examples a git diff:

Many other tools also support a color option, for example viewing a command’s man page is so much more interesting with a touch of color. If you have additional suggestions on how to make better use of color in terminal please don’t hesitate to comment below.

In this post I will to demonstrate how you can contribute to an Open Source project hosted on GitHub. I previously blogged about how you can make and apply patches, which is certainly one way. But on GitHub there is an even cooler method, one that is also less work on the project maintainer.

For the sake of this tutorial I shall pretend that there is an issue with a project by Matt Galloway, then I’ll fork his project. After fixing the issue I will push my changes online to my fork and finally notify Matt about these changes via a so-called “Pull Request”.

A prerequisite for this tutorial is to have your GitHub account and local git set up previously.

Step 1 – We have an Issue

Your first thought should be to document that in GitHub’s excellent bug tracking system. Each Open Source project there has a list of Issues (which are the bugs and feature requests). I found two things to improve in the project and so I added two issues.

We could hope for the project’s owner to fix that for us, but – being smart developers – we can also proceed to fix the issues ourselves. From the get go we won’t have writing access to the GitHub repository (unless the owner gives it to us, which is rare). You can see that this is the case if you only see the HTTP and Git Read-Only options on the repo URL:

The usual method is to make a copy of this project in our own area on GitHub where we CAN write to. This copy is what they call a “Fork”.

Step 2 – Stick a Fork in it

To make a fork you klick the “Fork” button at the top right of the project main page. After some “hardcore forking action” you end up on your own copy of the project. And there is now an extra option “SSH” telling you that you have read and write access to that.

A fork is a complete copy of the original project including all previous history. The reason for this that with Git every copy of a repository is always a full clone, so that each clone can also serve as a backup. This illustrates the distributed nature of Git, as opposed to centralized Source Code Management (SCM) systems like Subversion which has a central server.

Step 3 – Clone Wars

To work on our copy of the project we need to clone it to our hard disk. You can use the excellent official GitHub app but personally I prefer to work in terminal. I open terminal, go to a place where I want the copy of the project to be and issue the command:

git clone git@github.com:Cocoanetics/ios-library-with-resources.git

The URL used is the SSH one I copied from the web page. This results in a ios-library-with-resources folder appearing in the current working directory.

Step 4 – Fix-Tours

Now we can do our changes. So I open the Xcode project.

The first thing I am fixing for this example is to add the two sub-project products as dependency so that they get automatically built if somebody builds the containing app.

You see an M (for Modified) appear next to the root of the project tree which means that the project file has been modified. Each such fix should become one commit. So we right-click on MyLibraryTest, Source Control, Commit Selected Files. This opens a dialog that shows which files were modified and prompts for a commit message.

Since we fixed issue number #001 with that we also add this tag, because later we will see that GitHub conveniently links this commit with the issue via this number.

This committing basically saves the current state in your local Git repository (your clone) and adds your message to that. Actually it did a bit more than that, because Git has a an extra step called “staging” before committing.

In staging you select the modifications you want to be part of the commit by Adding them to the staging area. The second part then does this saving. If you do this via the Xcode interface you don’t need this staging phase because you specifically selected the files to commit already via the project navigator.

The second issue to fix is to change a build setting on the sub-project.

This time the M appears next to the MyLibrary.xcodeproj which really means the project.pbxproj file contained in the MyLibrary.xcodeproj bundle. (Remember that a bundle is a folder that looks like a file)

This time let’s do the committing on the command like, like a real pro. First let’s see the status.

git status
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.
#
# Changes not staged for commit:
#   (use "git add ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
#	modified:   MyLibrary/MyLibrary.xcodeproj/project.pbxproj
#
no changes added to commit (use "git add" and/or "git commit -a")

This shows that we are ahead of our fork (origin/master) by 1 commit, which is the previous one we did. And it shows that the project file for the sub-project was modified. We issue a “git commit -a” which stages all modified files for the commit and immediately prompts us for a commit message in the currently set up text editor.

There is a second shortcut that GitHub recognizes besides the #002 tag. If you write “closes” or “fixes” in front of the hashtag it not only references the issue, but also marks it as closed. How convenient!

“Set Skip Install YES for resource bundle to prevent copying that into Archive. closes #002″

I type my message into the VIM that opened and save/exit via ESC, :wq, ENTER.

Now we have two local commits (git log to see them), next we need to push them to our online repository. We just had modifications to the project file in these two examples, but – trust me – it works just as well for code changes.

Step 5 – Push Up

When we cloned the project to our local hard disk Git was nice enough to automatically configure the location where it came from as so called “Remote”, the default name is “origin”. You can have multiple remote places set up and choose which to push.

The command for us is:

git push origin master

“origin” is the remote I told you about, “master” means that you are pushing this to the master branch, which is the default branch.

To check if our changes have really arrived online I like to check the commits view on GitHub. And indeed they have.

We could end here and many people do. They just make a copy of an Open Source project for the case that the original repository disappears. But since we are playing nice we want to offer our improvements to the original project’s owner.

Step 6 – I just called to say “Please Pull Me!”

Just like for the fork we find a “Pull Request” button at the top right of the GitHub website of our fork. Push it and you get a nice overview of what is contained in the request.

There are some advanced options regarding milestones and assignments, but we’ll ignore these for now. You’ll notice that the pull request automatically generates a new number, #3 in our case. So you can not only reference issues with your commit messages, but the same thing is possible for pull requests.

The final step is not for us, but for the original project’s owner to perform.

Step 7 – Accepted!

As owner of a GitHub repo it is up to you whether you want to accept the changes into your repository. If you kept your commits small and – for code changes – added some good comments to aid understanding then the owner will be happy to accept them. It saves him from doing the work that you did and all he needs is to push the green merge button.

The button to accept the changes is only green if the merge of the code can be performed automatically. If the author did some changes on his own that conflict with yours then the button is gray stating that the changes cannot be merged automatically. In this unfortunate situation either one of you needs to pull the other’s changes and manually resolve the merging conflicts. But this is a topic for another day.

Conclusion

I suggest you find yourself a a GitHub project to contribute to and try out this process to shed the timid feeling. It really is quite idiot proof.

People put their projects on GitHub not only so that you get free source code to use in your apps. They also hope that other developers might find and fix bugs. And every once in a while you see a pull request that makes you think “OMG I’m not worthy!” because those are the gems that really teach you something.

And this is why they call it collaborative coding.

A month ago I was contacted by PacktPub with a “Review Request” and was provided with a ePub copy of the book for this exact purpose. PacktPub – which I had never heard of before – apparently is trying to get traction on the iOS developer market with a dozen books on the subject matter. But this pales in comparison to the hundreds of books they published for non-iOS ecosystems (Microsoft, Web, Java, etc.)

Is it just me or does it seem like more and more iOS developer are hoping to supplement their living from getting book royalties?

At the end of my procrastination I sat down and forced myself to read the first half of the book, taking notes to give this book a fair and balanced review. A word of caution: this will be a tragic comedy of epic proportions.

In Austria we have a saying: “Nothing is useless, it can always serve as a bad example”. It is my hope that this bad example serves to improve the overall quality of literature on our favorite subject matter.

The title of the work suggests that this book is not meant to be read in a linear fashion. You’d expect from a “Cookbook” to find “Recipes” for individual things that you would look up should you need to cook them. In the chapter about the book’s copyright you can read:

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied.

So far so good, but who is this book meant for? In the words of the author:

The iPhone Interface Cookbook is written from the ground up for people who are new to iOS or application interface design in general. Each chapter discusses the reasoning and design strategy behind critical interface components, as well as how to best integrate each into any iPhone or iPad application. Detailed and straight-forward recipes help give guidance and build understanding far beyond what is offered through Apple’s Human Interface Guidelines.

From these two parts of information – and possibly the short author’s bio – you come to expect a highly polished work that may sit on the shelf of a fledgling iOS developer getting his feet wet on his first couple of apps. A reference manual for design, if you will.

This puts me at odds with the premise. Neither am I fledgling, nor am I designer. I love to code and generally I like to delegate designing UI and UX to the companies that hire me. After having realized this I gave the book a second chance in my head: if it is worth the expense maybe I could recommend it to those designers sitting in my client’s companies for reference?

As an iOS contractor you often have to deal with companies that have established themselves on non-iOS platforms and now – due to public and marketing pressures – they also want an iOS app. Generally their in-house designers will know how to design a Windows app or website. Often I have wished for being able to recommend to them a guide as to what they have to do differently when designing for our platform.

This book isn’t that either.

Maybe I was “holding it wrong” because I read the book from the front cover to slightly more than half when I had to abort the mission. I’m detail oriented by nature and so I found boatloads of errors, formatting problems and blatant omissions with which I filled a note on Evernote with.

For example this is how e-mail addresses are formatted throughout. What do you think if you see this?

At first I attributed the problems I was seeing to the ePub format. Apparently the publisher did not invest in this being properly edited. Neither from a formatting point of view nor in terms of technical correctness.

Each chapter in the book slavishly follows this structure: Getting ready, How to do it…, How it works…, There’s more…, See also. The general style hides some interesting information in a tone ripe with fluff and half-truths.

And what’s even more annoying is that these mini-sections all take up space in the index which is destroying that as well. In this example you can see the worthless index. Why in god’s name would I want to directly jump to a There’s more in a section? You have no way of quickly getting an overview of what “Recipe” would be the right one for a real-life design case. Except wade through these superfluous entries of sub-sections.

I admitted that I am probably a) not the right person and b) not reading it correctly. But the problems with this book go far beyond the formatting.

Somehow I am getting the feeling that the individual chapters in this book might have been articles on a blog, that somebody just copy/pasted together and replaced all instances of “article” with “recipe” and “blog” with “book”. Is that truly the way how modern developers expect their information to be diced?

For one thing you’d expect from an experienced iOS developer to know how to spell the name of our IDE correctly. Instead Cameron (or is it a lawman editor?) insists on spelling it XCode. This book couldn’t have been edited on a Mac because even my WordPress autocorrected that to Xcode. Purists might call this a blatant disregard for what is holy.

There are many more examples of misspelled classes, apps or terms we iOS developers deal with every day. To give a second example: WebKit is one word, not two.

Ok, there are formatting problems (ePub’s fault), there are technical editing problems. To top it off there are factual problems as well.

The author details how to generated certificates on the provisioning portal, but omits the simple information that you can just connect your device via USB, click on the “User for Development” button in Xcode Organizer and have Xcode auto-provision the device in your “iOS Team Provisioning Profile”.

Then there is much commingling. Alert Views are thrown in together with modal view controllers. Later they are mashed up with push notifications and local notifications. And the rationalization for why everything is modal:  “iOS is a highly modal operating system, forcing one application at a time upon the user.”

There are 4 step guides where the second and third step are identical, just worded differently. There are in-depth explanations about topic that a designer has no interest in, and the interesting topics are covered incompletely. In truth I found two or three nuggets of information buried deep in what I read. For example I found the explanation on Fitt’s Law quite interesting, didn’t know that.

Sooner or later you realize that the statement from the Copyright section is a lie. “Every effort” was definitely not made and the information isn’t presented at an acceptable level of “accuracy”.

The second promise that the publisher did not keep is who the book is for. Where are the explanations “far beyond” of what you can find in Apple’s HIG? Where are the chapters that I can recommend to the UI/UX designer working at my client’s?

I had to stop reading half way through, when I had accumulated 3 screenfuls of “bugs”. I am not getting paid to correct this book, so that amount exceeded the level of bullish!t I was willing to put up with. My job is to evaluate whether this work of art is worthy of your attention.

By now you should fathom my opinion.

I chose this article’s title to try and grab your attention. Well, the product is still the same and does the same. The only difference is one that is under the hood. And as such it is your job – should you choose to accept it – to marvel at the benefits that the new old parsing engine brings us.

Ever since my friends at scribd showed me a first prototype for HTML parsing based on libxml2 I was – admittedly – jealous. This prototype basically worked by having libxml2 parse out the individual paragraphs of an article and then display each paragraph in its own cell of a UITableView. Back then I brushed off the suggestion and went with something I understood: NSScanner.

All the C-code necessary to deal with libxml2 seemed overly daunting, so I went with a simple basic structure for the parsing, something like this (pseudocode)

while (there_is_more)
{
   if (we_scanned_a_tag)
   {
      if (is_tag_a)
      {
         // a specific code
      }
      else if (is_tag_b)
      {
         //  b specific code
      }
      else if ...
   }
   else
   {
      // we must be inside a tag
 
      // deal with skipping over a tag that is incomplete, i.e. crap
 
      if (we_scanned_some_text_before_the_next_tag_open)
      {
         // append the text with correct formatting to the output
      }
   }
}

This structure had grown organically as I added support for additional tags and it went into the initWithHTML category method because that seemed to be the logical place. Little did I think ahead that this approach would preclude the possibility of having an event-based parser do callbacks into my code, because that would mean adding these event-handling methods to NSAttributedString.

Even the above pseudocode  is long, you can imagine how much of a Spaghetti the origin code became. Much of the problem didn’t actually come from all these tags, those where simply a big if-statement. Complexity came from having to deal with all these special cases where HTML might not be well-formed.

In my Open Source genstrings2 I saw how much faster pure C-code performs than NSScanner which also served to rekindle my wish to switch to libxml2 because this is also written in low-level highly optimized C.

I approached libxml2 in several steps:

1. Jealousy – “Boy wouldn’t it be great, but I’m afraid that this is out of my league”
2. Announce Intention – I wrote an issue on GitHub hoping for somebody to step forward
3. Do an Experiment and Document – I googled a bit and put together Part 1 of my libxml HTML tutorial.
4. Write a Wrapper – For Part 2 of my libxml HTML tutorial I wrote an Objective-C wrapper for libxml2.
5. Astonishment – The feeling you get when you find that you begin to understand the C-code needed
6. Benchmark – I removed all string building code and compared the raw parsing performance of both approaches
7. Transform the Pasta – Moved the code for building the attributed string into an aptly named class and have this driven from events generated by the new HTML parser.

The final step I called like this because if you break up Spaghetti code into several logical pieces and then layer these into several layers that’s a different kind of Pasta, that’s called Lasagne.

For step 6, the comparison I moved the NSScanner parsing loop into its own class and directly compared the running time on my iPhone 4S resulting in this tweet:

War&Peace HTML (3.4 MB), NSScanner: 4.264s, libxml2: 1.398s = 3x as fast on single thread, plus latter fixes HTML structure

At this stage it was clear to me that I need no extra self-convincing. So I went to work in a branch of the project. Most of the work was simply copy/pasting the attributed string building code into the right place, tag start, tag end or the characters found event method. This also allowed for omitting some workarounds that where needed to deal with non-well-formed HTML.

The second big BIG advantage of libxml2 is that its HTML parser fixes up the structure for you and also adds a missing html and body tag so that you end up with a perfect structure. It even adds a </br> right after a <br>. Even though this is completely unnecessary it still makes the HTML look like perfect XML. Much nicer to work with.

While I was doing the migrating new issues and pull requests with fixes stated to come in putting me a bit under stress because I needed to include the fixes in both branches. Which is why I decided to merge the branch back into master at the earliest possible time.

The initWithHTML method has shrunken to a much more manageable size:

- (id)initWithHTML:(NSData *)data options:(NSDictionary *)options documentAttributes:(NSDictionary **)dict
{
	// only with valid data
	if (![data length])
	{
 
		return nil;
	}
 
	DTHTMLAttributedStringBuilder	*stringBuilder = [[DTHTMLAttributedStringBuilder alloc] initWithHTML:data options:options documentAttributes:dict];
 
	[stringBuilder buildString];
 
	return [stringBuilder generatedAttributedString];
}

Benchmark after the Merge

And of course I did some further benchmarking on my iPhone 4S to compare the resulting speed increase. All tests where done by simply adding two NSLogs at the beginning and end of the initWithHTML method and subtracting the timestamps. Note that this is just the time for building the attributed string and does not include the layouting or drawing.

Demo HTML Snippet 

NSScanner: 50 ms

libxml2: 43 ms

War&Peace ePub HTML 

NSScanner: 10.968 sec

libxml2: 8.796 sec

This is an overall parsing and string building speed increase of between 14% and  22%.

You might now ask what happened to the 60% increase we saw from just comparing the parsers. The string building itself is still all happening on the same thread as the parsing and by itself has many opportunities for optimization.

For one thing the next thing I’d like to do is to put the string building operations onto its own GCD background queue. This way the events coming in from libxml would hand off to a queue running on a different thread and would immediately return to parsing. This could easily double the overall scanning performance because you can now make use of the two CPU cores present in the more modern iOS devices.

And there are obviously many more optimizations that now are feasible to do because you no longer face a daunting monster spaghetti but instead stand a chance of understanding what is happing in DTHTMLAttributedStringBuilder.

Conclusion

The main three advantages of libxml2 over NSScanner are: Performance, HTML non-wellformedness-resilience, simpler code through event-based handling of HTML.

This merge now makes it possible again for interested developers to contribute optimizations and new features because the code has become so much simpler to read and understand.

Here are my mental notes for today’s earnings call…

Peter Oppenheimer, CFO

“Highest Quarter Earnings and Revenue”, ”All Time Highs”

3-4 Weeks Mac channel inventory

MAS: 100 Mio Downloads in less than a year

128% YOY grown in iPhone, “tremendous popularity of iPhone 4S”. Target range 4-6 weeks of iPhone channel inventory

available in over 90 country

iPads increase 111%, accessories grew 99%. Within 4-6 weeks of iPad channel inventory.

With iBook Author anyone can create “iBook Textbooks, Cookbooks, picture books and more”

> 85 Million customers on iCloud

Half of Macs sold in stores are to New Mac users

“Easy Pay” checkout process key in handling the amount of buyers in store

They are actively discussing on what do do with their cash balance, but nothing to announce yet.

Extremely enthusiastic about their product pipeline.

Tim Cook, CEO – Q&A

iPhone grows at 3 times of the mobile phone market

Sold more iPhones than they expected, so called “short in channel inventory”. Caught up in some countries, but not all.

Component environment is and will stay favorable, except for hard disks (Thailand). Did get better prices for displays, NANDs, DRAM last quarter.

iPad: hugh opportunity for Apple, tablet market will be larger than PC market. “Limited function and eReaders” are a different category and thus not a thread. Who wants an iPads gets one. They’ll “continue to innovate like crazy in this area.”

Team at Apple “so incredible”. “We feel very good about where we are”

5 Reasons why this quarter will be earning less than last

1. Dec quarter had a 14th week = 1/14th of the Revenue, DOH
2. A week missing in March Quarter that is usually going well
3. Last year they produced more iPhones
4. “significant pent up demand for 4S”
5. Strengthening of Dollar against the Euro

About acquisitions: Apple generally believes in totally integrating acquired companies into the mothership. “You will be assimilated, resistance is futile” (well at least that’s what I thought to myself)

“There was not an obvious aspect on the numbers” regarding if the Kindle Fire had any positive or negative effect on iPad sales. iPad cannibalizes the Mac, but much more cannibalizes Windows PCs.

Mac has outgrown market for 20 quarters in a row. Seems like all of the data make it a very close race against Android, iPad is way ahead. Though it’s hard to get any “crisp numbers” on Android handset shipments.

“iOS is doing extremely well”. Prefers to ignore the “other horses” and focus on innovating.

And when asked about larger screens and LTE we got usual response.

The parsing of HTML necessary for my DTCoreText open source project is done entirely with the sophisticated use of NSScanner. But it has been long on my list to rewrite all of this parsing with a the industry standard libxml2 which comes preinstalled on all iOS devices. Not only is this potentially much faster in dealing with large chunks of HTML. It probably is also more intelligent in correcting structural errors of the HTML code you throw at it.

In part 1 of this series I showed you how to add the libxml2 library to your project and explained the basic concepts of using libxml2 for parsing any odd HTML snippet into a DOM. In this here part 2 we will create an Objective-C based wrapper around libml2 so that we can use it just like NSXMLParser, only for HTML.

There are basically two methods – approaches – to handling XML/HTML: DOM or SAX. DOM (short for Document Object Model) refers to building a tree of nodes and leaves which you then can query for information based on their path (with XPath). SAX (short for Simple API for XML) basically walks through the the document and sends you events as it discovers things. For example the begin of an element, the end of it, when a comment was found, etc.

NSXMLParser is a SAX parser for well-formed XML. That means you can use it only on XML, it cannot deal with the nature of HTML where some elements might not require that you close them. But if you look at the delegate protocol and compare it to the SAX interface of libxml2 you find an eerie resemblance. It might as well be that NSXMLParser is just a wrapper around the XML SAX interface of libxml2.

As we have previously learned we know that libxml2 also possesses a HTML parser which reuses most of the same internal data structures as its XML cousin. So we shall endeavor to create DTHTMLParser as a SAX parser for HTML.

Don’t worry about piecing together all the source code from this article, I will put that online in my DTFoundation project. Instead I want you to follow my lead and understand the pieces.

The libxml2 SAX Interface

The SAX interface works such that you register a handler function – in C – for each SAX event that you are interested in. This is done via a C-structure of type htmlSAXHandler which is nothing more than an aggregate of multiple C-function pointers.

In the libxml headers you often see some “html…” data structure typedef’d into one with “xml” as prefix. The reason for this is that it reuses whatever it can from the XML parser. But this also adds confusion – as I found through my experimenting – because some of the events that make sense for parsing XML don’t fire for HTML, like for example the one that reports CDATA elements. HTML does not have those, but the function pointer is still present in the htmlSAXHandler struct.

By CMD+Clicking on a type we can swim upstream to find the original definition of htmlSAXHandler:

htmlSAXHandler = xmlSAXHandler = _xmlSAXHandler =

struct _xmlSAXHandler {
    internalSubsetSAXFunc internalSubset;
    isStandaloneSAXFunc isStandalone;
    hasInternalSubsetSAXFunc hasInternalSubset;
    hasExternalSubsetSAXFunc hasExternalSubset;
    resolveEntitySAXFunc resolveEntity;
    getEntitySAXFunc getEntity;
    entityDeclSAXFunc entityDecl;
    notationDeclSAXFunc notationDecl;
    attributeDeclSAXFunc attributeDecl;
    elementDeclSAXFunc elementDecl;
    unparsedEntityDeclSAXFunc unparsedEntityDecl;
    setDocumentLocatorSAXFunc setDocumentLocator;
    startDocumentSAXFunc startDocument;
    endDocumentSAXFunc endDocument;
    startElementSAXFunc startElement;
    endElementSAXFunc endElement;
    referenceSAXFunc reference;
    charactersSAXFunc characters;
    ignorableWhitespaceSAXFunc ignorableWhitespace;
    processingInstructionSAXFunc processingInstruction;
    commentSAXFunc comment;
    warningSAXFunc warning;
    errorSAXFunc error;
    fatalErrorSAXFunc fatalError; /* unused error() get all the errors */
    getParameterEntitySAXFunc getParameterEntity;
    cdataBlockSAXFunc cdataBlock;
    externalSubsetSAXFunc externalSubset;
    unsigned int initialized;
    /* The following fields are extensions available only on version 2 */
    void *_private;
    startElementNsSAX2Func startElementNs;
    endElementNsSAX2Func endElementNs;
    xmlStructuredErrorFunc serror;
};

All these function pointer types are aptly named SomethingFunc and we’ll get to how these need to look like in a minute. The problem for us to solve how to bridge from our Objective-C paradigm (using self, properties, IVARs) into the C-function paradigm (no access to our parser object).

This problem is compounded by the fact that we are building this with ARC so that we don’t have to deal with memory management any more. Our resulting class will run on iOS 4 and above. Somehow we need to pass enough information to each event-function so that we can communicate with the parser object.

There are two reasonable possibilities: either we pass a reference to the parser instance or we pass a pointer to a struct that contains multiple values that we might need. I decided for the first because it seems cleaner to me.

libxml2 provides a concept of a user_data pointer that is passed to the event-functions together with other relevant information. We can use (abuse?) this for passing a reference to self. ARC requires this to be be cast to (__bridge void *) which in plain English means: “please treat this as just some typeless memory pointer and also don’t worry about retaining or releasing. I’ll take care of that, don’t panic.”

Like any self-respecting class we’ll start out with a good block of IVAR definitions and and init method.

@implementation DTHTMLParser
{
    htmlSAXHandler _handler;
 
    NSStringEncoding _encoding;
    NSData *_data;
 
    __unsafe_unretained id <DTHTMLParserDelegate> _delegate;
    htmlParserCtxtPtr _parserContext;
}

Notice that I’m defining the IVARs in the implementation, not the interface (in the header). This is possible as of using the new Apple LLVM compiler and provides for a nice way to hide the IVARs that we don’t want to be visible.

- (id)initWithData:(NSData *)data encoding:(NSStringEncoding)encoding
{
	self = [super init];
	if (self)
	{
		_data = data;
		_encoding = encoding;
 
		xmlSAX2InitHtmlDefaultSAXHandler(&_handler);
	}
 
	return self;
}

We save the data and the encoding into our IVARs, _data being a strong reference to it. That means that it retains it during the lifetime of our parser and will only release it when the parser goes away.

We also initialize the _handler structure via the xmlSAX2InitHtmlDefaultSAXHandler function. This sets up some initial internals and also chooses the SAX2 method for our work. There’s also an older SAX1 method, but we obviously prefer the newer one. This setup has to happen inside the init method because right after such an instance of a DTHTMLParser is create we want to be able to set the delegate which should take care of wiring up the needed functions.

Each event-function should have a corresponding delegate method in our DTHTMLParserDelegate protocol, but to avoid unnecessary function-calling we only add the functions to the handler for which the delegate actually implements the function.

#pragma mark Properties
 
- (void)setDelegate:(__unsafe_unretained id<DTHTMLParserDelegate>)delegate;
{
	if (delegate != _delegate)
	{
		_delegate = delegate;
 
		if ([_delegate respondsToSelector:@selector(parserDidStartDocument:)])
		{
			_handler.startDocument = _startDocument;
		}
		else
		{
			_handler.startDocument = NULL;
		}
 
		if ([_delegate respondsToSelector:@selector(parserDidEndDocument:)])
		{
			_handler.endDocument = _endDocument;
		}
		else
		{
			_handler.endDocument = NULL;
		}
// ...

This way when you set the delegate it is inspected one by one whether it responds to certain methods of the protocol. And whenever it does the corresponding function pointer is set in the _handler IVAR.

At the top of the file we need to define these functions. If we do just that then the compiler will warn us for all of these implementations that there was “no previous prototype”. In C a you call the first line of a C-function it’s prototype. So we just have an extra block for these.

#pragma mark Event function prototypes
 
void _startDocument(void *context);
void _endDocument(void *context);
void _startElement(void *context, const xmlChar *name,const xmlChar **atts);
void _endElement(void *context, const xmlChar *name);
void _characters(void *context, const xmlChar *ch, int len);
void _comment(void *context, const xmlChar *value);
void _error(void *context, const char *msg, ...);

Wherever you see a context parameter this will be the user_data pointer we will provide. And for example the _startElement function gets a name and an array of attributes as well. The xmlChar type is just a char. This is actually always provided as UTF8-encoded string regardless of what the data is encoded as.

Let’s look first at an easy event-function and then at a more complex one.

void _startDocument(void *context)
{
    DTHTMLParser *myself = (__bridge DTHTMLParser *)context;
 
    [myself.delegate parserDidStartDocument:myself];
}

You see how I cast the void pointer back into a pointer of type DTHTMLParser. I have to name the variable different than “self” because that is a reserved word. Then I can simply call the delegate method that corresponds to this event function.

By the way, these are the delegate methods we want to implement:

@class DTHTMLParser;
 
@protocol DTHTMLParserDelegate <NSObject>
 
@optional
- (void)parserDidStartDocument:(DTHTMLParser *)parser;
- (void)parserDidEndDocument:(DTHTMLParser *)parser;
- (void)parser:(DTHTMLParser *)parser didStartElement:(NSString *)elementName attributes:(NSDictionary *)attributeDict;
- (void)parser:(DTHTMLParser *)parser didEndElement:(NSString *)elementName;
- (void)parser:(DTHTMLParser *)parser foundCharacters:(NSString *)string;
- (void)parser:(DTHTMLParser *)parser foundComment:(NSString *)comment;
- (void)parser:(DTHTMLParser *)parser parseErrorOccurred:(NSError *)parseError;
 
@end

The @class is necessary before the protocol definition because a good delegate protocol also passes a reference to the caller. This also enables the casual reader of your code to know at a glance that this method belongs to a protocol connected to this class.

Now for something more complex, let’s look at didStartElement. The complexity of this method results from the arts parameter being alternating key, value, key, value etc.

void _startElement(void *context, const xmlChar *name,const xmlChar **atts)
{
	DTHTMLParser *myself = (__bridge DTHTMLParser *)context;
 
	NSString *nameStr = [NSString stringWithUTF8String:(char *)name];
 
	NSMutableDictionary *attributes = nil;
 
	if (atts)
	{
		NSString *key = nil;
		NSString *value = nil;
 
		attributes = [[NSMutableDictionary alloc] init];
 
		int i=0;
		while (1)
		{
			char *att = (char *)atts[i++];
 
			if (!key)
			{
				if (!att)
				{
					// we're done
					break;
				}
 
				key = [NSString stringWithUTF8String:att];
			}
			else
			{
				if (att)
				{
					value = [NSString stringWithUTF8String:att];
				}
				else
				{
					// solo attribute
					value = key;
				}
 
				[attributes setObject:value forKey:key];
 
				value = nil;
				key = nil;
			}
		}
	}
 
	[myself.delegate parser:myself didStartElement:nameStr attributes:attributes];
}

The while loop walks through the attributes and and when a NULL is encountered where a key would be then it breaks out of the loop. It is also necessary to deal with a situation where there is only an attribute name, but no value because HTML also allows that. Theoretically one attribute could also be occurring multiple times, but we ignore this. For later convenience we want the key/values for the attributes in an NSDictionary.

The final piece is how the parsing actually is started. For this we have a parse method, just like NSXMLParser.

- (BOOL)parse
{
	void *dataBytes = (char *)[_data bytes];
	int dataSize = [_data length];
 
	// detect encoding if necessary
	xmlCharEncoding charEnc = 0;
 
	if (!_encoding)
	{
		charEnc = xmlDetectCharEncoding(dataBytes, dataSize);
	}
	else
	{
		// convert the encoding
		// TODO: proper mapping from _encoding to xmlCharEncoding
		CFStringEncoding cfenc = CFStringConvertNSStringEncodingToEncoding(_encoding);
		CFStringRef cfencstr = CFStringConvertEncodingToIANACharSetName(cfenc);
		const char *enc = CFStringGetCStringPtr(cfencstr, 0);
 
		charEnc = xmlParseCharEncoding(enc);
	}
 
	// create a parse context
	_parserContext = htmlCreatePushParserCtxt(&_handler, (__bridge void *)self, 
		dataBytes, dataSize, NULL, charEnc);
 
	// set some options
	htmlCtxtUseOptions(_parserContext, HTML_PARSE_RECOVER | HTML_PARSE_NONET |
		HTML_PARSE_COMPACT);
 
	// parse!
	int result = htmlParseDocument(_parserContext);
 
	return (result==0);
}

There is a xmlDetectCharEncoding function in libxml to try and detect the encoding of the data which we invoke if the passed encoding is 0. The way to convert the NSStringEncoding into the xmlCharEncoding is only work shift, that needs some improving, possibly by simply having a big switch statement. Maybe later.

There are multiple ways how you can parse a document, some of which take individual parameters, some take a html parsing context. I settled on using the one for the push interface because there the htmlParseDocument function does not return a pointer to an html document tree (DOM), but instead a result that is 0 for success or -1 for error.

It is my understanding that this variant does not actually build the entire tree but keeps just enough state information to track the current hierarchy level inside the HTML tree. The push parser variant also has the ability to parse chunks of HTML data as they become available, e.g. while downloading. But exploring that is outside our scope for now.

Conclusion

Once your eyes have gotten used to the coding style of libxml2 you can start to appreciate the plethora of functions that it provides. You can also peruse the official reference manual online.

Finally I invite your collaboration on using and helping to polish DTHTMLParser if you have any need for parsing HTML text from within your iOS apps. Look for it on the GitHub DTFoundation project as soon as I can add a bit of AppleDoc-style documentation to it.

In my previous article about Sub-Projects in Xcode I showed you how you can have an Xcode project as a dependency. The advantage of this is that you can update your library from within the same project and debug into the sub-project’s code.

One thing that was not immediately obvious was how you deal with the problem that Xcode cannot find your sub-project’s library headers. Hence this writeup.

A header file generally informs the compiler what c-functions, global variables, Object-C methods and classes exist in an object file. All these are generally called “symbols”.

Build Process Review

The build process consists of these three steps:

1. Preprocessor – this takes care of including/importing header files into your source code, essentially pasting the content of the specified file right into your source code for building. This step also replaces all instances of #defined values with what you defined them to be.
2. Compiler – this goes through all the .m files and builds an object file with .o extension for each. Each item that is not coming from the same .m file is marked as an external symbol.
3. Linker – this merges all the object files into one big binary while at the same time linking the symbols with the actual implementations.

Header files do not actually contain executable code, but instead they tell the compiler which names and functions it should accept. It is the linker then that resolves these references. In short: If you want to use something which is implemented outside the current .m file than you need the appropriate import for that.

There are two kinds of imports:

• <path/header.h> with angle brackets – these are meant for system or “global” includes.
• “path/header.h” with double quotes – these are meant for items that are limited in scope to your current project

You have probably used both already, the one with the angle brackets for including headers that Apple provided. The one with the double quotes for accessing your own classes. Paths are usually relative paths because everybody has a different folder structure on his hard disk.

Get the Headers into Your App

The easiest way to get the headers into your project is to just add them. But you should generally refrain from duplicating your code for convenience, because then you have to perform an update on every copy of the file. A variation of this theme would be to not add the file itself, but just a reference. But this approach would also make your project dependent on your folder structure. Don’t do it, just don’t.

If you have followed the Sub-Projects guide you are already perfectly set up to get to the header while still keeping the project self-contained and independent from the file system layout. You only need to tell the outer Xcode project – probably your app that includes sub-projects for some static libraries – where it can find the necessary headers from the sub-project. Again you have multiple options: you can point it to the source code folder where the .h files are located. Or you can point it to the build output folder. Both are equally viable but if you work with static universal frameworks – as I do – then you can only use option two without losing your mind.

On my projects that contain reusable code I usually have multiple targets. One for a universal static framework – which is very convenient to add to a project because it automatically sets up the library and header paths for you when you drag it into your project. I have a second target for a static library because this allows for using the project as a sub-project and have the outer project link to the static library product. Because the framework conveniently bundles together multiple .h files you usually have the general name of the framework as path, i.e. you find <MobFox/MobFox.h> as the import of choice. For for example an Apple framework <Foundation/Foundation.h>.

Contrasting the MobFox/ path the actual location of the header file is PROJECT_ROOT/Core, so no MobFox path component there. This is the reason why we cannot point the containing app to the source folder because there is no header file MobFox.h inside a folder named MobFox. So via approach number 1 we would never find the header, we want to take door number two which is to point Xcode to the static library build output folder. But for this to work some setup is necessary.

Specifying Public Header Output Location

After building the static library Xcode will also copy certain headers to the output folder. A header can be public, project or private. Public means that it will be distributed together with the library. Project and Private won’t. You can set this for every header file individually. Headers for objects that you want to be able to see from your app should be Public.

If you now build the static library and look at the build log you will see that at the very end the public headers are copied somewhere:

We are astonished to find a path ending in Build/Products/Debug-iphoneos/usr/local/include for the two public header files. The reason for this being the default setting is that traditionally people would build binary files for the system they are working on. But that’s not what we want. So we go into the build settings for the static library target and adjust this. Don’t put this entry into either the Debug or Release lines, but into the line above these so that it may override both.

When we now build we see the output go to Build/Products/Debug-iphoneos/MobFox – as we want it. It is this extra MobFox path component that enables us to use the <MobFox/MobFox.h> style of including later on.

Now back to our app that is including this sub-project’s static library. There we now need to tell the compiler where it can find the above folder. It so happens that when building Xcode usually uses the same output folder for all products it needs to build, namely whatever is in the $(CONFIGURATION_BUILD_DIR) variable at the time. This in turn is set by default to $(BUILD_DIR)/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME).

The final piece of the puzzle to know is where to put that. This depends on the include style that I explained above. Double quoted includes need the path in “User Header Search Paths”. Angle-bracketed includes need the regular “Header Search Paths”. We are forced to use the latter due to the framework. We don’t need to check the Recursive box because at the given location there is a MobFox folder.

The <Multiple values> might cause you to pause for a second, but remember what I told you above. The configuration (Debug/Release) is part of this path and so even though you put the same variable in there, the actual result is a different one. Note that these values are bold because the target settings override the settings specified by the project. You can revert to the project settings by selecting the line and hitting CMD+Backspace.

There is an alternate option to put this path in the “User Header Search Paths” instead, but then you need to set “Always Search User Paths” to Yes. This will cause the compiler to also search the user header paths for angle bracket includes. Though personally I prefer the other option that needs less settings.

The other steps for building are explained in the Sub-Projects posts. If you carry them out (link with target), you will find that now your app finds the headers and successfully builds.

Conclusion

If Xcode complains that it cannot find a header then you should make sure that it is indeed in a place where it can find it. There is an abundance of default settings that may seem daunting, but if you know what to look for you can quickly find why your header cannot be found.

I wrote this article because I myself was stumped by not being able to explain why a project couldn’t find the MobFox header. So another learning of this episode is that you should not assume that the owner of your sub-project has set everything up correctly. In my case I too was missing the custom header output path setting. Errare humanum est.

Besides fixing several bugs reported by users this release focusses on implementing a new custom-built super-charged strings scanner.

Changes

• FIXED: missing strings file no longer aborts import
• FIXED: added – Button for deleting tokens
• FIXED: deletion of wrong row
• FIXED: over-escaping of \ on writing strings files
• FIXED: wrong token name updated on enter key
• FIXED: search filter not reset on adding a new token
• FIXED: Previous/Next buttons in wizard cease to function
• NEW: replaced genstrings with high-speed custom implementation (>20x speed improvement)
• NEW: support custom localization macro prefix

This new version does no longer support 32-bit because the new strings scanner was implemented using ARC. We hope that this does not inconvenience any of our existing uses. But then again, serious developers are on a 64 bit machine anyway since Snow Leopard…

The new version has been submitted to Apple today.

Over the past week I’ve been working on DTLocalizableStringScanner or in short genstrings2. The original genstrings dates back to the NeXtStep days. You know how it is “never change a winning team” BUT “the good is the enemy of the great”, because if something kind of works, why change it?

Besides of the other problems I’ve alluded to in my previous article genstrings is very slow. Internally it is written in Objective-C as you can tell from the occasional stack trace when it crashes again. But it was created in a time when Macs did only have single CPU cores and when we did not have the awesome LLVM with ARC, GCD and multi-threading.

At the time of this writing we implemented genstrings2 twice: once based extensively on NSScanner which I became intimately acquainted with when working on DTCoreText. The other is a rewrite of the rewrite totally ditching NSScanner and doing all scanning directly on the individual characters.

My feeling is that the lower-level scanning will be orders of magnitude faster and at the same time it is way more robust because for non-literal macro parameters it knows how to deal with pairs of parenthesis and commas contained in strings. But this just begs to be benchmarked.

In order to have a sufficient amount of code to scan through I downloaded the latest source bundle for Adium. Getting a time for the individual versions is easy with the time command:

time genstrings -q -s AILocalizedString ~/Desktop/adium-1.4.4/Source/*.m
 
real     0m7.872s
user     0m6.848s
sys      0m0.053s

When benchmarking you also need to mention the type of bench you are marking. These numbers come from my 2010 MacBook Air which is somewhat skewed because the disk IO times are possibly faster than usual. But we are mostly interested in the parsing efficiency. Here the benchmark machine has 2 CPU cores (1.6 GHz Intel Core 2 Duo).

You can see in activity monitor that original genstrings is single-threaded. It uses up to 100% of a single core and shows to have only 1 thread.

The version of genstrings2 using NSScanner clocked in much better. 1.7 seconds down from 7.9 seconds, 5 times as fast.

real     0m1.663s
user     0m2.950s
sys      0m0.070s

Now for the turbo version that does all the scanning without NSScanner. Instead it is working with the individual unichar characters that make up the strings. On iOS these are 16 bit because internally all NSStrings are using UTF16.

real     0m2.167s
user     0m3.973s
sys      0m0.074s

Whoa! Hold your horses, why is that slower than my version? My first thought was that maybe NSScanner really is very efficient. But once I fired up Instruments I found this, lots of CFRelease calls were bogging down the turbo scanner.

Looking at the sample distribution in _scanMacro revealed the culprit.

From this it was quite obvious that the creation of the mutable array to hold the scanned macro parameters was taking most of the CPU time in this area of the code. Turns out this was allocating and releasing this parameter array way more than necessary. We only need to created that once we are sure that this is actually one of the macros we are looking for, not some random word that happens to be made up of macro name characters.

So moving the array creation a few lines down and changing it to an alloc/initWithCapacity (we know that we probably will have less than 10 parameters) yielded this result in the benchmark:

real     0m1.385s
user     0m2.045s
sys      0m0.081s

Looking at Instruments again we see even more potential for optimization. If a line takes a quarter of the CPU time spent in a method then you have to ask yourself if there is a way to get rid of this work. Now this validMacroCharacters is already supposed to be a lazy initialization, so how to optimize this?

While it is lazy there is still quite some overhead to do the method call. On the first time around the character set is initialized and on subsequent calls the method returns the initialized IVAR. We can easily get around this by moving the initialization of the IVAR to the class init and reference it directly.

Normally this method call plus if plus returning the IVAR would not be a problem, but _scanMacro is called thousands of times and so – as proven by Instruments – this is an easy optimization. The benchmark improvement is very impressive:

real	0m1.065s
user	0m1.839s
sys	0m0.059s

At this stage the turbo scanner (with two changes) is already almost twice as fast as the NSScanner-version and 7 times as fast as original genstrings. BUT three is a charm and it so happens that I had a stroke of genius.

My train of thought went: if I can defer CPU-intensive work to the latest moment possible and if I can reduce the amount of times when it is necessary, what other method could I come up with to skip out of _scanMacro early? After the previous optimizations most CPU time was distributed between creating an NSString object and the containsObject of the macro dictionary. Two items that cannot be easily optimized further.

Upon closer inspection I found that far too many times words were found that were made up of characters in the macro words. This resulted in these two statements being called way more often than necessary, even for words that are not plausible macro names because they are too short. On the NSScanner version I had had the optimization to ignore short commands like “if” and “do” because these would look like macro characters but are in reality reserved words. But this did not yield and advantage.

The stroke of genius was to find the lengths of the shortest and longest macro names (also in the init) and then have a simple if comparison to return from _scanMacro if we character sequence is far too short for one of our macro names. The results blew me away.

real	0m0.442s
user	0m0.691s
sys	0m0.044s

Now we have truly reached the point of impossible further optimization. Looking in Instruments the CPU times are now much wider spread, only characterIsMember cumulates more CPU times then the other lines, but this method cannot be optimized further since that would mean writing a replacement for NSCharacterSet.

Conclusion

Let’s summarize the lessons learned:

• Good C-code easily outperforms NSScanner
• Defer creating objects to the latest possible moment
• Avoid lazy instantiations if they are called in tight loops that cause them to be called too often
• Try to find simple mathematical or logical abort conditions (quickly processed) to avoid CPU-intense work where possible

Granted I could spend more effort on working out the performance problems the NSScanner version has, but I doubt that I could get anywhere near the double turbo version. The final results thus are:

• genstrings: 7.9 sec
• genstrings (NSScanner): 1.7 sec
• genstrings (optimized turbo): 0.4 sec

The optimized turbo version is 20 times as fast as the original, let’s be content with that.

LEGO’s Life of George game brings together 5 worlds like nothing else before:

• 8-Bit Design
• iPhone
• LEGO bricks
• manual interaction, a “touch interface” if you will
• computer vision for gamification

Let’s have a quick look at this fun pastime that is all the rage.

The LEGO Life of George app is a free download on the app store. To play you need the box which contains some additional ingredients you need to play. You get 6×1, 8×2, 4×3, 4×4 and 2×6 pieces in red, green, blue, black, yellow, white. Any self-respecting LEGOist probably has these at home. Then there is a piece of cardboard that has a grid of dots which aide in recognizing the shapes you put together. Plus there is a very brief Getting Started guide and a sticker showing George in all his 8-bit glory.

The product is smartly priced at a level where you probably prefer the convenience of having it all in a box over pirating the pieces from your kids and using a scan of the mat. Also this level of innovativeness needs to be rewarded, so I purchased the original product at my Apple Store. And so should you: if not for yourself then for all the LEGO+iPhone fans around you.

You select a model from initially 130 models and your goal is to complete these in as little time as possible. So you select a model, build that from the included pieces, drop it onto the grid, tap the cam button on the iPhone app which displays the model and countdown. Then you position the iPhone such that it sees the entire grid. It does not matter where on the grid you place the finished model.

When playing you should make sure that you have sufficient space to lay out the pieces for faster access. Also the bricks have a tendency to fall to the floor if permitted. The models are rated in 12 difficulty levels.

The most fun, but also the biggest weakness of this game is Multiplayer. There is no support for Game Center and Online competition. Instead you have to make due with pass’n'play mode. Players alternate building different models, because it would be useless to be building exactly the same model if the person before you has already found all the pieces. Though there seems to be a balancing glitch: On one of my multi-player games I was getting a very difficult model while my opponent got something extremely simple. Unfair!

The other problem is that on several occasions the model was not recognized properly causing no points to be awarded. Instead you just get a display with tips to improve recognition, like tilting to avoid shine or making sure that the entire mat is visible. But no option to retake the picture. Annoying!

LEGO plans to sell us additional models for certain holidays, the first one being the Christmas pack for $2 which contains “3 new levels of Christmas content with 30 new Challenges and 45 models”. Don’t ask me how this works out mathematically and what the difference between challenge and model is. That the price is still $2 even after the turn of the year reminds us that Apple still does not permit In-App-Purchases for free.

Finally – what might be even be the coolest feature of all – you also get to make your own models. I tried making my Cocoanetics logo here:

The image recognition software tries to match the contained pieces with what it “sees” on the mat. But the pieces need to be upright. We build a ciqua logo, but with the pips pointing to the side and that was not properly recognized.  The software cannot differentiate between two adjacent pieces of the same color. If you please a 2 and a 1 next to each other, it is recognized as a 3. So far the models the app tries out all the variations that are possible and seems to always try to take the longest bricks possible.

Their low-cost technology circumvents the problem you have with normal computer vision of having to adjust for a wide variety of image quality by simplifying the shapes it has to recognize, little squares in primary colors, and by using this grid mat for calibration. It’s is orders of magnitude more simple to recognize and align a grid of primary color blocks versus having to do edge and shape detection. This is an smart approach makes the technology robust enough for use by everybody. If you are used to buying LEGO bricks then you also have no issue with using this cardboard thing. So this partnership turns out to be a very nice fit between LEGO and Eye Cue.

LEGO is smart to exploring this unknown territory between digital and physical, between virtual and real. Normal apps which are digital offer much less satisfying value to the player than LEGO can.

If what I am hearing is any indication then LEGO has a hit. What’s next? Maybe an app to remote control LEGO robots?

Actually I am on vacation, but I couldn’t help myself using the breather to work on a little hobby project. This I shall reveal to you today!

If you localize your apps (iOS or Mac regardless) you are probably used to working with genstrings. Probably painfully working, which is why we built the Mac app Linguan which remote-controls genstrings and merges the results with your previously existing tokens.

But genstrings has several big problems.

The most pressing problem of using genstrings from Linguan is that it is very picky about the syntax of the string macros. If you have a parameter for the macro that is not a literal @”string” then it will crash. Several Linguan users have been stumped by this behavior because this makes it impossible for Linguan to scan their source code.

The second problem is that we don’t have access to genstrings source code, and so we neither cannot know what this tool really does, nor can we modernize it to make use of modern multi-processor systems. genstrings is single-threaded.

The third problem is that before now the only way how we could call genstrings from within Linguan is basically open an invisible shell and then read the output files from some hidden temporary output folder. You get genstrings with Xcode, but what if you don’t want to install Xcode on the machine you want to use with Linguan? Having to spawn a process for each source file individually has also proven to be impracticable because instead of a 0.3 seconds this would take 12 seconds for scanning just the Linguan source files.

If that is any indication then the genstrings man page has not been updated since May 7, 2007. Any substantial improvements to the tool are probably even older than this, somebody from Apple told me that this goes back as long as the NeXt days.

My goal was to reverse-engineer the functionality of genstrings, put that into a static library and thus be able to bring native high-performance string scanning to Linguan without the need for genstrings to be present any more.

My latest Open Source project DTLocalizableStringScanner (on GitHub) contains two targets: one for a static library which we can use for scanning the source files. The other is a re-implementation of a command line tool which can take the place of genstrings.

genstrings2 is built around NSScanner for the scanning and GCD queues for multi-threading. There might be faster ways to scan for the macros but there is no reason to do that at the expense of the code being easy to read. I’d rather fill up a Grand Central Dispatch queue with one block per source file and thus make full use of multiple CPU cores. It works on as many files in parallel as GCD permits.

You can help hone this into a worthy successor for genstrings by testing it on your own source code and telling me about edge cases that it does not handle properly. For one thing it should ignore invalid macros instead of crashing like the original, that’s one immediate advantage that you get from symbolic-linking genstrings2 to be used instead of genstrings.

There is one (undocumented) behavior that is not implemented yet. genstrings apparently expands tokens containing lists in the format [one, two, three] into three lines with [one], [two] and [three]. People would use this to localizes NSPredicateEditor. But that’s an easy exercise that I will do in the next few days.

If there are no show stoppers then DTLocalizableStringScanner will replace genstrings already in Linguan 1.0.3.

Imagine me getting a mani-pedi during the Christmas holidays. No, really! We have a lady come to our house regularly and doing all willing feet.

Somehow our casual conversion came to the iPhone. Now imagine this healthcare professional telling me:

“For me the iPhone is no option because it does not work with the gloves we use”.

One does not have to say THAT to me twice. This myth was begging to be busted.

First thing I tried out was one of the original Latex gloves. There was almost no difference in the ability of my iPhone and iPad detecting touches.

Can you imagine how belief-shattering this was when I showed it off to my witnesses? ”Oh my, then I had been told a lie!”

When I mentioned this finding on Twitter one of the responses was:

They’re probably thin enough. Regular gloves don’t work, unless they’re specially prepared.

Thicker. Hm, where do I get thicker gloves? Ah, my wife got these cleaning gloves made from rubber. Those only worked like 80%, you had to push slightly more than usual and also their rubbery surface made sliding over the display a bit awkward. But it still worked well enough for emergencies.

So thickness does not play a role either as this step 2 in our experiment showed.

John Blanchard told me:

“I recently noticed I can use my iPhone through my motorcycle gloves. Didn’t expect that!”

And here is picture proof of him using his iPhone with his real leather Stryker Gloves. They don’t even mention “iPhone-compatible” in the product description. Maybe they should!

I asked for other people to contribute photos of them using their iPhones with gloves. Hermano Queiroz was the second person to present photo proof of his gloves working with his iPhone:

Haha, very funny Hermano! We all know that you’re cheating on this photo. Clearly these are your wife’s fingers, not yours.

But Hermano also had this picture proof. He need 3 tries for that. On the first you could only see the screen showing my original tweet, but not the material of the gloves. The second only showed a blurry screen. The HDR setting finally got the best of both worlds.

Finally the coolest entry in the “show your gloves contest” did not involve gloves at all. Timothee Boucher mentioned:

I’m sure it would work with a Ziplock™ bag. I used that to still listen to and control Pandora while painting.

It just so happens that I hav such a bag at home, who doesn’t? I have been using it to waterproof my iPhone while walking the dog while it rains. In Europe these are sold by Toppits.

Here you see me aceing the final test:

If we learned anything from this exercise then it is that our iOS devices are even more amazing than we previously realized.

It is true that most material that are used for gloves won’t work. I tested wool and rough leather to no avail. But smooth leather has been proven to work on John’s gloves. Apparently there are a couple of elusive factors at work that are more complex than simply material or thickness.

This post would not be complete if I didn’t mention that – of course! – there are winter gloves with specially coated finger tips. These guys in Michigan have compiled an extensive list: Best iPhone, iPad & Android Touchscreen Gloves.

People just love to use their iDevices with gloves.