We are lobbying since 2009 to get Apple to publish a proper API for downloading all kinds of reports.

The first reaction we got was prohibition of ITC scraping. The second reaction was that Apple created the Mobile ITC app which unfortunately lacks any kind of possibility to get the reports out or get monetary amounts. The third reaction was a half-harted publishing of a Java class that is able to download daily and weekly sales reports.

This changes today, at least if you are like me and feel uneasy to use Java for downloading reports.

The DTITCReportDownloader project is a complete rewrite of the Autoingest Java class in proper Objective-C. This way we iOS and Mac developers can at least download these two kinds of reports without having to have a JVM installed.

Please support our cause by duping Radar rdar://6807195 which is still lacking any kind of response. I was told back in 2009 that “if enough developers wanted it” Apple would finally give us the API we are wishing for.

On GitHub: https://github.com/Cocoanetics/DTITCReportDownloader

I chose this article’s title to try and grab your attention. Well, the product is still the same and does the same. The only difference is one that is under the hood. And as such it is your job – should you choose to accept it – to marvel at the benefits that the new old parsing engine brings us.

Ever since my friends at scribd showed me a first prototype for HTML parsing based on libxml2 I was – admittedly – jealous. This prototype basically worked by having libxml2 parse out the individual paragraphs of an article and then display each paragraph in its own cell of a UITableView. Back then I brushed off the suggestion and went with something I understood: NSScanner.

All the C-code necessary to deal with libxml2 seemed overly daunting, so I went with a simple basic structure for the parsing, something like this (pseudocode)

while (there_is_more)
{
   if (we_scanned_a_tag)
   {
      if (is_tag_a)
      {
         // a specific code
      }
      else if (is_tag_b)
      {
         //  b specific code
      }
      else if ...
   }
   else
   {
      // we must be inside a tag
 
      // deal with skipping over a tag that is incomplete, i.e. crap
 
      if (we_scanned_some_text_before_the_next_tag_open)
      {
         // append the text with correct formatting to the output
      }
   }
}

This structure had grown organically as I added support for additional tags and it went into the initWithHTML category method because that seemed to be the logical place. Little did I think ahead that this approach would preclude the possibility of having an event-based parser do callbacks into my code, because that would mean adding these event-handling methods to NSAttributedString.

Even the above pseudocode  is long, you can imagine how much of a Spaghetti the origin code became. Much of the problem didn’t actually come from all these tags, those where simply a big if-statement. Complexity came from having to deal with all these special cases where HTML might not be well-formed.

In my Open Source genstrings2 I saw how much faster pure C-code performs than NSScanner which also served to rekindle my wish to switch to libxml2 because this is also written in low-level highly optimized C.

I approached libxml2 in several steps:

1. Jealousy – “Boy wouldn’t it be great, but I’m afraid that this is out of my league”
2. Announce Intention – I wrote an issue on GitHub hoping for somebody to step forward
3. Do an Experiment and Document – I googled a bit and put together Part 1 of my libxml HTML tutorial.
4. Write a Wrapper – For Part 2 of my libxml HTML tutorial I wrote an Objective-C wrapper for libxml2.
5. Astonishment – The feeling you get when you find that you begin to understand the C-code needed
6. Benchmark – I removed all string building code and compared the raw parsing performance of both approaches
7. Transform the Pasta – Moved the code for building the attributed string into an aptly named class and have this driven from events generated by the new HTML parser.

The final step I called like this because if you break up Spaghetti code into several logical pieces and then layer these into several layers that’s a different kind of Pasta, that’s called Lasagne.

For step 6, the comparison I moved the NSScanner parsing loop into its own class and directly compared the running time on my iPhone 4S resulting in this tweet:

War&Peace HTML (3.4 MB), NSScanner: 4.264s, libxml2: 1.398s = 3x as fast on single thread, plus latter fixes HTML structure

At this stage it was clear to me that I need no extra self-convincing. So I went to work in a branch of the project. Most of the work was simply copy/pasting the attributed string building code into the right place, tag start, tag end or the characters found event method. This also allowed for omitting some workarounds that where needed to deal with non-well-formed HTML.

The second big BIG advantage of libxml2 is that its HTML parser fixes up the structure for you and also adds a missing html and body tag so that you end up with a perfect structure. It even adds a </br> right after a <br>. Even though this is completely unnecessary it still makes the HTML look like perfect XML. Much nicer to work with.

While I was doing the migrating new issues and pull requests with fixes stated to come in putting me a bit under stress because I needed to include the fixes in both branches. Which is why I decided to merge the branch back into master at the earliest possible time.

The initWithHTML method has shrunken to a much more manageable size:

- (id)initWithHTML:(NSData *)data options:(NSDictionary *)options documentAttributes:(NSDictionary **)dict
{
	// only with valid data
	if (![data length])
	{
 
		return nil;
	}
 
	DTHTMLAttributedStringBuilder	*stringBuilder = [[DTHTMLAttributedStringBuilder alloc] initWithHTML:data options:options documentAttributes:dict];
 
	[stringBuilder buildString];
 
	return [stringBuilder generatedAttributedString];
}

Benchmark after the Merge

And of course I did some further benchmarking on my iPhone 4S to compare the resulting speed increase. All tests where done by simply adding two NSLogs at the beginning and end of the initWithHTML method and subtracting the timestamps. Note that this is just the time for building the attributed string and does not include the layouting or drawing.

Demo HTML Snippet 

NSScanner: 50 ms

libxml2: 43 ms

War&Peace ePub HTML 

NSScanner: 10.968 sec

libxml2: 8.796 sec

This is an overall parsing and string building speed increase of between 14% and  22%.

You might now ask what happened to the 60% increase we saw from just comparing the parsers. The string building itself is still all happening on the same thread as the parsing and by itself has many opportunities for optimization.

For one thing the next thing I’d like to do is to put the string building operations onto its own GCD background queue. This way the events coming in from libxml would hand off to a queue running on a different thread and would immediately return to parsing. This could easily double the overall scanning performance because you can now make use of the two CPU cores present in the more modern iOS devices.

And there are obviously many more optimizations that now are feasible to do because you no longer face a daunting monster spaghetti but instead stand a chance of understanding what is happing in DTHTMLAttributedStringBuilder.

Conclusion

The main three advantages of libxml2 over NSScanner are: Performance, HTML non-wellformedness-resilience, simpler code through event-based handling of HTML.

This merge now makes it possible again for interested developers to contribute optimizations and new features because the code has become so much simpler to read and understand.

Over the past week I’ve been working on DTLocalizableStringScanner or in short genstrings2. The original genstrings dates back to the NeXtStep days. You know how it is “never change a winning team” BUT “the good is the enemy of the great”, because if something kind of works, why change it?

Besides of the other problems I’ve alluded to in my previous article genstrings is very slow. Internally it is written in Objective-C as you can tell from the occasional stack trace when it crashes again. But it was created in a time when Macs did only have single CPU cores and when we did not have the awesome LLVM with ARC, GCD and multi-threading.

At the time of this writing we implemented genstrings2 twice: once based extensively on NSScanner which I became intimately acquainted with when working on DTCoreText. The other is a rewrite of the rewrite totally ditching NSScanner and doing all scanning directly on the individual characters.

My feeling is that the lower-level scanning will be orders of magnitude faster and at the same time it is way more robust because for non-literal macro parameters it knows how to deal with pairs of parenthesis and commas contained in strings. But this just begs to be benchmarked.

In order to have a sufficient amount of code to scan through I downloaded the latest source bundle for Adium. Getting a time for the individual versions is easy with the time command:

time genstrings -q -s AILocalizedString ~/Desktop/adium-1.4.4/Source/*.m
 
real     0m7.872s
user     0m6.848s
sys      0m0.053s

When benchmarking you also need to mention the type of bench you are marking. These numbers come from my 2010 MacBook Air which is somewhat skewed because the disk IO times are possibly faster than usual. But we are mostly interested in the parsing efficiency. Here the benchmark machine has 2 CPU cores (1.6 GHz Intel Core 2 Duo).

You can see in activity monitor that original genstrings is single-threaded. It uses up to 100% of a single core and shows to have only 1 thread.

The version of genstrings2 using NSScanner clocked in much better. 1.7 seconds down from 7.9 seconds, 5 times as fast.

real     0m1.663s
user     0m2.950s
sys      0m0.070s

Now for the turbo version that does all the scanning without NSScanner. Instead it is working with the individual unichar characters that make up the strings. On iOS these are 16 bit because internally all NSStrings are using UTF16.

real     0m2.167s
user     0m3.973s
sys      0m0.074s

Whoa! Hold your horses, why is that slower than my version? My first thought was that maybe NSScanner really is very efficient. But once I fired up Instruments I found this, lots of CFRelease calls were bogging down the turbo scanner.

Looking at the sample distribution in _scanMacro revealed the culprit.

From this it was quite obvious that the creation of the mutable array to hold the scanned macro parameters was taking most of the CPU time in this area of the code. Turns out this was allocating and releasing this parameter array way more than necessary. We only need to created that once we are sure that this is actually one of the macros we are looking for, not some random word that happens to be made up of macro name characters.

So moving the array creation a few lines down and changing it to an alloc/initWithCapacity (we know that we probably will have less than 10 parameters) yielded this result in the benchmark:

real     0m1.385s
user     0m2.045s
sys      0m0.081s

Looking at Instruments again we see even more potential for optimization. If a line takes a quarter of the CPU time spent in a method then you have to ask yourself if there is a way to get rid of this work. Now this validMacroCharacters is already supposed to be a lazy initialization, so how to optimize this?

While it is lazy there is still quite some overhead to do the method call. On the first time around the character set is initialized and on subsequent calls the method returns the initialized IVAR. We can easily get around this by moving the initialization of the IVAR to the class init and reference it directly.

Normally this method call plus if plus returning the IVAR would not be a problem, but _scanMacro is called thousands of times and so – as proven by Instruments – this is an easy optimization. The benchmark improvement is very impressive:

real	0m1.065s
user	0m1.839s
sys	0m0.059s

At this stage the turbo scanner (with two changes) is already almost twice as fast as the NSScanner-version and 7 times as fast as original genstrings. BUT three is a charm and it so happens that I had a stroke of genius.

My train of thought went: if I can defer CPU-intensive work to the latest moment possible and if I can reduce the amount of times when it is necessary, what other method could I come up with to skip out of _scanMacro early? After the previous optimizations most CPU time was distributed between creating an NSString object and the containsObject of the macro dictionary. Two items that cannot be easily optimized further.

Upon closer inspection I found that far too many times words were found that were made up of characters in the macro words. This resulted in these two statements being called way more often than necessary, even for words that are not plausible macro names because they are too short. On the NSScanner version I had had the optimization to ignore short commands like “if” and “do” because these would look like macro characters but are in reality reserved words. But this did not yield and advantage.

The stroke of genius was to find the lengths of the shortest and longest macro names (also in the init) and then have a simple if comparison to return from _scanMacro if we character sequence is far too short for one of our macro names. The results blew me away.

real	0m0.442s
user	0m0.691s
sys	0m0.044s

Now we have truly reached the point of impossible further optimization. Looking in Instruments the CPU times are now much wider spread, only characterIsMember cumulates more CPU times then the other lines, but this method cannot be optimized further since that would mean writing a replacement for NSCharacterSet.

Conclusion

Let’s summarize the lessons learned:

• Good C-code easily outperforms NSScanner
• Defer creating objects to the latest possible moment
• Avoid lazy instantiations if they are called in tight loops that cause them to be called too often
• Try to find simple mathematical or logical abort conditions (quickly processed) to avoid CPU-intense work where possible

Granted I could spend more effort on working out the performance problems the NSScanner version has, but I doubt that I could get anywhere near the double turbo version. The final results thus are:

• genstrings: 7.9 sec
• genstrings (NSScanner): 1.7 sec
• genstrings (optimized turbo): 0.4 sec

The optimized turbo version is 20 times as fast as the original, let’s be content with that.

Actually I am on vacation, but I couldn’t help myself using the breather to work on a little hobby project. This I shall reveal to you today!

If you localize your apps (iOS or Mac regardless) you are probably used to working with genstrings. Probably painfully working, which is why we built the Mac app Linguan which remote-controls genstrings and merges the results with your previously existing tokens.

But genstrings has several big problems.

The most pressing problem of using genstrings from Linguan is that it is very picky about the syntax of the string macros. If you have a parameter for the macro that is not a literal @”string” then it will crash. Several Linguan users have been stumped by this behavior because this makes it impossible for Linguan to scan their source code.

The second problem is that we don’t have access to genstrings source code, and so we neither cannot know what this tool really does, nor can we modernize it to make use of modern multi-processor systems. genstrings is single-threaded.

The third problem is that before now the only way how we could call genstrings from within Linguan is basically open an invisible shell and then read the output files from some hidden temporary output folder. You get genstrings with Xcode, but what if you don’t want to install Xcode on the machine you want to use with Linguan? Having to spawn a process for each source file individually has also proven to be impracticable because instead of a 0.3 seconds this would take 12 seconds for scanning just the Linguan source files.

If that is any indication then the genstrings man page has not been updated since May 7, 2007. Any substantial improvements to the tool are probably even older than this, somebody from Apple told me that this goes back as long as the NeXt days.

My goal was to reverse-engineer the functionality of genstrings, put that into a static library and thus be able to bring native high-performance string scanning to Linguan without the need for genstrings to be present any more.

My latest Open Source project DTLocalizableStringScanner (on GitHub) contains two targets: one for a static library which we can use for scanning the source files. The other is a re-implementation of a command line tool which can take the place of genstrings.

genstrings2 is built around NSScanner for the scanning and GCD queues for multi-threading. There might be faster ways to scan for the macros but there is no reason to do that at the expense of the code being easy to read. I’d rather fill up a Grand Central Dispatch queue with one block per source file and thus make full use of multiple CPU cores. It works on as many files in parallel as GCD permits.

You can help hone this into a worthy successor for genstrings by testing it on your own source code and telling me about edge cases that it does not handle properly. For one thing it should ignore invalid macros instead of crashing like the original, that’s one immediate advantage that you get from symbolic-linking genstrings2 to be used instead of genstrings.

There is one (undocumented) behavior that is not implemented yet. genstrings apparently expands tokens containing lists in the format [one, two, three] into three lines with [one], [two] and [three]. People would use this to localizes NSPredicateEditor. But that’s an easy exercise that I will do in the next few days.

If there are no show stoppers then DTLocalizableStringScanner will replace genstrings already in Linguan 1.0.3.

If you have ever looked at UIPasteboard you might have seen that there are a variety of public types, like colors, images, plain text or URLs. But applications are also free to implement their own types to be put on the pasteboard when the existing types don’t do justice to your content.

One such custom type is being used between Apple’s iOS apps, like mobile Safari, whenever you copy HTML snippets: “Apple Web Archive pasteboard type”. At first glance this looks really secretive because all you can see there is a long string of numbers representing the NSData for it.

Today I am unveiling an Open Source solution to consuming this pasteboard type as well: DTWebArchive. Using this you can let your users copy something from Safari or Mail and pasted it into your app while preserving the rich text. I put this code into a new GitHub repository because even if you don’t dabble with CoreText and NSAttributedStrings+HTML then this project can be very useful to you.

When I started implementing cut/copy/paste for my DTRichTextEditor I found that text copied from Safari is both available on the pasteboard as plain text and as the above mentioned type. At that point I thought “Boy wouldn’t it be great if we could get at the HTML that was pasted”. Well, now you can.

The reverse-engineering began by inspecting the NSData with a text editor. All I needed was to see the “BPLIST” tag at the beginning to know that this is in fact a binary plist. I deserialized it and was astonished to find that this is simply an NSDictionary.

There are several elements to this archive, the most important one being the main contents in a WebArchive class which has mime type “test/html” and is the pure HTML that we are seeking. In addition to that – if you have copied images as well – you see an array of WebRessource elements where each encapsulates a file, typically an image, sometimes a CSS file. Those are basically cached copies. The original HTML still has the web URLs in it, but you could search for the WebResource with the same URL to find the local version.

Finally if you have IFRAMEs in the main HTML then each IFRAME will also have a corresponding WebArchive in an separate array. For example a HTML5 YouTube video would be showing in an IFRAME.

Because WebKit is Open Source could can see for yourself why my cleanroom re-implementation is way more portable. Heres the source for WebArchive.mm and here for WebResource.mm which relies heavily on LegacyWebArchive.cpp.

Here’s an example where I copied text with one image.

I decoded this pasteboard item with DTWebArchive output the HTML.

DTWebArchive *archive = [[DTWebArchive alloc] initWithData:data];
 
DTWebResource *resource = archive.mainResource;
NSString *string = [[NSString alloc] initWithData:resource.data encoding:NSUTF8StringEncoding];
 
NSLog(@"%@", string);
 
[archive release];

This is not entirely correct, because it hard-codes NSUTF8StringEncoding but should take the actual encoding of the WebArchive contents. But you get the idea.

I copied this into a snippet to look at it with the NSAttributedString+HTML demo project and this is what you see:

(Actually I had to cheat a bit because I found that I first had to fix something in NSAS+HTML that would cause the image to shift upwards.)

The next step for me is now to add this project as a submodule to NSAS+HTML and implement convenience methods to convert between attributed strings and web archives. Also maybe some convenience methods might be nice to get the UIImage for a specific URL.

For you this means that you should add copy and pasting of rich HTML to your applications as well.

You might have noticed that I blogged much less during the past 3 months, that was for the most part because almost all of my programming time went into a secret project for Scribd. Something that is finally revealed to the public on July 19th 2011.

Preempting the next question I am usually asked at this point: “What is Scribd?” Scribd is often described as “the  YouTube of Documents”. You can upload and share any kind of document on their network and they have an HTML5 reader that you can embed on your blog.

At the time the official statement was that Scribd is working on a mobile reader for iOS and they needed much more control over the rendering and interactivity of HTML-based content than UIWebView would afford them.

Let me tell you how Scribd has completely ditched UIWebView and is revolutionizing the way you read on your iPhone. Introducing Float.

What is Float? Think of it like this: everything digital that you are reading today (blogs, tweeted articles, articles you have on a reading list, eBooks, documents, etc.) in one amazing mobile native app that displays the content in the most perfect way possible on your device.

What follows here is a look at Float from a technical point of view, which I can provide because I was put in charge of the rich text display. If you are interested in reading more about the user experience itself then have a look at the reviews on all the other major blogs. I especially liked the review on Wired.com, which mentions me by name.

Only The Best Ingredients

My story with Float started 3 months ago. Sam Soffes – whom I greatly admire – discovered my work on CoreText and got me in contact with Scribd’s CTO Jared Friedman. We both were excited by the prospect of doing something on iOS that had never been done before, both technically as well as for the user experience.

The basic approach that almost everybody in our industry takes when it comes to displaying rich text is to use UIWebViews. Which leads to several problems because Apple does not let us at the inner workings of WebKit which ticks inside of these web views. Instead you have to resort to complicated hacks to find out things like the content size of a web page or inject JavaScript commands to implement any sort of non-standard communication.

Oh and it is slow to load, takes enormous amounts of memory and is almost impossible to customize visually if you don’t want to swizzle methods or hack the view hierarchy. Long story short: UIWebView sucks. Ok, you get the picture, now let me tell you how the grown-ups do it.

With iOS 3.2 Apple brought over CoreText to the mobile platform. CoreText brings you NSAttributedString which is basically an NSString where certain areas of text can have different attributes. These attributes – like a certain font or text color – are contained in NSDictionaries that have a certain range for which they are valid. Problem is, Apple forgot to include the initWithHTML method that exists on OSX. So on Macs you can easily create an attributed string from an HTML file and render this in a CATextLayer. Not so on iOS, where you have to painstakingly construct the attributed strings yourself and not just that, CATextLayer exists, but it ignores certain attributes altogether to make it practically unusable.

Enter NSAttributedStrings+HTML, my Open Source project to make it all much easier. It contains three major parts:

1. A category for NSAttributedString that gives you an initWithHTML
2. Objective-C wrapper classes to wrap CoreText objects
3. UI Classes to render this rich text

The main reason why I chose to make it FOSS is because as a single person I can never hope to implement all of HTML. If I had chosen to make this a for pay software people might have felt entitled to get certain things implemented. Instead, on GitHub, they can make they additions they need and can send me a pull request to merge their changes into the master to benefit all.

Scribd is a big believer in Open Source and so you can thank them for sponsoring the project for the past 3 months. Almost all the commits I made during this time came out of that. A few of the most recent ones came from John Engelhart who famously created the fastest JSON parser of the world: JSONKit. BTW, John recently signed up full-time to work on Scribd’s iOS Team. And so can you, they are now hiring more than ever, especially since Sam Soffes moved on to new horizons.

Personally I am too old and too married here in Austria to move to San Francisco, so I’ve been working under contract with Scribd. Because of this our contract contains three levels of licensing in to cover the various levels of ownership and rights of the source code involved:

• Open Source – code is owned by me, licensed publicly, Scribd would pay for the continued development necessary to provide the foundation for the other levels
• Proprietary – there is a great deal of proprietary code I developed on top of the open source. This code is owned by Scribd.
• Cross-Licensed – a few bits of code are owned by me and developed for use in my rich text editor. These I need to own so that I can sell it in other components, but Scribd has an automatic license.

At this point I probably should give you a demo of the app so that you see what I am talking about.

http://www.youtube.com/watch?v=OmIIxXFhBvs

All the rendering you see is in the Open Source project. Everything related to user interaction is the proprietary part. To give some examples for Scribd proprietary tech: dynamic switching between paging and scrolling mode, fast font resizing and style changing and multi-threaded layouting. Imagine me tinkering, programming, experimenting for the better part of two months to get the performance and responsiveness to where it is today.

How Float Got It’s Name

Let me also tell you the story of how Float got it’s name because this is closely related to this topic. Scribd’s CEO Trip Adler had a certain vision of how the ideal reading experience would look like. He felt that the user/reader should be able to decide for himself whether he preferred scrolling or paging through text. This is what he called “the Floating Reading Experience”. The user should be able to float around without such artificial bounds.

I have to admit – at that time – I was unsure how I would pull that off, secretly I thought Trip was crazy, pardon the pun: tripping. But I kept experimenting, tinkering, threw away a couple of failed approaches until I ended up with what you see today. Lo and behold, it is amazingly close to the original vision which is why Scribd rebranded the app as Float.

I learned one thing from this experience: as an engineer you might consider some ideas that your leaders come up with as crazy and impossible. But if you put enough work into it, even the impossible becomes possible after a while. The “Impossible” takes around 2 months.

There is one other ingredient in Float’s secret sauce that makes this possible. Scribd has developed amazing server-side technology to break apart fixed-structure documents of any kind and make then reflowable. For example they can dissolve a PDF and provide output where you can change the font size and pagination based on the device you are reading on. If there is something that is too complex to be put as pure text (like tables or a signature) then they render an image of that.

This is actually the much greater feat because at my end CoreText only has to worry about displaying this custom “clean room” HTML as opposed to having to worry about malformed tags and what not. I keep telling people that NSAS+HTML is great if they control the HTML and this is the perfect example. I know exactly which tags can come out of this reflow process and this predictability is key in having the display of text work all of the time.

So these are the two main ingredients that make Float unique and miles ahead of any kind of competition. And of course it gives me as a contributor a good feeling to be associated with a company that is in this position.

Try It Out Now!

Only time can tell if users will understand and accept how revolutionary Float actually is. But from a technical perspective I can honestly say that we did all that is humanly possible to make the greatest app (and service) we can. Please try it out now, it’s free. And once you have sufficiently marveled at it, let us know – preferably publicly – how you like it.

For my new version of Summertime I am building a time zone picker. You can get the known time zone names from the NSTimeZone class, but unfortunately Apple does not give us any localization of these. The localizedName:local: method gives you localized names of the time zones itself (e.g. “Pacific Standard Time”) in various formats. But what I found to be missing is a way to have the geo names localizable as well.

If I have my iPhone set to German I want to find my timezone by entering “Wien”, not “Vienna”.

My initial thought was to keep this to myself, but since I only speak German and English I can never hope to have the translations be perfect unless I would pay several translators to comb through them. And you know, Google Translate is great, but not 100%. So I started a new Open Source project on GitHub: NSTimeZone+Localization which aims to remedy this.

I wrote a quick command line tool which passes the knownTimeZoneNames through Google Translate for German, Spanish, French and Dutch as these are the languages that Summertime is currently localized in. More languages can be easily added the same way, but in general I would want to ask you a favor: if you speak any of these languages please have a quick look if you see any spelling or translation mistakes.

I had a modified version of LKGoogleTranslator handy which I had to modify again to include my API key. This still uses version 1.0 of the API while 2.0 is a newer one, but since it worked I did not bother.

#import "LKGoogleTranslator.h"
int main (int argc, const char * argv[]) {
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
 
	LKGoogleTranslator *translator = [[LKGoogleTranslator alloc] init];
 
	NSMutableString *string = [NSMutableString string];
 
	for (NSString *timezoneName in [NSTimeZone knownTimeZoneNames])
	{
		// replace slashes with commas, works better in Google Translate
		NSString *searchName = [timezoneName stringByReplacingOccurrencesOfString:@"/" withString:@","];
		searchName = [searchName stringByReplacingOccurrencesOfString:@"_" withString:@" "];
 
		NSString *translation = [translator translateText:searchName fromLanguage:@"en" toLanguage:@"nl"];
		translation = [translation stringByReplacingOccurrencesOfString:@"," withString:@"/"];
		translation = [translation stringByReplacingOccurrencesOfString:@"/ " withString:@"/"];
		translation = [translation stringByReplacingOccurrencesOfString:@" " withString:@"_"];
 
		[string appendFormat:@"\"%@\" = \"%@\";\n", timezoneName, translation];
 
		NSLog(@"%@ = %@", timezoneName, translation);
	}
 
	NSLog(@"%@", string);
 
	[translator release];
 
    [pool drain];
    return 0;
}

The resulting string blog I just pasted into strings files. With the help of a small category extension on NSTimeZone you can now get the localized geopolitical name.

#import "NSTimeZone+Translation.h"
 
@implementation NSTimeZone (Translation)
 
- (NSString *)localizedGeoName
{
	return NSLocalizedStringFromTable(self.name, @"LocalizableTimezones", @"NSTimeZone+Translation");
}
 
@end

This works because the localizable strings files for this are called LocalizableTimezones.strings. Instead of the regular NSLocalizedString macro I’m using NSLocalizedStringFromTable with the middle parameter being the name of the strings file minus extension.

You can use this functionality in your apps if in turn you feed back errors to me so that I can correct them. If you want additional languages to be added, let me know. I’ll make the same starting list via Google Translate for any newly requested languages and then you can manually polish them.

One of the things that people know me for is for continuing to develop on MyAppSales, my favorite tool to download and keep those elusive sales report from iTunes Connect. It’s always been a hobby and until now I’ve allowed people to access the source in exchange for a mandatory donation. This went on for almost two years now.

Those donations never made sufficient money for me to pay for professional development. But I felt that I had to ask the approx. 500 people on the google group about their opinion as they might see their donation as a purchase and not like the idea of this software now being available for free. Boy was I wrong. Resoundingly people voted “+1″ for going OpenSource.

So here it is. It’s Open. Read on for how I moved the repo from my SVN server to GitHub, including the entire revision history.

I wanted to move the entire history so far onto GitHub so that it was preserved. Fortunately this capability is already built into git. It will get all revisions and convert them into commits.

Create a local folder and init for a git repo:

mkdir MyAppSales
cd MyAppSales
git init

Set an SVN URL and fetch the entire history with all revisions. Amazing! This seems to be built into git. On the first go I made the mistake of importing the project root including trunk, tags and branches. So I aborted it, since I only wanted the contents of the trunk.

git svn init https://svn.cocoanetics.com/myappsales/trunk
git svn fetch

Set the remote repo and push all there: The pull was necessary because I had commited and pushed a README before the import.

git remote add origin git@github.com:Cocoanetics/MyAppSales.git
git pull origin master
git push origin master

I almost couldn’t believe how easy it was, but from the upload taking a couple of seconds and not getting an error message I figured that it worked flawlessly. Upon checking the commits section of the online browser I found that amazingly the entire commit history was present. Made me smile to see the first import dated March 29, 2009.

One thing to notice is how my user name changed every time I moved to a new server. Commits by “oliver” where done to the first repo which my friends at Spielhaus hosted. Then I moved to my new server, changing to “odrobnik”. Now on github I’m “Cocoanetics”.

Also it turns out that the project is desperately in need of some cleanup work regarding where files are located in the folders. Class files in the root, images all over the place.

Two more things I had to do: I copied a license and a .gitignore file so that it would not push some unwanted files. The first is created by OSX, build folders we never want and the last two are user settings which you don’t need either.

.DS_Store
build
*.mode1v3
*.pbxuser

I have several updates to this source still in progress on my Laptop. I’m working on migrating the entire database to CoreData because at present bearable performance is only achieved by caching to compressed files. So this is what I’ll bring to this project next.

It’s also my hope that with the source being now open that this will encourage current and future users to contribute. The repo is located at https://github.com/Cocoanetics/MyAppSales. And of course donations are still very welcome, my PayPal address is oliver@drobnik.com.

After having released SpeakerClock, a speech countdown clock, I got a couple of suggestions, one was not about the app itself, but about how you could stand the iphone on it’s side.

Kevin Jamison: “Now we just need to come up with a simple little stand similar to the iPod Touch uses to hold it in a tilted landscape mode. Maybe rubber coated feet so it won’t slide if you are using a podium.”

I remembered that every once in a while the blogosphere brought to light yet another clever way of solving such problems. A quick search on Google and YouTube yielded a couple of great solutions. Some that you can purchase , same that you can make yourself at little to no cost.

First let’s look at the professional solutions which typically cost between $5 and $10. After we’ve set the benchmark we’ll explore how we can achieve landscape stability ourselves. Building something useful out of physical things might be a welcome distraction from hours of coding Cocoa.

Professional Stands

MovieWedge

from MovieWedge.com

Crabble

from Seskimo:

The Crabble is the only wallet-sized stand to offer an easily and continuously adjustable landscape viewing angle, ranging from 45 to 90 degrees, and, with its rubber claws, it is the only wallet-sized stand to give you that grip.

GoGoStand

from GoGoStand:

The GoGoStand is a plastic folding stand that you can tuck away in your wallet for quick access whenever you need to prop up your phone or gadget. This stand was designed for all generations of iPhones and iPod Touches, but also works with many other phones including the Android G1, Droid, Palm Pre and more.

Travel Stand

from Griffin Technology:

Travel Stand flips open to safely hold your iPhone or iPod touch in landscape mode for comfortable, hands-free viewing of your movies, videos, music, photos and more. When the show’s over, wind your earphones around the included cord wrap and flip the Travel Stand closed. Compact enough to slip into your pocket, Travel Stand is the perfect blend of stand and storage.

Do it Yourself

Paperclip

Mini-DV Tape Box

LEGOs

The maker made the building instructions available at my request.

Paper

$100 Bill

Same folding technique as the paper version, by Enrique Pardo.

Laser-Cut Acrylic

Make Magazine has the guide, all you need is access to a laser cutter…

Credit Card

All you need are a pair of scissors, some tape and an expired credit card. By Rod Porterfield.

Cardboard Sleeve

A cup sleeve that you would get free with coffee at – say – a Starbucks.

Paper Cup

A dixie cup with optional cancer society rubber wristband for additional grip.

Business Card

By far the easiest, most spontaneous possibility, no tools required.

Conclusion

Having reviewed quite a few ideas that other people had for either sellable products or DIY options I think that I would go for a solution that I could carry amongst my credit cards. There the GoGoStand looks to me the best professional solution even though the media is hyping the MovieWedge. But I don’t see that fitting into my wallet.

Amongst the DIY it depends if you have time and tools at hand. The variant that has the most likelyhood of being portable is one using nothing but a business card. You don’t need any tools for it and the folding necessary is easier to remember than for the bill or paper versions.

Please comment below what you’re using or if you found a version that I have not covered.

It is with a certain pride that I am announcing that truly was involved with the creation of an app that made it to the top. Well, at least in Austria and Germany, and at least in a category.

I’m referring to Infrared Photography which is an app that was contracted out to us by none less than Michael Görmann, professional photographer and in need of a signature app that he could hand out as a sort of business card. Via a short detour he arrived at our virtual doorstep and was promptly served a cute little app.

To put the credit where it is due and put it more precisely the programming itself was carried out by my brother-in-law whom I had the pleasure to help out here and there and generally have a mentoring eye over, when it comes to iPhone programming. He was and still is to this date an export for server-side Java, Android and Symbian. But about iPhone development I can proudly say: “He got that from me.” His name is Rene Pirringer and his company is ciqua. We are closely working together on iPhone projects.

Infrared Photography is unique in a way because it gives you an endless supply of beautiful black&white photos which where made with expensive Infrared Euqipment and by the unmatched eye of Mr. Görmann. So unique in fact that Apple approached him and asked him to put a couple more screenshots online, because they were planning to feature it. And so they did.

You could see an immediate jump in downloads, but for the most part in Germany and Austria where it was featured. It seems that those countries are served by the same iTunes team at Apple. Being featured like this glued the app to the first place like you can see here:

We are thankful for having had this chance find us and all you out there who keep downloading the app. It’s an amazing reference for us.

Opinions differ on what constitutes “Augmented Reality”. This can generally range from sophisticated 3D graphics overlayed on a stereoscopic view of the word to simply overlaying floating icons related to a location over a live camera view. In between some also count as AR when you have a special marker in your hand which you webcam can track and inserts a small 3D model on top of it.

On the iPhone it is currently not possible to do any sophisticated motion tracking or shape recognition was you can only get live images via a backdoor which Apple recently kind of opened just a bit so that useful applications like RedLaser or uStream can make use of it. Still it seems that the consensus is that you can only use images that have no UI elements over them as the workaround does captures of the whole screen.

I figured it would be an interesting addition to my Dr. Touch’s Parts Store and so I set up to create DTAugmentedRealityController. The goal for V1.0 is to provide a view controller which you can feed icons and geo locations and the view controller would overlay said icons over live video from the iPhone’s camera.

I’ve been working on this for 3 days now and because of popular demand I’ve documented each evolutionary step in a YouTube video per day.

Day 1

This was all about figuring out how to overlay my own views on top of UIImagePickerController. Turns out you have to do a bit of transform-magic to resize the video to be full screen, hide the UI elements and provide your own view for the overlayView property. Then I found that UIImagePickerController does not auto-rotate, so I had to apply my own transform. Maybe there is a way to get this rotation by subclassing it, but I have not tried that. I was thrilled to have a turning compass rose on top of the live video.

Day 2

Stuff started to come together. On a static horizontal plan I displayed my first floating icon. I chose Steve Jobs because his location in Cupertino should be every iPhone Developers place of silent thankfulness not to sound overly religious. But Mr. Jobs was the best augmentation I was able to come up with for the reality of my office. Don’t you think so, too?

Day 3+4

For the third episode I improved the engine tremendously.  First change was to pack all sensor-related activities into the DTRealitySensor class to have a central location for all sensing. Then I added rotation to the previously strictly parallel icon plan so the horizon would always stay parallel to the real one. Finally I was facing the challenge of implemented a true attitude independent compass.

The built-in compass has a major drawback: it only works properly if the iphone is flat on it’s back. Which is of little use for a serious component that I am working on. Thankfully I discovered a paper “A NewSolution Algorithm of Magnetic Azimuth” which gave me the necessary insight to use both the gravity vector as well as the raw magnetic vector to calculate a compass bearing which works for any iphone attitude.

Still to Come

I already have several customers lining up to get this component into their hands. Once of them has a deadline of January 12th. He claims “if I don’t get it by then, I’ll code it my self”. “Yeah, right!” would be my appropriate response now that I have seen how much work it is to get the basics lined up.

Version 1.0 of the component still has a couple for things to work out:

• Make a protocol that allows any UIView to be used for geolocated floating. You would add this view with a method to the controller and the controller would continuosly query your UIView via the protocol. This information would then be used to display the view on screen at the correct location.
• Invent some sort of dampening so that icons don’t jump around even though you are holding the iPhone still. Maybe have some detection on DTRealitySensor as to wether the iPhone is being moved or not. Still uncertain how to do this properly without loosing the generall responsiveness.
• Allow the user to set a maximum distance to show and maybe try to show icons smaller and foggier in the distance.
• There also has been a request to give the icons an altitude. Right now I am positioning in a simplified fashion. Implementing altitude would require that I start to calculate a pitch for looking vector.

Then there are already a couple of ideas for future versions, but I need serious support would I go to tackle those:

• Do some simple geometric recognition if possible and stabilize the view on detected lines.
• It would be great if we could detect shapes in general.
• It should be fairly easy to also overlay an OpenGL layer and display 3D models overlaid over the live video.

The final question that I’ve been asked a lot is what I am planning to charge for DTAugmentedRealityController. You have to bear in mind that I have so far invested 4 full working days (1600 Euros) into this project. Also with that much interest I have to make sure that I don’t make it too cheap and also make it expensive enough so that I can justify investing even more time. Finally AR seems to be a hot topic right now, especially on the iPhone. There are only very few well done AR applications, but I aim to be at the forefront in usability and ease of implementation.

So to get this settled, I chose 300 Euros, with a distinct upward trend. The first 5 customers will get it and keep it for that amount, after that I will reevaluate. Same terms and conditions apply as do for all other components on the Dr. Touch’s Part Store.

Now that I am able to work on my apps full time I finally get to fulfilling my user’s wishes. Amongst those, for iWoman, was a Calendar view. I did not want to use another person’s code and deconstructing something that Apple made proved to be extremely instructive.

Here a video demonstration of my result so far. Please disregard the incorrect labelling at the top. At least this proves that this is not fake.

The whole thing consists of three classes, a CalendarViewController which takes care of the general management of views, a CalendarDayView which derives it’s capabilities of responding to target/actions from UIButton and a CalendarHeadView which is everything you see at the top.

What makes it look special and professional is that you have multiple gradients at work. Besides of the obvious one at the bottom, there are black and gray gradients shading most of the labels. The regular gray background I custom draw in drawRect, but the blue selected box was too complicated so I opted to use a UIImage for that.

For most of the calendar calculations I needed an NSCalendar, so I made a helper class with category extensions to NSCalendar to give me things like first day of next month, first day of current month, number of days and so on. I found that I could quite reduce my memory footprint by reusing things. Like instantiating the NSCalendar just once for the whole view controller.

By far the trickiest business was to properly set up the grid of CalendarDayViews on an invisible view where I enabled cropping of children so that stuff could appear smoothly from below the gradient at the bottom. The animation itself is trivial, but it took some time to figure out that I would have to overlay the bottom or top line with alpha 0 with another set of days when going forward or backward and then animate it to 1 so that the selected day would move from being a gray rectangle (if you tap on a day that belongs to another month) to being blue.

For quickly switching between months you can tap and hold one of the arrows. For this I did not need any animations, so I pimped the method to set up my sheet with an animated: BOOL parameter. For regular switching this would be YES, for fast-switching it would be NO. Et voila!

That all worked nicely until I started to use it on device. I was asked about the performance on device, especially for fast-scrolling. I found that I could tremendously improve this by reusing the UIViews I am creating one for each day. Before my optimization all subviews to my sheet where removed from superview when they left the visible area. Since the sheet was the only view retaining them this also lead to their deallocation. Then I simply introduced an NSMutableSet to hold onto removed Views and also reuse CalendarDayViews from there. This speed things up quite a bit. Reuse if you can as much as you can!

Next things to do on the CalendarView control is to add a table view below it, so that I can query a data source for table view sections and rows pertaining to the currently selected date. I plan on using this in version 2 of iWoman, the first major update to my best selling app. I owe this to my customers since I did not get to update this app in over a year.

CalendarView will be the first of many things in my “Dr. Touch Parts Store” where you can obtain high quality components to use in your own projects for a small donation. The concept is that this small amount of money pays for the time I need to continue to improve it, support it and help you with implementing it.

Applyzer.com has a SOAP-based API, which is also how AppViz is getting the ranking data. I’ve been putting it off for a while, but finally I tried to find a painless way to generate proxy classes for the webservice. I failed.

Well, not entirely, there IS a plethora of frameworks and toolkits that promise to automatically generate proxy classes for you. Only problem, these are either C++ based, or use libxml or use other xml parsers like are present in CoreFoundation on the Mac, but not on the iPhone. Also the code that gets generated by tools like wsdl2objc gives me the creeps.

A proxy class is a class that looks like a normal class that you can instantiate and use it’s methods. But behind the scenes it packages all calls in SOAP or HTTP GET/POST, calls the server, retrieves the answer and unpackages the returning SOAP envelope. Basically you program against the web service as if the functionality offered is part of your local code.

So I started the “Kernseife” project. Originally I was tempted to call it SOUCH, like SOAP Touch, but I found this term to by synonymous with the pain of being kicked in the family jewels. Second idea was to call it CoreSOAP because it’s something that I would have expected for Apple to provide. But then I figured, how about the German translation? Core means “Kern” and Soap means “Seife” and the Soap product called “Kernseife” is synonymous with a very healthy and very thorough cleansing. Just what I had in mind for this project.

Don’t tell me now that “SOAP is dead” and now everything is JSON, XML+RPC or REST. Most of the web service world around Microsoft, IBM, SUN and Java knows how to speak SOAP and since it’s the standard transport for the .NET framework based web services I run, there will be no discussion regarding SOAP’s viability. It’s there, I am going to talk with it.

So far the project is able to generate proxy classes for very simple web methods:

• A single return value per method
• No complex types
• only string, int, double, dateTime
• Supported transports are SOAP 1.0, 1.2, HTTP POST and HTTP GET
• No error handling
• No SOAP exception handling

Because there is still lot of work necessary to make it complete I chose to make it Open Source and created a project on Google Code. I will continue to update and improve the code as I need for my ongoing projects and I am happy to share development with anybody who has some ideas on how to do stuff even smarter. Specifically the next improvements I am looking for are:

• Add SOAP exception handling, possibly using NSError or objC exceptions
• Add complex data types by autogenerating classes or structs for them
• Add the conversions for the missing simple datatypes in the XML schema
• Unit Testing against a variety of WSDL files
• Documentation and Samples

To get the code into your hands you can check out a working copy via Google’s instructions. If you’re interested in contributing then contact me so that I can make you a project member.

When you have downloaded and built the code you get a Mac binary called Kernseife which has one parameter: a file path or URL.

$ ./Kernseife http://www.drobnik.com/services/testservice.asmx?wsdl

This analyzes the WSDL file and outputs a the .h/.m files for a proxy class:

These ports where found:
- TestServiceSoap
- TestServiceSoap12
- TestServiceHttpGet
- TestServiceHttpPost
 
Writing class files for first port
Done.

Have a look at the header to check if all the input and output parameters could be generated:

// TestService.h
 
#import
#import "WebService.h"
 
#import "NSString+Helpers.h"
#import "NSDate+xml.h"
 
@interface TestService : WebService
{
}
 
- (NSString *) helloWorld;
- (NSDate *) dateInWithInDate:(NSDate *)inDate;
- (NSDate *) jetzt;
- (double) addWithOne:(double)one two:(NSInteger)two;
 
@end

Add these files to your project and also add all the files in the “xml” group of the Kernseife project.

Then you should be ready to test the proxy class.

// #import "TestService.h"
 
TestService *service = [[[TestService alloc] init] autorelease];
 
NSDate *nun = [service jetzt];
NSLog(@"Time now from service: %@", nun);
 
NSDate *dann = [service dateInWithInDate:nun];
NSLog(@"Pass-Through Time: %@", dann);
 
double ret = [service addWithOne:1.5 two:2];
 
NSLog(@"Added double and int: %f", ret);

From the initial feedback I have gotten for the basic premise of Kernseife I can tell that there is a tremendous need for such a project. So I appreciate your feedback and/or help going forward.