Chomping at Chandler

This is what we’re going with unless there’s any outstanding problems. W00t!

The Chandler logo

Its still a work in progress, but I really like the latest rev!

What do you think?

The Google Mac blog talks about accessing various Google APIs from Cocoa, including the Google Calendar API.

Nice.

This was sent to the chandler-dev list and I wanted to have a nicely-formatter version handy.

---

Profiling Chandler is tricky because there are lots of moving parts. Here are a few tips and tricks that come to mind:

1. To compare numbers, always use the same data. Recreating your repository everytime changes the data in subtle ways because you get new UUIDs everytime and the order of things, random in mappings, is conditioned by the hash of their keys, very often UUIDs. Instead, load and prepare your data, backup the repository and always restore it when re-running the testcase.

   rc -r ~/Desktop/__repository__.001 (or whatever your path may be)

2. Preload all items, that way the vaguaries of item loading are out of the way, unless that’s what you’re interested in. Running check() loads all items into the UI view.

   rc -r ~/Desktop/__repository__.001 --undo check

   –undo check runs check() and undoes bogus versions until check() passes at startup time. If check() passes right away, as I’d expect from a clean restore, –undo will undo nothing and you will have started your chandler will all items pre-loaded.

3. The -P flag using the perf tests produces profiles that crash hotshot (at least, for me, on my Mac). Also, with -P you cannot be sure how much of the perf test cruft you’re profiling along with it. Know what you’re measuring, and add the profiling hook yourself.

   Find the block of code or call you’re interested in and wrap it with a prof.runcall() call, creating a local function for the block of code if need be. For example, to profile import, I just change ImportExport.py around line 267 as follows:

   import hotshot
prof = hotshot.Profile(’import.log’)
collection = prof.runcall(importICalendarFile, fullpath,
        self.view, coll, filterAttributes, activity, None, logger)
prof.close()

   Then process the resulting import.log file with the hotshot APIs as is done, for example, in my t82.py file included below. Really, how to process the hotshot output is up to you, whatever works best for the kind of output you’re looking for. t82.py, when run the first time on a .log file, produces a .stats file from it which is vastly faster to reload. This is can be a huge timesaver when profiling things like a large import, for example.
4. Remove commit() calls. Currently, our code includes commits() in less than ideal places and this has the effect of slowing things down considerably. commit() is an expensive operation that we’d like to do *between* user actions instead of *during* user actions. Measuring it during a user action is pointless unless it is critical to the action.

   For example, the criticality of calling commit() at the end of CalendarCanvas.CreateEmptyEvent() is dubious. John has offered to review all such commit calls and remove them if possible, leaving it to the user event that started the action to do the commit, if needed.

5. Instead of only looking at the amount of time spent doing things, consider how many times they’re done. Currently, I sense that there are a lot of perf gains to be made in the app by doing things less instead of faster. Why are we loading 41 images when creating an event in the calendar, for example ? This leads to the next point.
6. The app loads things lazily. Items are loaded lazily. Images are loaded lazily. If you see 41 images loaded, the next thing is to profile creating a second event instead. Indeed, the calendar doesn’t load 41 images the second time. Run the action multiple times so that all the init cases are out of the way. But, we’re still calling MenusAndToolbars.appendDynamicBlocks 336 times nonetheless… something to look into there even if we’re only spending 0.007s there, the fact it’s called that often may lead you onto something where significant time is spent overall.
7. When communicating about profiles, don’t send the .log files around, they can be huge. Use a .stats file instead. As said before, the t82.py file below creates a .stats file from a .log file whenever it is invoked with a .log file. Don’t send a text version out either, the .stats file can be opened with hotshot APIs and queried in various ways to look for callers, callees, different sort orders, etc, etc. The Stats class has several interesting APIs to poke around a .stats file: docs.python.org/lib/profile-stats.html

That’s all I can think of for now. See my t82.py script below.

Andi..

---

import os
import hotshot, hotshot.stats, pstats

def cum(stats, length=50):
    stats.sort_stats('cum')
    stats.print_stats(length)

def calls(stats, length=50):
    stats.sort_stats('time', 'calls')
    stats.print_stats(length)

def show(file, call='cum', length=50):
    if file.endswith('stats'):
        stats = pstats.Stats(file)
    else:
        stats = hotshot.stats.load(file)
        stats.strip_dirs()
        stats.dump_stats('.'.join((os.path.splitext(file)[0], 'stats')))
    if call == 'cum':
        cum(stats, int(length))
    elif call == 'tot':
        calls(stats, int(length))
    else:
        stats.sort_stats(call)
        stats.print_stats(int(length))

    return stats

if __name__ == "__main__":
    from sys import argv
    show(*argv[1:])

A Chandler logo

I have to say I like this logo very much, certainly compared to any others I’ve seen. I think we should change the Triage icon to use this.

Here’s a little csh script that puts properly-named ics versions of all your ical files on the desktop.

#!/bin/csh -f
set ical_sources_dir=~/Library/"Application Support"/iCal/Sources
foreach dir ("$ical_sources_dir"/*.calendar)
    if(! -r "$dir"/corestorage.ics) continue
    set cal_name=`defaults read "$dir"/Info Title`.ics
    if(-e ~/Desktop/"$cal_name") then
        echo "File ""'""~/Desktop/$cal_name""'"" exists, skipping"
    else
        ln "$dir"/corestorage.ics ~/Desktop/"$cal_name"
    endif
end

Update: Here’s the Python version.

#!/usr/bin/env python
from glob import glob
from os.path import expanduser, expandvars
from os import system, popen, link
from string import strip

ical_sources_dir=expanduser("~/Library/Application Support/iCal/Sources")
for dir in glob("%s/*.calendar" % ical_sources_dir):
    p = popen("defaults read \"%s\"/Info Title 2>/dev/null" % dir)
    cal_name = strip(p.readline()) + ".ics"
    p.close()
    if cal_name != "":
        try:
            link("%s/corestorage.ics" % dir, 
                "%s/%s" % (expanduser("~/Desktop/"), cal_name))
        except:
            pass

When I work on day-to-day Python scripts, I wonder if there are a lot of script writers who start every script with something like:

from sys import *
from os import *
from os.path import *
from string import *


This would make writing Python more Perl-like in its availability of many functions through simple calls.

I just made my first successful conflict in Chandler’s PyShell. Woo!

PyShell 0.9.5 - The Flakiest Python Shell
Python 2.5 (r25:51908, Jan 31 2007, 21:59:50)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>uuid="27e5e1d4-53ad-40e4-860d-503a137a0806"
>>>item = view.findUUID(uuid)
>>>from osaf import sharing
>>>sharing.SharedItem(item).add()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Users/rae/work/osaf/chandler/chandler/parcels/osaf/pim/stamping.py", line 112, in add
    "Item %r already has stamp %r" % (self.itsItem, self)
StampAlreadyPresentError: Item <Note: 27e5e1d4-53ad-40e4-860d-503a137a0806> already has stamp SharedItem(<Note: 27e5e1d4-53ad-40e4-860d-503a137a0806>)
>>>sharing.SharedItem(item).generateConflicts()
>>>item
<Note: 27e5e1d4-53ad-40e4-860d-503a137a0806>
>>>cstates = sharing.SharedItem(item).conflictingStates
>>>cstates
<DBRefList: It's a New Event.osaf.sharing.shares.SharedItem.conflictingStates<->conflictFor>
>>>conflictsGenerator = sharing.SharedItem(item).getConflicts()
>>>for conflict in conflictsGenerator:
…    f = conflict.field
…    v = conflict.value
…    p = conflict.peer
…    print "[%s]=%s (%s)" % (f, v, p)
…
[location]=San Jose (conflict@example.com)
[title]=XYZZY (conflict@example.com)
[body]=PLUGH (conflict@example.com)
>>>

.. one small step for conflicts..

TechCrunch has a review of the Nokia N800, an iPhone-like device available now that runs Linux and is dedicated to supporting open source. The reviewer decries the lack of a PIM.

Hmmm…..

OSAF has been investigating new names for the combined Chandler and Cosmo product. While we were listening to a presentation on this in today’s staff meeting, I made up a few of my own, using both this and another web2.0-name-generators, as well as my own head.

Snapcat Livevine Zatz Buzzchat Chatterspot

LiveCal Skimia Riffr* Yakimbu Sked

OpenKitchen CalTalk eEsther Meetr* Plannr*

Jeeves CalHub Itemz Veni Vidi

Vici Tumblr* Connectr* ChitChat Arrangr*

* - indicates maybe too-close-too-Flickr name

Leave your ideas here!