Zeitgeist daemon extensions explained

While working on this Zeitgeist bug #612344 I recalled that Stuart Langridge had once quoted

Zeitgeist has extensions. These aren’t brilliantly documented yet, but you can drop a Python file into .local/share/zeitgeist/extensions and if it’s got the right sort of class in it then that class will get run as a part of Zeitgeist.

Sometimes it is said that code is the best documentation. This becomes the only solution when there is no documentation available on wikis or help pages. When working on that bug, I started hunting for documentation on the web. After a few minutes I understood that looking at the extension code is the best way to go ahead.

How extensions are searched and loaded

Every extension needs to inherit Extension base class which is present in _zeitgeist.engine.extension namespace. So basically the first line has to be

from _zeitgeist.engine.extension import Extension

When the daemon is started, it looks for the environment variable named ZEITGEIST_DEFAULT_EXTENSIONS to hunt the extensions which are supposed to be loaded. If this is not set then extensiondir variable contains the path of all the extensions. You can know about this

>>> from zeitgeist._config import extensiondir

>>> extensiondir

‘/usr/share/zeitgeist/_zeitgeist/engine/extensions’

It even searches for the extensions in your local path. This means an extension can be enabled globally as well as locally. The path in your local home directory is the one fetched by

>>> from _zeitgeist.engine import constants

>>> constants.USER_EXTENSION_PATH

‘/home/manish/.local/share/zeitgeist/extensions’

All these files are then scanned for class which inherits Extension class.

The structure of an Extension

Extension has a few methods

1. pre_insert_event(self, event, sender) – The control passes through this method before every time an event is going to be inserted. So the method gets the instance of the event and the dbus Busname of the client which logged the event. If it wants to block the event from being inserted, this method simply returns None. The fields of the event can also be modified if required (probably useful for privacy control). Before using the event, please check it for None since another extension which has the event instance might have set it to None.

For this hook, the event enters one extension and the output of this extension(ie. event) is then passed to another extension.

2. post_insert_event(self, event, sender) – Same as pre_insert_event except that it is called after the event has been inserted. In this method the nothing needs to be changes or returned as the event has already been inserted.

3. get_event(self, event, sender) – This method/hook is called everything an event is fetched and to be sent to the client. This hook also behaves like pre_insert_event in the sense that the even returned from this method is then passed to other extension’s get_insert_event and can be changes similarly to pre_insert_event.

4. pre_delete event(self, ids, sender) – This method/hook is called before an event is to be deleted. Please note that the input argument is not event but a list of event ids. Event id is the unique way to identify an event within a zeitgeist database instance. It does not return anything

5. post_delete_event(self, ids, sender) – This method/hook is called after the event having the event ids in the input argument ids has been deleted. It returns nothing.

Example extensions

By default zeitgeist has two official extensions which is shipped with the daemon. It is

The project Zeitgeist extensions is for hosting the extensions. One excellent extension for zeitgeist is full-text-search extension which can also be installed in Ubuntu by installing the package named zeitgeist-fts-extension

Right now Blacklist, Datasource Registry and fts also expose their functionality via DBus. For that, the only magic you have to do is to also inherit dbus.service.Object

Conclusion

I don’t think writing zeitgeist extension is a rocket science from any angle. All your need is patience and courage to poke the zeitgeist engine developers on IRC.

Happy hacking!

Zeitgeist in my blood

For the last three months, I have been working with the Zeitgeist team for achieving the goal of a semantic Linux desktop. Right now I am not much involved in the daemon, but not for long.

Basics

Daemon is just one part of the solution. A server is of no use if there isn’t any client to use it. Similarly now what zeitgeist needs is a deep integration with most of the applications and the shell. Unity is already on it’s way rocking ahead with Unity Places. There are many applications for which plugins have been developed to push events to the daemon. Apart from these Banshee plugin is there in banshee-community-extensions and Rhythmbox plugin is already upstream in the source tree.

The awesome team

I think this is my first experience working in an open source project with a team. Believe me, the team is awesome. To put it straight – MIND BLOWING. Mikkel is the architect who is the role model for being always right. Seif is always encouraging. Markus Korn’s keen eyes on details and the knack for extreme review sessions make sure that the mistake I commit is always the first and last time. Same goes for Micheal Hruby.  Siegfried does a lot of heavy-lifting and is the person I catch first when some information is needed about the daemon/API/backend.

The development process is pretty much democratic. Before any new feature is added or before something is finalized a lot of discussion takes places. There is a voting in spirit with democratic values. This is a sign of QA. No one is allowed to push a change to the trunk without any other person reviewing it. The patches are then fine tuned and ultimately the merge happens.

Seif’s role is that of a manager. No! I am not talking about the managers in your office. Manager role here is about keeping an eye on the overall thing so that every other person can concentrate on their work. He is also our community manager. That’s why you can see his face everytime zeitgeist is involved. Even though he started the project, we don’t have a benovalent dictator.

back to top

Role

When I started contributing, everyone in the team mostly worked on python and C/Vala applications. This left a gap for integrating zeitgeist in CLI based applications. Randel Barlow had tried sometime back, but he hit the rock(ndesk-dbus) and stumbled over. (Die ndesk-dbus). I took over the work of creating a CLI wrapper over DBus API. Finally after some lengthy hack sessions and constant hair-pulling, I managed to finish the work with help of Mirco Bauer who created the build script(autofooling in our language). All the time during the development, ndesk-dbus was at the receiving end of my rage.

back to top

Packaging

Jo Shields came into picture when he did the necessary work for packaging the library. Well, after the decision to make Banshee the default media player for Natty. most of the focus has shifted to slim down Banshee. This needs a lot of work and the Debian CLI team became very busy. The only work left was packaging the library.

Yesterday I caught Jason Gerard DeRose on #novacut and asked for help to package it. Even though he had never packaged CLI based applications, he offered his help to the best of his capabilities. I worked till 5 in the morning to get the build working. His step-by-step guidance and explanation made me his instant fan. Jason – we need people like you. Seriously! Be convinced that novacut will receive all the help possible from me.

Here is the PPA. This is my first package(for Maverick and Lucid), so please be calm. It should not screw up your system. Be relaxed! The name of the package is libzeitgeist0.1-cil and if you want to build the dataproviders yourself, then you need to get libzeitgeist-cil-dev

back to top

Banshee and Tomboy Dataprovider for the impatient

If you love Banshee and Tomboy and feel that they need Zeitgeist love, then here is a dirty solution. I am providing the binary files for it. Manually dropping the files in the respective locations is a bad practice. Actually getting these dataproviders packaged can take some time, so for those people who can’t wait can try downloading both of these files. Remember to delete them when these are packaged. I will warn you again next time when the package will land.

Banshee dataprovider [code]: Put the file in /usr/lib/banshee-1/Extensions

Tomboy dataprovider [code]: Put the files in ~/.config/tomboy/addins

back to top

Disclaimer

If you want to submit your objections to mono, please go and talk to mono developers. For a non-techie end user, mono hardly matters. For them having an application with better desktop integration matters. I work for them. I serve them. Again if you think my work is going to be the reason for the upcoming apocalypse on this planet, do feel to contact me via Launchpad page. Please keep your mails polite.

You can ignore the title of this post if you were about to take it literally.

back to top

FUD and facts

Mono is a very controversial topic in Free software world. We have no shortage of proponents and opponents. There are people who want to clean the FOSS world by removing mono from the picture and there are developers claiming that they are comfortable with mono, so it’s fine for them to use it.

Now imagine a situation when you try to “get the facts” but the information itself is wrong? Guy Van Sanden wrote a post about Get the facts on mono which attracted a lot of comments. The follow up, Cleaning mono from your system (Ubuntu) was full of huge mistakes. If you want to spread FUD, please do it properly. As he said, entering this command

$ sudo apt-get remove –purge mono-runtime mono-common libmono0 mono libmono1.0-cil libmono2.0-cil

will clean all your sins and you will attain salvation. The catch was probably he copy-pasted from somewhere else. Just by a quick glance today morning I could make out that the whole command won’t work. There is no package named “mono”. Lower down the comments Debian/Ubuntu mono packager Jo Shields leaves a comment explaining why he is *wrong*.  mono-common too does not exist. libmono2.0-cil doesnt have any Microsoft namespaced packages. As per the description of this package

This package contains various Mono libraries for CLI 2.0:

  • Mono.CompilerServices.SymbolWriter
  • Mono.Http
  • Mono.Web
  • OpenSystem.C

Apart from that it looks like he did not even try out mononono since all the comments point it out and he actually acknowledged that it doesn’t work.

Flames

All over these years I met two kind of people who talk about Mono

  1. MONO IS GOING TO DESTROY FOSS AND LINUX. ALL MONO PACKAGES VIOLATE MICROSOFT PATENTS AND CAN KILL YOUR CAT AND WILL BLEED YOU TO DEATH
  2. Hey, are you sure all the packages/components are protected by CP? Can you provide some hints on how packaging is done to avoid or lessen the patent risks? These are the patents #xxxxx and #yyyyy which mono can possible infringe. What care can you take to avoid them?

The sad part is that most of the people I met use language 1 and their statements contains a lot of speculations and hardly any proof. Finding people of kind 2 is tough, but they are overshadowed by the people of kind 1. If you are one of the person of kind 2, come over on IRC and let’s have a sane discussion on mono based on facts and proofs rather than speculation and FUD. I would love to be influenced. By the time I simply ignore trolls of kind 1

Hate and Love

There was a time when I take any development from a critical angle.  Like many others the decision was either white or black (Love or Hate). Well, I did not spread and FUD but black-white did not last long. I gave up hating anything. Now I don’t hate any technology, any language, any library etc. I have my like and dislike which does not map to love and hate.

Due to Android-Google-Java-Oracle incident you can see people suggesting that Java is patented which people should not use it. I don’t share that armchair-advisor kind of mentality. If I want to use it, I will use it. If it’s not fit for my use, then I won’t. The would have more to do that just hating technology.

For me technology is a tool for getting my job done and making my life easier. It isn’t a religion which I need to follow. I nearly stopped caring about haters. Why? I remember once when David Siegel recalled “Haters gonna hate” in an interview.

Strong passions is fine since loving something doesn’t automatically means hating other things. There are other colours in this world apart from black and white.

Release: Zeitgeist-sharp 0.1.0.1 “Stark”

How many of you love Tomboy? How many love Banshee? F-Spot? Some of you might not but some of you might swear by it. Yes, for people like you – how about if your favourite app is sweetened by the spoonful of zeitgeist love? After call for testing, the zeitgeist-sharp team was unable to find any major issues with the library. So, on behalf of the zeitgeist-sharp team, I am happy to announce the first release of zeitgeist-sharp 0.1.0.1 “Stark” which is the client wrapper over Zeitgeist’s DBus API.

So what does this mean? This means that sooner of later, you will see zeitgeist integration with these apps. The work is basically divided into two parts – Dataprovider plugins and Dataconsumer plugins.

Dataprovider

The word dataprovider needs no special explanation. It is basically a plugin/addon whose sole work is to “push events” in the daemon. This data is then available for different kinds of work ranging from checking history or any complex data mining activities.

Dataconsumer

It is basically those applications which consume the events pushed in the daemon. You can actually have a list of all the tracks you listened till now in decreasing order of count and also the exact time when you hit Play for that specific track. You can check when you added a track, when you deleted and a gazillion of other activities. Let your imaginations fly.

Packaging

I would like to thank Mirco Bauer(meebey) and Jo Shields(directhex). meebey was the one who did created the autoconf based build. Jo Shields is the brave man who is handling the packaging. Thanks a lot – Debian CLI Team rocks. Thanks for your efforts in packaging.

Future

My hopes from zeitgeist is really high such that you can call it unrealistic. We really need people who can write dataproviders. Here I am not calling people who any language and not just mono. If you can think/want to write a dataprovider for any app, contact us. We will guide you. It will be fun. We have bindings for mono, C/Vala and Python as of now. The java bindings is in development. If you want to contact us, please come on IRC channel #zeitgeist on Freenode and catch any of us.

  • Seif Lotfy: seif or seif_ or seiflotfy
  • Manish Sinha: m4n1sh or manish
  • Mikkel Kamstrup Erlandsen: kamstrup
  • Siegfried-Angel Gevatter Pujals: RainCT
  • Markus Korn: thekorn
  • Micheal Hruby: mhr3

UPDATE: There can be a bit of confusion in the versioning. The ABI version is at 0.1.0.1 but the release version is at 0.1.02 . As long as ABI isn’t broken release versions are fine. 0.1.0.2 has some patches against 0.1.0.1