Ekam Continuously Builds

I just pushed some changes to Ekam adding a mode in which it monitors your source code and immediately rebuilds when a file changes. As usual, code is at:

http://code.google.com/p/ekam/

Just run Ekam with -c to put it in continuous mode. When it finishes building, it will wait for changes and then rebuild. It also notices changes which happen mid-build and accounts for them, so you don't need to worry about confusing it as you edit.

I've only implemented this for FreeBSD and OSX so far, but things are still broken on Linux anyway (see previous post).

kqueue/EVFILT_VNODE vs. inotify

Speaking of Linux and monitoring directory trees, it turns out that Linux has a completely different interface for doing this vs. FreeBSD/OSX. The latter systems implement kqueue, which, via the EVFILT_VNODE filter, can monitor changes to a file pointed at by an open file descriptor. This includes directories -- adding or removing a file from a directory counts as modifying the directory.

Linux, on the other hand, has inotify, a completely different interface that, in the end, does the same thing.

There is a key difference, though: kqueue requires you to have an open file descriptor for every file and directory you wish to monitor. inotify takes a path instead. Normally I prefer interfaces based on file descriptors because they are more object-oriented. However, the number of file descriptors that may be opened at one time is typically limited to a few thousand -- easily less than the number of files in a large source tree. On OSX (which seems to have lower limits than FreeBSD) I was not even able to monitor the protobuf source tree without hitting the limit.

Furthermore, watching a directory using inotify also means watching all files in the directory. While I have to assume that this is NOT transitive (so you can't watch an entire tree with one call), it still (presumably) greatly reduces the overhead of watching an entire tree, since the OS doesn't have to flag every file individually.

I'm still waiting for confirmation from the freebsd-questions mailing list on whether EVFILT_VNODE is really so much less scalable, but lots of Googling didn't produce any answers. I just see lots of people asking for inotify on FreeBSD and being told to use EVFILT_VNODE instead, as if it were an equal replacement.

This may mean that FreeBSD and OSX will have to use some sort of gimpy polling strategy instead, but that will have its own scalability issues as the directory tree gets large. Sigh.

I may give up and switch my desktop to Linux. Fully-supported Chrome and Eclipse would be nice, as well as a more robust package update mechanism (I'm getting sick of compiling things). Figuring out how to do everything on FreeBSD was fun, but getting a bit old now.