Running around London

During the past few months, for very boring reasons, I’ve been spending a bit of time running relatively long distances around London.

It’s been quite a revelation. London is, as every Saint Etienne fan knows, a very big place1.

It’s hard to find the time to walk around very much of it, just because the distances are so big. I used to cycle occasionally, but everything goes by a bit quick that way and you’re liable to crash if you spend too much time looking around you. Running turns out to be a pretty good compromise. I’ve been through more interesting bits of London—many of them via canal or river routes—during these few months than I have in years.

From Chelsea Bridge

I’ve lived in London for nearly 18 years and in west London for ten, yet I’d never before seen the splendid Peace Pagoda in Battersea Park up close, been along the leafy Thames path on the south bank west from Putney, realised that Trellick Tower (below right) sits on the canal side, or had any sense of the relationship between the westbound road, rail, and canal routes from the Paddington basin. (Did you know the canal goes over the North Circular?)

I’d worked near Mile End for six years without ever having been along the Lee Navigation or seen the dramatic Bow locks—never mind peering at the near-future Olympic site.

Trellick Tower

And I’ve worked in Docklands, yet the only time I’d ever been through Wapping and joined the dots between docks and City was on 7/7, walking home across London when all the transport was out.

I even realised recently that, no matter how many times I may have passed the front of the Palace of Westminster, I’d never been along the length of the building to get any real impression of the scale of it.

I’m not completely ignorant of this place: I know my way around the City and West End well enough, I’ve spent a lot of time walking in various areas further out, I’ve lingered in the Barbican and on the South Bank and so on—which is partly why it’s been so much fun to be reminded how much even of central London I’ve still never properly seen.


1 Apparently the quote about London at the start of You’re In A Bad Way is from the film Billy Liar. I’ve seen the film since becoming familiar with the song, and I didn’t even notice the line. Hopeless, I am.

New 1.2 release of EasyMercurial

At the SoundSoftware project we’ve just released a new version of EasyMercurial, our user interface for the Mercurial version control system.

This release has one change that I think is quite interesting—we’ve removed one of the five big buttons that used to occupy the main toolbar. It turned out that the meaning of the “Refresh” button wasn’t anywhere near as clear to users as we had foolishly imagined it to be during development. Our article about the release explains in a bit more detail what the problem is and what we did about it.

This is one advantage of having a constantly-shifting population of fairly autonomous users around you, as you do in a university research lab. Although our users don’t always have the time to test much, we do get a fairly broad range of use cases being tested. Working out how to act on the feedback we get is the hard bit!

(One exception, though, is that we have relatively few in-house users running EasyMercurial on Windows. So if you’re a Windows user, your feedback is especially valuable.)

At last, a PC with a decent screen resolution

Pity it’s so small!

I’ve been longing to see a resolution as good as 2048×1536 on a flat-panel 14″ laptop or 19″ desktop screen. Clearly it was possible, but it never arrived.

Perhaps that’s because the one company that seems to take an interest in these things nowadays (Apple) has a desktop operating system that won’t let you change the font size.

So there’s never been much point in their making a screen with such a high resolution, because you wouldn’t be able to read the text—unless the screen was huge… and a huge screen needs to be widescreen or else it’ll be too tall to scan by eye easily… and so here we are, surrounded by rubbish monitors. Hey ho.

No, the last time anything at all comparable was available, it looked like this:This 19″ screen, from about a decade ago, managed 1920×1440. That’s looking pretty good by now.

Spare us humans from XML

XML appeared in 1996, was refined during 1997, and was standardised in 1998.

I remember a lot of excitement about it at the time, from managers who imagined it would solve all their data portability problems. I was conscious of some of this enthusiasm before I really looked at the format.

When I did, I thought

ugh

But it wasn’t as bad as all that. The idea of XML was to standardise the lexical layer for a file format, helping to cut down on the programmer’s natural tendency to just bodge something up when saving and worry about loading later.

It worked, and it made and still makes a tolerable format for all kinds of things—within limits. Of course it’s horribly verbose and slow to parse, but hey, it compresses well. And you still don’t get reliable interchange unless you have a known storage structure on both sides, something a series of increasingly complex helper standards evolved atop XML to help you with.

One thing XML never was, though, was nice for humans to read.

At the time this seemed OK, because it obviously wasn’t intended to be for humans. We humans would never be editing it. We’d only ever be looking at it through filters and lenses and programs that knew what it really meant. We’d never actually have to see the format.

Fifteen years later, here I am sitting looking at

<if>
    <condition>
        <isset property="tested.project.dir" />
    </condition>
    <then>
        <property name="tested.project.absolute.dir" location="${tested.project.dir}" />
        <xpath input="${tested.project.absolute.dir}/AndroidManifest.xml"
            expression="/manifest/@package" output="tested.manifest.package" />
        <if>
            <condition>
                <isset property="tested.manifest.package" />

Oof! Enough of that.  But that’s Android development: of course that’s for robots.

Windows Phone is for people, though. How about:

<phone:PhoneApplicationPage.ApplicationBar>
    <shell:ApplicationBar Opacity="0">
        <shell:ApplicationBarIconButton Text="previous" IsEnabled="False"
            IconUri="/Shared/Images/appbar.left.png" Click="PreviousButton_Click"/>
        <shell:ApplicationBarIconButton Text="page jump"
            IconUri="/Images/appbar.book.png" Click="JumpButton_Click"/>
        <shell:ApplicationBarIconButton Text="settings" IconUri=

Aargh.

These aren’t pathological examples that I’m having to grub around the internals of some graphical environment in order to find. That last one, for example, is copied verbatim from a beginners’ programming book for Windows Phone 7.

I’d far, far rather see developers use XML than go back to rolling completely unique file formats for every application. But surely by now there are enough widely-supported data representation languages—formats that simplify the presentation by standardising object relationships as well as lexical details, such as JSON or RDF/Turtle—to cover the majority of situations in which humans may end up having to edit something?

Wrapping a C++ library with JNI, part 4

In this series…

  • Introduction, outlining the general steps from starting with a C++ library to being able to build and run simple tests on some JNI wrappers;
  • Part 1, in which I design some simple Java classes and generate the stub wrapper code;
  • Part 2, in which I add just enough of the implementation to be able to do a test build;
  • Part 3, discussing object lifecycles in C++ and Java;
  • Part 4 (this post), the final episode covering a few remaining points of interest.

A More Complex Method

The most complex function in our API is the process method in the Plugin class. In C++, this is

typedef std::vector<Feature> FeatureList;
typedef std::map<int, FeatureList> FeatureSet;

virtual FeatureSet process(const float *const *inputBuffers,
                           RealTime timestamp) = 0;

That is, the process method of a Plugin object takes a two-dimensional array of floats and a RealTime object as arguments, and returns a map from an integer to a FeatureList, which is a sequence of Feature objects stored in a vector. That’s fairly complicated.

Java lacks typedef or any very satisfactory alternative. So, we render this as

public native Map<Integer, ArrayList<Feature>>
    process(float[][] inputBuffers, RealTime timestamp);

where Feature and RealTime are classes we must provide separately. (It’s good form to declare the return type using an interface such as Map in cases where the caller doesn’t need to care which specific container implementation is used. In this case our concrete return type should probably be a TreeMap.)

I’m not going to explain every detail of the implementation of this in the JNI wrapper, but I want to illustrate a couple of aspects:

Constructing Java objects from JNI code

In this particular case, the objects we want to return take rather a lot of work to construct. However, we can illustrate a simple example. At the innermost level, all of our returned objects have type Feature, a Java class we define which has a constructor that needs no arguments.

To construct one of these, first we need to look up its class:

jclass featClass = env->FindClass("org/vamp_plugins/Feature");

Then we find the method in the class that corresponds to the constructor. The GetMethodID function is used for looking up all non-static methods, with the method name supplied as the first string argument. For a constructor, the method name is <init>. The final argument gives the signature of the method to look up; in this case ()V means a method taking no arguments with a void return type.

jmethodID ctor = env->GetMethodID(featClass, "<init>", "()V");

Then, to construct the object we simply call the constructor:

jobject feature = env->NewObject(featClass, ctor);

Handling Generics through JNI

This is dead easy. Generic types are there only for the purpose of compiler type-checking: they don’t exist in the virtual machine or in the .class file.

In terms of Java objects, including from the perspective of any JNI code, our ArrayList<Feature> is simply ArrayList.

So, to construct the TreeMap<Integer, ArrayList<Feature>> we intend to return from our function, we only need to do this:

jclass treeMapClass = env->FindClass("java/util/TreeMap");
jmethodID treeMapCtor = env->GetMethodID(treeMapClass, "<init>", "()V");
jobject map = env->NewObject(treeMapClass, treeMapCtor);

Of course we then need to make sure the objects we put in it are of the right type, as the Java compiler is no longer there to help us with type-checking.

Similarly, if a generic container gets passed to a native function, we can (and must) just ignore the type specialisation when unpacking it.

Getting Data from Multi-Dimensional Arrays

My process function takes a two-dimensional array of floats as one of its arguments. (This actually represents multi-channel audio sample data.)

Multi-dimensional arrays in Java are easier to deal with reliably than in C++. An array is an object, so a two-dimensional array is an array of array objects. It appears in the JNI implementation as a jobjectArray:

jobject
Java_org_vamp_1plugins_Plugin_process
    (JNIEnv *env, jobject obj,
     jobjectArray inputBuffers, jobject timestamp);

JNI provides simple accessor methods for getting at array elements: to pull out an object from an array of objects, we use GetObjectArrayElement. That will enable us to get hold of the second dimension of our arrays.

Then, to access a sequence of non-object type elements such as our float values all at once, we need to use a symmetrical pair of calls: one to lock the elements in place so the garbage collector can’t get at them, and the other to release them again. To wit, GetFloatArrayElements and ReleaseFloatArrayElements.

Putting these together to retrieve our two-dimensional float array in C++, we have as the body of our process function:

int channels = env->GetArrayLength(data);
float **input = new float *[channels];

for (int c = 0; c < channels; ++c) {
    jfloatArray cdata =
        (jfloatArray)env->GetObjectArrayElement(data, c);
    input[c] = env->GetFloatArrayElements(cdata, 0);
}

Then we do something with the C array that is now stored in input, hang on to the output for a moment, and tidy up:

for (int c = 0; c < channels; ++c) {
    jfloatArray cdata =
        (jfloatArray)env->GetObjectArrayElement(data, c);
    env->ReleaseFloatArrayElements(cdata, input[c], 0);
}

delete[] input;

and we’re done. It remains only to construct and return our rather complex return value based on whatever we calculated with the input array earlier.

That’s all

That’s all for this series—thanks for reading, and I hope it’s been useful.