Note on “Explorations in Time-Frequency Analysis” by Patrick Flandrin

Patrick Flandrin is a physicist and signal-processing researcher whose name I first encountered as co-author (with François Auger) of a 1995 IEEE Transactions on Signal Processing paper called “Improving the Readability of Time-Frequency and Time-Scale Representations by the Reassignment Method”.

This crunchy publication (21 pages, dozens of equations and figures) took a pleasing idea — replacing the familiar grid-format time-frequency spectrogram with a field of precisely localised points calculated using both magnitude and phase of the frequency bins, rather than only magnitude as a traditional spectrogram does — and set out the mathematics of applying it to a number of different time-frequency and time-scale representations.

Illustration from Auger & Flandrin (1995)

I read this paper about 15 years ago and didn’t understand it. I have since realised this is partly because it isn’t all that clear with its notation, but there is also a big gap between the naive programmer’s view (that’s mine) of a spectrogram and the mathematical analysis used in the paper.

To explain. For a programmer, a spectrogram comes from taking short overlapping slices of a sampled signal, multiplying each by a smoothing window shape, applying a short-time Fourier transform, and taking the magnitudes of the complex output bins to get one column of the spectrogram per slice of input. The short slices are because you want a fixed, smallish number of output bins, and you have various tradeoffs — time and frequency resolution and computational efficiency — to consider in that. The smoothing window is because your Fourier transform — a thing which matches up sinusoids of different frequencies against a signal to identify which ones would add up to it — operates on an infinite signal, consisting of the input you give it repeated forever in both directions: this will have a discontinuity each time it wraps around, and the smoothing window removes some of the frequency artifacts from these discontinuities. There is nothing particularly mathematical about the implementation of this, and any intuition used by the programmer is a mixture of the visual and techniques from the world of engineering. The language used in a publication like the DAFx book is typical in this world.

The Auger & Flandrin paper instead comes from a world that summarises a spectrogram as a two-dimensional Wigner-Ville distribution filtered with a smoothing window leading to a time-frequency representation of the Cohen’s class. Signals are finite-energy functions over infinite domains, and a spectrogram is a double integral over time and angular frequency. Both time-domain functions and time-frequency representations are continuous, and practical questions about overlap and window length don’t arise. I can dimly remember this world, because my undergraduate degree — who am I kidding, my only degree — started out as pure maths, but I haven’t inhabited it for any of my working life.

So I didn’t really understand the paper, and a programmer has plenty to do, and that is one reason why Sonic Visualiser’s “Peak-Frequency Spectrogram” layer calculates instantaneous frequencies from the phase difference between successive columns, something which I found much easier to understand. (It turns out there are other good reasons one could make this choice, but I didn’t know that.1)

Returning to the paper recently, I learned that Flandrin had written a book on the subject, and I bought a copy hoping it might bridge the conceptual gap. It turned out to be a good experience.

* * *

“Explorations in Time-Frequency Analysis” is a monograph digressing on things the author has found interesting in the past 30 years, which — what luck! — happen to be about time-frequency analysis. It’s short, about 200 pages, and nicely printed. There are lots of diagrams, and although equation-heavy it doesn’t hang about proving things, sending you to the references instead. It begins with a glossary of notation (I like it when books do this) and ends with a 9-page bibliography. The writing is crisp and friendly and the scene is set by the first two chapters, a philosophical outline and a chapter of examples with the lovely title “Small Data Are Beautiful”.

Although the book provides a lot of the background to the paper that defeated me, I still spent a potentially embarrassing amount of thought on things I imagine that anyone properly within the target market finds obvious. An example is what it means for a Gaussian function to be “circular” in time and frequency. The book goes over this in far more detail, but briefly a Gaussian — the bell-shaped normal distribution curve found in probability — has the property that its Fourier transform is also a Gaussian. The “wider” the bell shape in the time domain, the “narrower” in the frequency domain: at some point it must be equal in both, and then if you plot it in a spectrogram-like heat map you will see a circle. When does this happen? It’s shown that it happens for the Gaussian corresponding to a normal distribution of variance 1. But at this point I am worrying about units. What does it mean to be circular? The figures illustrating this lack units in either axis — in fact detail-wise many of the figures are more like sketches — and the little bit of engineer in me is wondering: how can you possibly have a circle if you lack units?

The answer I eventually recalled is that the units in one domain define those in the other. In this case, if the time axis is in seconds then angular frequency is radians per second, and a circle is a distribution whose extent in seconds is the same as that in radians per second. Other units such as samples (in time) or STFT bins (in frequency) have similar correspondences in the other domain. This is a place where going back to basics took significant thought, but I did actually appreciate being expected to think about it.

So a nice rehearsal with some interesting bumps, but for me the thrilling twist arrives in chapter 12, “Spectrogram Geometry 2”. This reframes the spectrogram as a complex plane and the reassignment operator in terms of motion in a potential field proportional to the log-spectrogram. This mathematical leap is also an intuitively visual one, and it’s exciting for me because it is a little like how I pictured the spectrogram, with no meaningful mathematical analysis, when developing a certain feature of the Rubber Band timestretcher.2 This chapter is like seeing the vaguely-realised ground beneath your feet resolve into a larger, recognisable object — the moment when you realise you are standing on the back of a giant Pokémon, if you will.

There is a lot more in this book, and I think it will repay repeated visits. I’m not sure whether you could implement anything directly from it, but you could, say, pick a random page and follow up all the references until you really feel you understand it. I think this would be a rewarding exercise that, for someone like me, would probably take around a month per page.

* * *

On that note, one of the first references given is to a book called “Visible Speech” by Potter, Kopp, and Green, 1947. I looked this up and was so intrigued that I tracked down an ex-library copy. It is a lavish presentation, perhaps with both training and PR elements, of a then-new idea called the “sound spectrograph”, i.e. a spectrogram. The title “Visible Speech”, incidentally, is borrowed with attribution from an earlier (1867) work about phonetic alphabets.

The authors of the 1947 book were writing about work done at Bell Labs to try to make the telephone accessible to the deaf. Their experimental devices used paper tape or phosphor display to show spectrographs of the speech sounds, and users were specially trained to interpret speech from them. Here’s a picture from the book of someone using one.

Operator sitting at a table in front of a large box with a tiny screen on itThe spectrographs were produced by automatically recording the speech to tape and playing the tape repeatedly through a filter of 300Hz bandwidth, whose centre frequency was incremented linearly between passes in 15Hz steps from 0-3500Hz. (They also had a version using 45Hz bandwidth filters, but it was found to be less legible.) The system was of course analogue.

In this image the top spectrograph is the one with 45Hz bandwidth, which is used to point out some interesting features, but the 300Hz bandwidth spectrograph below it is the form used throughout the rest of the book:

It’s striking how clear these spectrographs are, and it makes a useful reminder that we really aren’t always looking for the most precise representation of something — 300Hz bandwidth at speech frequencies is pretty wide! — but instead the most appropriate in some human dimension.

 


1 The Sonic Visualiser peak-frequency spectrogram precisely localises stable frequencies, but for each frequency bin it draws a short horizontal line across the whole duration of the bin at the proper frequency rather than localise the bin to a point in time. A very similar output could have been produced using reassignment, because the frequency calculated from phase difference should be very close to that calculated with reassignment. But a decision to do that would have meant ignoring the other reassignment operator, localisation in time, which gives a single point rather than a horizontal line for each bin. Had I understood the reassignment paper, I would probably have felt compelled to do that part properly. For it to work well, a greater bin overlap and much more sophisticated rendering would have been needed, and the result would have been much slower and possibly less clear for real music. I think.

2 This feature, which I gave the vague name “phase lamination”, was worked out in a hurry after discovering that the “phase locking” technique of Jean Laroche and Mark Dolson which I had used in the very first release of Rubber Band was patented. Phase locking reduced audible phasiness with the nice side-effect of making the phase vocoder faster to compute, but it also lent a robotic tang to the sound which certain listeners found even more unpleasant than the phasiness. The scheme I came up with to replace it was based on picturing a gradient field and making adjustments to bins near a peak or trough in proportion to the distance from it — tuned by ear rather than worked out mathematically. Although it lost the improved speed of phase locking, it usually sounds better. The idea seems reasonably obvious, but I hadn’t seen it described anywhere else and I was delighted to find it.

Can you swap the keyboards between a Thinkpad T40 and an IBM SK-8845?

No.

(I’m posting this because it’s something I searched for and didn’t find an answer to.)

The SK-8845 and its sibling the SK-8840 are IBM-branded keyboards with built-in trackpoint and trackpad.

They appear to be essentially the keyboards from Thinkpad T40 (or T41, T42, T43) laptops, pulled out and put in a separate case for use as a compact keyboard/mouse controller in a server room. They’re pretty good keyboards in a classic Thinkpad laptop sort of way.

I have an SK-8840, which is the (less common?) PS/2 variant. As far as I can tell from photos, without having seen one in person, the SK-8845 is the same keyboard with a USB plug on it.

Here’s the SK-8840:

IBM SK-8840 keyboard. I’m sorry, I removed the trackpoint. I always do – I never use it and it gets in the way a little. I do keep the trackpoints in case I want to sell up though. Have you noticed the trackpoint pin is rotated 45° from the normal Thinkpad keyboards? I wonder why.

Visually this is much the same as the keyboard on the T4x series of Thinkpad laptops, which are truly excellent. It doesn’t feel like a T40 keyboard though. It’s more like the later, still good but not quite so good T60 series. So I wondered whether it was possible to swap the keyboard part with that from (ahem) one of my T4x laptops.

The answer is no: it’s physically incompatible. The plate at the back is different, the connections are different, the stand-offs are different. It may be possible to adapt one into the other and that could be an interesting project for someone more ambitious than me, but it definitely isn’t a case of just pulling out the keyboard part and plugging it in. The same goes for the T60.

Here’s the innards, and the back of the keyboard plate, of the SK-8840:

Here are the backs of the keyboards from a T40 (above) and T60 (below):

That’s all.

Films by Aki Kaurismäki

We sat down, my wife and I, during a quiet period that could otherwise have been a little gloomy, to watch as many feature films directed by Aki Kaurismäki as we could lay hands on.

We watched them in chronological order, so this could have been an interesting article about the director’s development from his earliest days. But we were geeky enough to mark them out of ten, so what we’ve actually made is a list ranked from our least to most favourite.

These aren’t our first impressions, in most cases – we’ve seen many of these before, some three or four times, but watching them in order with a vaguely critical eye was new and pretty good fun. I should say that we don’t know any Finnish (other than words like raha and miksi? which come up all the time in these films) and watched with English subtitles. We got all of these from the Curzon DVD box set, except for I Hired A Contract Killer which isn’t in it for some reason.

A constant in these films is the cinematographer Timo Salminen who seems to have worked on every one of them. A few more than half are in colour, the rest black-and-white, all analogue film. They’re all beautiful to look at, and since Kaurismäki spends a lot of time just looking at things, that’s certainly fortunate.

I have placed a useful star * next to the names of the films in which the male hero is violently attacked for no reason other than to set up the rest of the film. This Kaurismäki trope is so common that one might as well mark it. I have not specially marked the films in which a cute dog plays a sympathetic role, the films in which the heroes drive around in a car much older than any of the others on the streets, the films that cut away to a complete live performance of a song by a folk or rock ‘n roll band, or the films in which someone impulsively performs an act of great kindness without expecting or receiving any acknowledgement for it.

17. Leningrad Cowboys Meet Moses (1994)

Leningrad Cowboys, the worst band in the world, return from the Americas to Europe on the instigation of a man who strangely resembles both their manager and the Biblical Moses.

16. Juha (1999)

An adaptation of an apparently noted Finnish novel from 1911 in the form of a black-and-white silent movie (with music soundtrack) with film noir stylings and references.

I like its confusion about its era, as when the silly, soon-to-be-tragic couple (“they were as happy as children”) suddenly switch from serving and eating porridge out of a big pot to heating up a microwave meal, as a symbol of their fall.

I very much like that Marja packs a rubber duck when she moves out.

But the story, setting, and ending felt a bit too miserable for us. Its silence felt as if we’d been left on our own to digest it, which made it feel gloomier.

15. Hamlet Goes Business (1987)

A straight-ish Hamlet, considering it’s a contemporary film about the rubber-duck industry, filmed in super-crisp black-and-white with a terrific ending. Not bad at all.

14. Calamari Union (1985)

This film can’t possibly live up to the summary of its plot: Fifteen desperate men, all called Frank, try to cross Helsinki to reach the fabled Eira district. Most don’t make it.

Their journey is a terrible struggle that takes… months? years? – whereas maps of the real world suggest that Eira is maybe an hour’s walk from where they began.

This was the second Kaurismäki film I ever saw, after Take Care Of Your Scarf, Tatjana!. It’s messy and puzzling in comparison and some of the jokes are a bit too in-joke for my head, but it has a similarly glorious commitment to its idea. We’ve made a note to watch it again soon.

13. Crime and Punishment (1983)

Much more fun than might be expected from an adaptation of a grim Russian novel as the début work by a famously downbeat Finnish director.

12. La Vie de Bohème (1992)

Sparkling, then it hits the wall. This adaptation of the book that also spawned the opera La Bohème is set in Paris and acted in French, even though some of the main actors apparently didn’t speak French. Since those people are portrayed as both foreign and, more importantly, bonkers fabulists, this actually works ok for me, though I might not think so if French was my first language. This was Kaurismäki’s first film with delightful antihero André Wilms, and its first half-hour is as overtly comical as anything here.

When things start to get real, it drags a little, perhaps because by that point we are not really in the mood for realism. So although the ending should be moving, and for better people than us it probably is, we found ourselves thinking this was the first time one of his films had run longer than it should have.

11. The Match Factory Girl (1990)

Sent to divide us.

This is the only film by Kaurismäki to appear in the TSPDT Top 1000, which I understand to be a crowd-sourced list of the 1000 best films in the world. The director has apparently said this was the first film of his that he found good enough to watch. Reviews call it “devastating” and “a sobering parable”. Kati Outinen says she didn’t know she was expected to play the lead until she turned up for filming. It’s about a woman who is misused by her family, abused by a lover, and seeks revenge.

It’s a beautifully posed and filmed study of people, relationships, and work, a grotesque story about sexism and class, and also a potentially funny comedy, if you are not a child. It is very good, but it’s bleak, rather linear, a bit slight, and I don’t think I could ever be persuaded that it’s the best thing Kaurismäki has made. I wonder whether the high rankings come from people thinking it must be important because it’s grim, or that they ought to include Kaurismäki on their list and this looks like the least foolish option. I’d love to hear opposing views.

10. * Lights in the Dusk (2006)

This is an odd one. It’s in a more realistic style than the films that came before and after it (The Man Without A Past and Le Havre). Helsinki is modern, shiny, and unfriendly, and there is, unusually, no ambiguity about when the film is set. It’s a film about a man who ignores the world: he is picked up and used in a heist, and takes the fall for it with no indication that he recognises what is happening. The idea is not unfamiliar, but it’s unusually concrete and blunt. Perhaps the world is not as good as some of these other films suggest it can be. Kaurismäki’s previous film had been almost a fairytale. Was he sickened by it?

This is surely the most macho of Kaurismäki’s films. It has taciturn mob bosses and bullet-headed thugs, the female lead is a classic femme fatale, and other women get merely a line or two. It has what appears, to ignorant outsider me, to be a slightly laboured red vs white class divide (poor honest folk with Finnish names, rich gangsters with Swedish ones). There’s little warmth to the film, though there are certainly mysteries, of which the prime one is why the hero refuses to acknowledge Aila the hot-dog seller.

It is funny, sometimes, and definitely compelling to look at, and has a fine soundtrack, but it left me feeling uneasy. Is it me? Am I misunderstanding? I fear that my response to this film might be something like the response an English-speaking viewer ought to have to Kaurismäki in general.

9. I Hired A Contract Killer (1990)

This film is in English, set in London. If you happen to know London, it’s well worth watching just for the amazing adaptations of London scenery.

For the English-speaking viewer, it also offers an opportunity to test the theory that we only like these films because they’re in a foreign language, set in foreign places, and subtitled.

We just about passed that test. London looks splendidly grubby, there are some amazing tableaux and great dialogue, and I enjoyed the either dispassionate or dismissive delivery, and it’s charming in a satisfyingly bleak way. It does feel a little bit arbitrary and discontinuous in comparison with the best here, but (unlike most of these) we’ve only seen this film once, and would happily see it again.

8. Leningrad Cowboys Go America (1989)

Definitely our kids’ favourite Kaurismäki film: the only one they find truly acceptable. This is the tale of the worst band in the world, who are too awful for their home country and must go to America to be tolerated.

Part of the fun, besides plenty of low comedy, is that the Cowboys are portrayed as dreadful while being in fact pretty good. They may have to subsist on onions but they can definitely play, and the audiences, which are apparently real, generally seem to think so too. (The band now has an extensive performing history outside of the movies.)

A brilliant film in its stupid way.

7. * The Other Side of Hope (2017)

The story of an emigrant from wartime Syria arriving in Helsinki and then trying to find his sister. More realistic than Le Havre, the film that it is temporally and thematically following, and less hopeful. Its bleaker outlook is curiously reflected in the soundtrack, which is much sparser than usual. Although the film does have comic moments, they don’t always work for me. The Japanese restaurant sequence here is the only scene in any of Kaurismäki’s films that feels as if it’s making fun of the protagonists, although it’s redeemed a little by the gravity with which the restaurant owner responds to the humiliation.

That aside, this film is full of tender and generous acts. It feels like a handbook for behaviour in difficult times. I’m oddly reminded of Robin Sloan’s wordy but conceptually neat Proposal for a book to be adapted into a movie starring Dwayne The Rock Johnson, the thrust of which is that The Rock will inevitably be US President at some point, so we should prepare by placing him into a dramatic situation which enacts the compassion and empathy that we expect a US President to have. This film immerses us in a specific situation, like an aeroplane emergency instruction sequence, to give us an overdue preparation for a crisis we are already in.

6. * Ariel (1989)

Everything in this film is terrific. The opening, the drive across Finland, the scene where the couple meet, the breakfast with the kid and the gun, the scene with the cake, that scene at the end whose punchline has been set up from the start. It’s glorious, and by some measure it’s as brilliant as any film ever was, but by our measure it’s a little too much a series of great set-pieces, so it’s not allowed to be the best here.

5. * The Man Without A Past (2002)

I wasn’t keen on this the first time I saw it. I think that was because: it has a Hollywood plot (man wakes up with no idea who he is, learns to live among people he would have previously overlooked); it starts with an unpleasant act of violence that is hard to forget; and the lead actor is fairly hard-looking and doesn’t give us a lot to go on.

I changed my mind completely the second time around. That may be because in the meantime I’d seen its hero Markku Peltola in Drifting Clouds, and I felt warmer toward the man whose hands could no longer whip up a porridge. Or I had become a more sympathetic person myself, perhaps because of the times, and I realised how beatific The Man Without A Past is. It’s the first of a series of recent Kaurismäki films that are essentially richly coloured fairytales, illustrations of how we all could be, if we admitted our better selves. Despite the brutal act that sets up the plot, this is a peaceful film.

It also now seems to me like a wonderful showcase for Markku Peltola, the men who embraces the difficult life and looks just gently humorous at just the right moments, and for Kati Outinen in the most uncompromising and uncommunicative of her leading roles.

There are problems with it I think. Life in difficult places is made to seem rather easy, and we have a not entirely workable dichotomy between the happy poor and everyone else. But having seen this film twice, I look forward to watching it again.

4. Take Care of your Scarf, Tatjana! (1994)

This ludicrously-titled black-and-white road movie is for me the ur-Kaurismäki, in that it was the first of his films I ever saw. I taped it from a late-night TV broadcast in the 90s (no way I was staying up for something so random) and still have the VHS tape somewhere. It looks much better on DVD though, because the photography throughout is just gorgeous. I mean really gorgeous.

It’s sort of a loser-buddy comedy, filmed in stark monochrome as if it’s a gritty exposé. The (male) heroes are a silent idiot and a violent alcoholic braggart who have no depth and, in principle, nothing much to like about them. That makes them sound like action figures, but they have no action going on either. They are totally outshone by Tatjana and Klavdia, the friendly but unsentimental women they meet.

The film is full of lugubrious gags and comic ideas, and those are what I generally remember about it. A celebratory quarter-of-a-sandwich with tea to cement the friendship between nations, repairing a car by pulling bits out of the engine and throwing them away, Valto’s in-car coffee maker, Reino’s worryingly excitable monologue about punching someone.

But when watching it, it’s the spaces between those moments that make the film what it is. I found that I love this film much more when I’m actually watching it than I do in my memory.

And it’s only an hour long.

3. * Shadows in Paradise (1986)

Bone-dry and beautiful. This provokingly slow film spends a lot of rich colour film looking at its lead actors, Matti Pellonpää and Kati Outinen, who are everything that matters in it. It’s only 80 minutes long, but feels a little longer even if you’re enjoying it. I like that. It’s funny and just sufficiently kind.

It’s no accident that Kati Outinen has a leading role in all the films at the top of this list. I love her later, more reserved middle-aged figures, but in this earlier one her slightly shifty character, on the edge, suspicious, always ready to abscond is a delight.

2. Le Havre (2011)

A boy escapes when a migrant family is stopped by the police in France, and the people around try to help him. A fairytale, a romance about human beings that pretends to be a police story or thriller. Such a clean and beautiful film, and I think better when you’ve seen it once already, know what happens, and can stop worrying about the plot.

This film is supposedly a sequel to La Vie de Bohème and has a lot in common with it, including being in French, but it is built the other way up. La Vie de Bohème spends a lot of time on everyday transactions, which are satisfying and entertaining, but becomes harder work when it goes ethereal later on. In Le Havre it’s the other way around: the transcendent is normal, and the rest of life is just telegraphed there to be the context for it. And this one has a slightly higher proportion of actors who actually speak French.

Kaurismäki goes to exceptional lengths to blur when this film is set. Taxis and police cars are from the 80s, some of the locations are set up to look much older, photographers use manual-winding film cameras, phones have rotary dials, nobody has a mobile phone, but the plot is precisely contemporary for the date the film was made. This film was comparatively well-funded I think, which made me wonder whether he would have gone this far every time if he could have afforded it. The result is a highly personal feeling in which the world around us is subjective and dream-like, and only the people in it are real.

1. * Drifting Clouds (1996)

A glorious love story that, in my alternate world, is a Christmas film that the family gathers to watch, a bit like It’s A Wonderful Life is supposed to be.

Drifting Clouds is a film about a middle-aged couple facing financial disaster, deep in debt, having lost their jobs, with a tragedy in their past, in the bleak economic climate of mid-90s Finland. It’s also a comedy, one that doesn’t demean its characters, who are proud, admirable, and committed to one another. Kati Outinen is excellent again but the solidity and good humour of Kari Väänänen is essential to prop things up and there’s a compelling supporting cast. The staging is beautiful, the photography cautious and sympathetic, and there’s a po-faced joke in every other line. Although slow-moving, it’s never slow.

“I Am Not As Well-Suited To This As I Had Hoped”

I wrote this in May 2020, and am publishing it in March 2021. I didn’t publish it back then because it felt entirely superfluous, but I was quite interested to find it just now.

Well, this Covid lockdown business is tricky. I’m in a super-privileged position: at home with family I love and who all get along well; job I can do from home; no pressing money worries; not all that sociable; don’t bore easily. But even then I haven’t settled in to it as well as I had expected. Here, partly for the purpose of counting my blessings, I write down some of the good and bad things about the past weeks of lockdown for me. I’ve included as many pointless, trivial things as I can.

I like Not having to think about clothes. I get up and put on jeans and a T-shirt. If I find I’m wrongly dressed for the weather, I can just get changed.

I don’t like Worrying about everyone. The health and sanity of my parents, who would normally live a more social life than I would, but have reasons to be far more fearful. The stressful constraints on my children, particularly the 18-year-old. The low-level unease about everyone I see out and about.

I like Not having to set an alarm clock for 6.30am to get the boys to school.

I don’t like Getting up and going to work… in our bedroom, the only place I could put my desk in, in our two-bedroom flat with two full-time working adults and two school children in it.

I like The sense that the social contract is open to review, even if temporarily. Ideas such as providing free accommodation to the homeless, removing cars from the streets, and turning upside-down the entire hierarchy of perceived merit in the workplace are up for consideration. Maybe the field is open for more radical ideas about energy and work.

I don’t like Remembering why it matters so much that we consider new ideas about energy and work. This thing is a dangerous current that makes it inadvisable to swim from this beach today. Climate change is the coming tsunami.

I like The coffee. Much better coffee here than in the office.

I don’t like Video calling for work. Oh god, the video calling. I’m beginning to feel a little sick whenever I see a video window. I am bad at video. I will sit in a video meeting silently, unable to imagine that anything I have to say is worth unmuting for. I have trouble listening as well. I’m too aware of my own presence, what I look like, whether my microphone is going to make any weird noises if I have to use it, the fact that I’m transmitting from my desk in our bedroom, the hairbrushes visible on the chest of drawers behind me, the sudden unwanted transparency of our formerly private home. It reminds me of being a school child summoned to a teacher’s office for “a chat” about your terrible behaviour, and sitting there thinking about your own awkward presence in the room, nodding, then leaving with no recollection of what anyone said.

I like Knowing that there is always someone at home and never anything significant to go out for. It’s bizarrely liberating to know that I don’t actually need to remember my wallet, phone, or keys when I leave.

I don’t like The landed-gentry relationship I now have to anyone who is actually doing real work. I’m officially working, but I’m just sitting at home typing at a computer. I might as well be padding around in my slippers sipping a brandy. Then if I decide I’m a bit short on something, I put in an order and someone else, someone who is actually working, delivers it to me. I’m in a separate social plane from the real economy, and it’s a little sick. I am essentially useless, but I get to command people as if I mattered. This is not unlike the way it always was, at least since the last time I stopped drawing the dole to start a pointless web-based software job. It’s just made visible.

Hayao Miyazaki and work

I see it’s the 80th birthday of the noted animator Hayao Miyazaki.

I once read his book of essays Starting Point: 1979–1996 and was seized by a desire to do something well, with vigour and clarity, instead of tiredly poking around. At the time this made me a little miserable, and I wondered why.

* * *

The book contains essays, interviews, and documents from Miyazaki during a period in his 30s and 40s in which he directed six highly-regarded anime feature films: The Castle of Cagliostro, Nausicaä of the Valley of the Wind, Laputa: Castle in the Sky, My Neighbour Totoro, Kiki’s Delivery Service, and Porco Rosso. This is an excellent run, and history tells us that the two films he made immediately afterwards would be even more celebrated.

The book contains planning documents, interviews, and some essays written between films. The planning documents describe a film before it has been made, and the interviews talk about it afterwards, when in every case it had been a success.

This structure, no doubt accidentally, gives a sense of a narrative like this: Miyazaki thinks of an idea and writes it up with in clear and forceful terms; he starts drawing without a script or storyboard and develops the plot as he goes; he works solidly for three years; the film is finished; it’s a wonderful piece of work that is just as he had envisioned; the process repeats.

Reinforcing this impression is some of the content of the interviews. For example, when talking about My Neighbour Totoro — a glorious film whose origin seems to speak of as pure a work of art as ever appeared in a cinema — Miyazaki remarks that he felt “tremendous happiness” while working on the film, that he knew throughout how it was going to work, that he could avoid any directorial tricks and keep the plot as simple as possible, that he could almost have made the entire film be just about the excitement and fear of a typhoon passing near a house at night; and that if he were to make the film again, given the same technical constraints, the only thing he would change would be to allow more time with the main characters, two small children, while they live their lives, ignoring the plot.

* * *

This is very lovely to read, but also a little daunting. There is something there, real or imagined, that I found I longed for in my work.

What is that longing? It isn’t about talent. I’m delighted by the talent of Miyazaki, but I don’t yearn for it, probably because I can’t imagine having it. It also isn’t about whether my work is worthwhile — I have doubts, but those are long-standing, unchanging doubts. I think it is about focus and application.

I believe that we all have the ability to create work that satisfies our own critical judgement as a coherent artistic effort, but that we don’t do it, because we can’t find the clarity of mind and the conviction to complete an idea with the quality that we first imagine for it.

As a mere programmer, I’m aware that much of the software I write will never be used, or not in the way I imagined it. I believe this is the unspoken experience of all programmers. Software that is made with care but not used is obviously unsatisfying. But the other side of it is software that is not made well enough to merit users at all. If it hasn’t been made well enough, then the more popular it is, the more people might be damaged by mistakes in it, and so the worse it is. Experience seems to consist partly in learning to suppress the fear of this, and to find sufficient trade-offs to ensure that anything useful ever gets published.

* * *

The perceived narrative I referred to above is not the whole story. There is plenty in the book about team work, and it hints at the vast amount of manual labour going on, with stories about overworked colour artists, slipping schedules, and the continually unmet expectation that everything will be easier next time. And at the end of the book is a retrospective timeline, from which we can see that the narrative doesn’t flow linearly either — the team must often have been working on more than one Miyazaki-directed film at once, and some films were based on ideas that had been sketched decades before.

Focusing on team effort does change the picture. You need a lot of people to make a film, and I suppose before you animate the film, you need to animate the people. If the impulse is enough to carry everyone along, then a team can maintain a direction even if their director isn’t certain where they will end up. Perhaps the sense of purpose that seems so desirable is something that one person can’t readily sustain on their own.

* * *

My reverence for the film artifact might not be shared by its makers either. Throughout this book, as well as in more recent interviews I’ve seen, Miyazaki is actively grumpy about the value of anime: there’s too much animation being made already, this is all a waste of time, we contribute nothing to the world, I only want the industry to continue because I know too many people who are animators and I don’t want them to lose their jobs. It’s only when he is sunk deep into a project that he appears to be happy about it.

On macOS, arm64, and universal binaries

A handful of notes I made while building and packaging the new Intel/ARM universal binary of Rubber Band Audio for Mac. I might add to this if other things come up. See also my earlier notes about notarization.

Context

I’m using an ARM Mac – M1 or Apple Silicon – with macOS 11 “Big Sur”, the application is in C++ using Qt, and everything is kicked off from the command line (I don’t use Xcode).

To refer to machine architectures here I will use “x86_64” for 64-bit Intel and “arm64” for 64-bit ARM, since these are the terms the Apple tools use. Elsewhere they may also be referred to as “amd64” for Intel, or “aarch64” for ARM.

Universal binaries

A universal binary is one that contains builds for more than one processor architecture in separate “slices”. They were used in the earlier architecture transitions as well. Some tools (such as the C compiler) can emit universal binaries directly when more than one architecture is requested, but this often isn’t good enough: perhaps it doesn’t fit in with the build system, or the architectures need different compiler flags or libraries. Then the answer is to run the build twice with separate output files and glue the resulting binaries together using the lipo tool which exists for the purpose.

How does the compiler decide which architecture(s) to emit?

The C compiler is a universal binary containing both arm64 and x86_64 “slices”, and it seems to be capable of emitting either arm64 or x86_64 code regardless of which slice of its own binary you invoke.

Perhaps the clearest way to tell it which architecture to emit is to use the -arch flag. With this, cc -arch x86_64 targets x86_64, cc -arch arm64 targets arm64, and cc -arch x86_64 -arch arm64 creates a fat binary containing both architectures.

If you don’t supply an -arch option, then it targets the same architecture as the process that invoked cc. The architecture of the invoking process is not necessarily the native machine architecture, so you can’t assume that a compiler on an ARM Mac will default to arm64 output.

I imagine the mechanism for this is simply that the x86_64 slice of the compiler emits x86_64 unless told otherwise, the arm64 slice emits arm64 likewise, and when you exec the compiler you get whichever slice matches the architecture of the process you exec it from.

There’s also a command called arch that selects a specific slice from a universal binary. So you can run arch -x86_64 make to run the x86_64 binary of make, so that any compiler it forks will default to x86_64. Or you can do things like arch -arm64 cc -arch x86_64 to run the arm64 binary of the compiler but produce an x86_64-only binary.

If you invoke a compiler directly from the shell without any of the above going on, then you get the machine native architecture. I assume this is just because a login shell is itself native.

For my builds I found it helpful to provide a cross-compile file to tell Meson explicitly which options to use for the architecture I wanted to target. That avoids the defaults being just an accident of whichever architecture Meson (or its Python interpreter, or Ninja) happened to be running in, without having to litter the build file with explicit architecture selections. I then scripted the build twice from a separate deployment script, using a different cross file for each, rather than try to have a single Meson file build both at once.

How do I target a particular version of macOS?

Use a flag like -mmacosx-version-min=10.13 at both compile and link time.

For ARM binaries, the oldest version you can target is 11. But you can still build a universal binary that combines this with an Intel binary built for an older version, and the result should run on those earlier versions of macOS as well.

How does a version of macOS decide whether my binary is compatible with it?

I had this question because I had built a universal binary (as above) in which the Intel slice was, I thought, built for macOS 10.13 or newer, but when I brought it to a machine with macOS 10.15 it showed as incompatible in the Finder and could not be opened there.

The answer is that it looks at the relevant architecture slice of the universal binary, and inspects it to find a Mach-O version number. In “older” versions of the macOS SDK this version is written using the LC_VERSION_MIN_MACOSX load command; in “newer” versions (I’m not quite sure when the cutoff is) it is tagged as the minos value of the LC_BUILD_VERSION load command instead. The linker quite logically decides which load command to write based on the value of the version number itself, so if you build -mmacosx-version-min=10.13 you get a binary with LC_VERSION_MIN_MACOSX specified.

You can display a binary’s version information with the vtool tool, and it also appears in the list of information printed by otool -l. In theory you can also change this tag using vtool, but (a) that’s a bad idea, fix it in the build instead and (b) vtool segfaulted when I tried it anyway.

And after all that, in my case the cause turned out to be that I’d failed to supply the -mmacosx-version-min flag at link time.

Why is my program being killed on startup?

It appears that if you build a program for one architecture and then rebuild it for the other arch to the same executable file without deleting the executable in between, sometimes it doesn’t run: it just gets “killed (9)” on startup. I failed to discover why and I failed to reproduce it just now in a test build. I guess if that happens, delete the executable between builds.

* * *

Bonus grumble about Mac trackpad and mouse options

This is not useful content. Please do not attempt to read it

I haven’t used a Mac in such earnest for a while now, so of course I’ve been rediscovering things about macOS that I don’t get on with. One that I find particularly maddening is the way it handles scroll direction for the trackpad and an external mouse.

I switch between the two a lot, and I like to use the “natural scrolling” direction (touchscreen-like, so your fingers are “pushing” the content) with the trackpad, but the opposite with the mouse, which has a scroll wheel or wheel-like scrolling zone whose behaviour I became accustomed to before touchscreen devices started sprouting everywhere.

Fortunately, macOS provides separate touchpad and mouse sections in the system preferences, which contain separate switches for the scroll direction of the trackpad and mouse respectively.

Unfortunately, when you change one of them, the other one changes as well. They aren’t separate options at all – they’re just two different switches in different windows that happen to control the same single internal option! So every time I go from trackpad to mouse or back again, I have to also go to system preferences and switch the scroll direction by hand. That is so stupid.

(Linux and Windows both have separate options that actually work as separate options. Of course they do. Why would they not?)