Clojure Sound - 5 - Double Click with Foot Control

Need help with your custom Clojure software? I'm open to (selected) contract work.

July 30, 2022

Please share: Twitter.

These books fund my work! Please check them out.

In the last article we created a receiver function that listened to signals from our foot controller and started or stopped playback on consecutive clicks. The trouble with our player it that it only works until the end of the track. The start! function does not automatically rewind the playback. Neither stop! does that. Our program is responsible for detecting that the track should be rewound, perhaps by detecting that we reached the end of track (but even that is not fool proof, since audio infrastructure is not that precise). Now we have to think about a foot user interface that is useful and simple at the same time - there's not many different precise actions that a foot can do.

My idea at this time is the following: distinguish three gestures:

Button (un)pressed (value 0)
Button pressed (value 127)
Button pressed twice in a short time span (doubleclick). This can have two varieties:
- First pressed (value 127), then (un)pressed (value 0)
- (Un)pressed (value 0), then pressed (value 127).
- The values 0 and 127 always interchange, there's no way to send the same value twice in a row (with the MIDI controller I have).

This gives us 4 different signals from one button that we can work with, which doesn't seem that much, but on the other hand, we can't assume that the feet of our user that is doing all this stomping is able or eager for much more complicated stuff.

So, if I assume that the particular button is dedicated to a particular clip playback, I see the following actions:

If the button is (un)pressed (value 0), the clip should stop, regardless of the previous state. If the clip has not been playing, nothing changes.
If the button is pressed (value 127), the clip should start, regardless of the previous state. If the clip has been playing, it just keeps playing.
If the button is clicked twice in a row, the things might get complicated!

It's not exactly rocket science level complicated, but we still need to think what to do. In our desktop user interface toolkits, we are accustomed to working with a very high level API. We write functions that react to gestures (click, left click, right click, double click, wheel scroll, touch, etc.) but we are not responsible for detecting these gestures from the signals that our Magic Trackpad sends; the drivers and the operating system take care of that complexity. We just write actions that happen onDoubleclick.

But here, we have to take care of the whole stream of raw signals. How should we distinguish between an action that happens when the button is clicked once, and an action that should register that first click, and wait for the second one, which may never come? And, we would like the solution to be simple!

I drew a simple state transition diagram (with a pen on a napkin) and concluded that for our limited foot control it can be simplified to this:

As soon as the button is pressed, evaluate either start! (value 127), or stop! (value 0) function.
When the button is pressed shortly after it has been pressed the previous time (say, 600 milliseconds), treat it as a double click, and initiate the alternative action.

This leaves us with two tasks:

Detecting doubleclick
Deciding on a simple decision process for the alternative action (remember, the user should anticipate the action while playing the guitar).

The usual imports:

(ns my-midi)
(require '[uncomplicate.commons.core :refer [close! info]]
         '[uncomplicate.clojure-sound
           [core :refer :all]
           [midi :refer :all]
           [sampled :refer :all]])

Detecting doubleclick

If you went back to one of the previous articles and connect the println as a receiver for the controller you have, you'd see that each message contains its timestamps in microseconds. We can detect whether a "click" message comes shortly after the previous "unclick" simply by subtracting the previous message timestamp. The only trouble is that we don't have access to message history.

If we saved the last message, we have all the information we need; not only that we know both timestamps, we can even recognized whether the last message is of the same kind, and coming from the same button! Fast pressing different buttons does not count as "doubleclick". Clojure gives us several elegant facilities for state management. In this case, the state is internal, so we don't have to care about synchronization. We don't need what refs, agents, or atoms offer. I think that the simple volatile! can serve our needs rather well.

(defn play [line]
  (let [previous-message (volatile! (short-message :control-change 0 0))
        previous-timestamp (volatile! 0)]
    (fn [message timestamp]
      (future
        (case (command-type-key (status message))
          :control-change (let [doubleclick (and (= (data1 @previous-message) (data1 message))
                                                 (< (- timestamp @previous-timestamp) 600000))]
                            (vreset! previous-timestamp timestamp)
                            (vreset! previous-message message)
                            (case (data1 message)
                              80 (play-control (data2 message) repeated)
                              true))
          :program-change "We'll do something with other buttons later!"
          :default)))))

Building on the play function from the last article, I've added closures via let to keep track of the previous message and its timestamp, and a simple case to distinguish :control-change vs :program-change messages. I've also delegated the action logic to an extracted function play-control.

Alternative action

The play-control function receives the value and whether the click is double. What should it do? Of course, the first click starts or stops the clip. But, what happens with doubleclick when value is 0, and what when value is 127? The first detailed state transition chart was not that trivial. It all depends whether the clip position was 0, maximum value, or something in between. My idea is to enable rewind, so the main theme is "doubleclick should rewind the clip". But, is there a difference whether the clip was stopped or not?

For example, let's say that the clip was in the middle, and stopped. The first click starts it, and the second is detected after 300 milliseconds. We can rewind the clip, and it's logical to me that the clip should stop, instead of instantly emitting sound. After I analyzed all other situations, it turned out that the simplest logic does quite a logical thing.

I left out the state transition diagram and my analysis on purpose. I hope that this inspire you to think about your own applications that use similar controllers. It's unlikely that you practice guitar at this moment, and it's equally unlikely that you have the same foot controller. Or you do! Anyway, here's very simple logic of my play-control:

(defn play-control [clip ^long v rewind]
    (if (< 64 v)
      (start! clip)
      (stop! clip))
    (when (or rewind (< 0 (frame-length clip) (+ (frame-position clip) 1000000)))
      (frame-position! clip 0))
  clip)

Here's what it does. At the start, the clip is not playing, and the button 80 is either in on (value 127) or off state (value 0), depending on the state you left it when you used it prior to that. If the button is in on state, the first click stops the clip, effectively doing nothing that changes what you hear, since the clip was stopped anyway. That was needed because on my controller, there is a red led light that indicates the on/off state, and I want it to match the playback.

Then, when you press the button again, the clip starts playing. Whenever you press it again, it starts or stops the playback, matching the red light. But, you might want to rewind it for whatever reason in the middle of the clip. If the clip was playing, the first click will stop it, and the subsequent (double) click will start it again, and immediately move its position to the beginning. If the clip was stopped, the first click will start it, but the second will stop it immediately, and rewind it, so it's ready to play when you click it the third time. Effectively, if you press the button twice in a short timeframe in the middle of the clip, it will just continue what it did (play or wait) but from the beginning.

If the clip reaches the end, it will be silent, but the red light will still be bright (there's no way I know of that I can direct my controller to change that, as it seems to not listen to MIDI commands). The first click will issue the stop command, and the second will start the playback. Now, I decided to relax the requirement for rewind in that case. Even if these two commands were not issued quickly, there's no reason for the clip to stay at the end, because there's no sound there whatever you do, which is not very useful (in this use case).

If the clip somehow ended in end of clip, but stopped state, with the led light off, the first click will start it, and immediately rewind it to the beginning.

I have also noticed that the clip that is very close to its end, but not exactly there, is a good candidate for rewinding. So, everything under a second until the end is treated as "rewind whatever happens" (in this use case!).

At the end, the playback logic turned out to be much simpler than I anticipated when I analyzed all states where clip and the controller (with its red LED light) can be. It is an easy trap to overcomplicate things and create sophisticated universal solutions. In this case, even if we had a more high level API, it would need to be configured, and we would miss the special case when the clip is near the end. Sometimes the humble code is the right stuff. Clojure and simple go well together!