The cat has good tasteThe cat has good taste

This site has a song. Nothing plays until you ask for it, and there’s no button in the header saying so. You pull the cord at the top of the page, or press M, and the cat puts his headphones on. It’s also right here, and you should press play now and leave it on, because everything in this note is listening to it.

mr.ledger 2↳ the cat has good taste

0:00

2:40

Press play and the page starts moving, all of it, in time. It looks like one effect. It’s really about a dozen small decisions about what should hear what, and most of them were decisions to make things hear less.

Everything that hears itEverything that hears it

Here’s all of it, running. Some of these light up on a beat, and some of them follow the sound continuously. That split is the whole design, so it’s worth seeing before anything else.

A browser can tell you how loud each pitch is at this exact moment, and that’s all it can tell you. It doesn’t know what a kick is. It can’t tell a drum from a bass note on the same pitch, and it certainly can’t tell you a beat is coming. For bars that’s fine: bars are supposed to show everything. For letters it’s hopeless. Letters that react to everything just shake, so they needed something the browser can’t give them: a list of the moments that matter, written before the song starts.

Taking the song apartTaking the song apart

You can’t list the drums of a finished song by looking at the song: by then everything is one wave. So the first step happens long before anyone opens the site. A separation model, trained to pull a mix apart, splits it into four files of exactly the same length: vocals, drums, bass, and everything else.

That picture is the argument for everything that follows. Look at the lanes. Vocals and the rest are long, soft shapes that come and go in phrases. The drums are the one stem made of separate spikes, and each spike is a moment you could clap on. The two thin rows underneath are what I keep from all this: every kick, and every snare and hat, with a time and a strength.

The stems never reach your browser. They’re 28 MB each, they stay on my machine, and what comes out of them is a list of numbers: 248 low hits and 917 high ones, in 16 KB. The site still only plays the one mixed file.

{"low":[[1.689,0.55],[1.933,0.59],[4.841,0.64],[7.993,0.66], …

Which sounds get a sayWhich sounds get a say

My first idea skipped all of that: if something is loud right now, the letters react. Then I made a better version that listened to all four stems. Then I threw most of it away. They’re all below, on the same title, and the song decides. Press play and switch between them.

“Every sound” is what you’d build first, and it looks like a broken screen. There’s always something loud, so nothing ever stands out.

“All four stems” looks good until you listen. It took the kick and the bass for low hits, and the high end of the drums, the vocals and the rest for the sparks. It’s much calmer, and every hit is real. But the bass line is a melody, not a rhythm: of the 364 low hits, 116 are notes you can hear but couldn’t clap along to. Take those out and you get the 248 from before. The title moves at moments your body didn’t expect, and it reads as nervous.

“Only the drums” is the one I kept. It barely looks different on paper, and it’s the difference between a title that’s busy and a title that’s playing the song. So now there’s a rule: the letters only listen to the drums, and the bars keep the whole song. Rhythm goes to what jumps, and everything else goes to what flows.

// All four stems: the first version
const low = envelope(decode([file("bass"), file("drums")], "…lowpass=f=170…lowpass=f=140…amix…"));
const high = envelope(decode([file("drums"), file("other"), file("vocals")], "…highpass=f=5000…"));

// --drums: the one that plays. The kick below, snare and hats above.
const low = envelope(decode([file("drums")], "[0:a]lowpass=f=140,lowpass=f=140[o]"));
const high = envelope(decode([file("drums")], "[0:a]highpass=f=2500,highpass=f=2500[o]"));

Finding a hitFinding a hit

Even a drum stem is only a wave. To turn it into a list, I cut it in two with filters: under 140 Hz there’s almost only the kick, and over 2500 Hz there are the snare and the hats. Then a script walks through each half, about 170 steps a second, looking for jumps in loudness. These are eight real seconds of the song, and the script’s real steps, running in your browser.

The top line is how loud the sound is. The gray bars underneath are how quickly it’s getting louder, which is what a hit actually is: not a loud moment, but a sudden one. The blue line is the bar a jump has to clear, and it isn’t fixed. It’s worked out from the half second around each moment, so a busy passage needs a bigger jump than a quiet one to count.

Move “Picky” down and it starts seeing hits everywhere. Move it up and it misses the quiet ones. Then switch to “The song” and look at how much less clean that top line is, with everything else playing over the kick. Press “Hear only this” while you’re there, it’s a strange way to hear a song.

// A hit is a jump in loudness that stands out from the half second around it.
const rise = db.map((v, i) => Math.max(0, v - (db[i - 4] ?? v)));
const limit = mean + sensitivity * Math.sqrt(Math.max(0, sq / n - mean * mean));
if (rise[i] < limit || rise[i] < 2.5 || i * STEP - last < gapS) continue;

Two small things make the list usable. A hit’s strength is measured against the whole song and not against its neighbors, so a quiet verse stays small and a drop hits hard. And the stems have to line up with the file that really plays, which the script checks by sliding one against the other until they match best. Here, that’s six milliseconds. You’d never see it listed anywhere, but it’s the kind of thing you check once, so you can stop wondering.

Two things a letter can doTwo things a letter can do

A list of hits is still not an animation, so the next question was what a hit should look like. The drums make two very different sounds, and I wanted you to be able to tell them apart with the sound off. So they get opposite gestures. Try them, with your own fingers.

A kick is big and it has a place. It picks a spot, spreads outward from there in one color, and the letter in the middle goes first, jumps highest and stays longest. The harder the hit, the further it goes. A hat is the opposite: one letter, for an instant, in its own color. It also goes as far as possible from the last one, so that two quick hats read as two sparks and not as one thing moving.

const count = Math.round(least + (pool.length * most - least) * strength ** 1.4);
const center = Math.floor(Math.random() * pool.length);
const color = pick(SCR.colors);

pool.forEach((L, i) => {
  const k = Math.abs(i - center) / Math.max(reach, 1); // the middle hits first, hardest and longest
  const lift = (hop + (strongHop - hop) * strength) * (1 - k * 0.7);
  setTimeout(() => flash(L, color, hold * (1 - k * 0.5), lift), k * spreadMs);
});

There’s no fade and no easing back into place. A letter is scrambled, and a moment later it’s the letter again. Between hits the title is completely plain, and I’d say that’s the most important part: the silence is what makes the hits felt. It also keeps the words readable, since a title that’s always moving is a title nobody can read.

Only letters on screen are ever touched, and if you’re at the top of the home page, it’s only the big sentence. The cat nods on the kick. The cursor gets a push on every hit, a big one for a kick and a tiny one for a hat. All of that is the same list being read in different places.

A tenth of a second earlyA tenth of a second early

Once all of that worked, it still felt wrong. The letters were on time, I checked, and they looked late. Here’s a kick and a title, so you can feel it yourself. Start it and move the letters around.

“On time” feels like the letters are following the kick. “100 ms late” feels broken. And somewhere around 100 ms early, something strange happens: they stop looking early. Your brain doesn’t file the sound and the movement as two events any more, and it just becomes one thing that makes a noise. So every letter on this site moves a tenth of a second before the drum it’s reacting to.

const HIT_LEAD_S = 0.1; // the letters move this long before the sound: on time, they feel late

// What's heard is a little behind what's playing, so the hits wait for the speakers, minus the lead.
export const musicHitTime = () =>
  audio.currentTime - (context.outputLatency || context.baseLatency || 0) + HIT_LEAD_S;

This is the one trick a list makes possible and a live signal doesn’t. To be early, you have to know the future, and a list does.

What a waveform actually isWhat a waveform actually is

Those stems up there look like solid shapes, and it’s easy to forget what they’re drawn from. Sound in a file is just a list of numbers: how far the speaker should be pushed, thousands of times a second. This is the song, with nothing done to it. Every pixel across is one number, and the line goes through all of them.

Now zoom out. There’s soon more than one number per pixel, and a line through all of them turns into a scribble, while one picked at random lies about the shape. So you do what every audio app does: for each pixel, you find the loudest number going up and the loudest going down, and draw a line between them. That’s the “Loudest per pixel” view, and it’s what most people mean by a waveform. Bars are the same thing with a few pixels grouped together, which is the version you see in every app now, because it’s easier on the eyes.

The whole song is about seven million numbers per channel. The lanes in the picture above are 420 numbers each. Same idea, one more step out.

// The loudest moment of each slice, as a whole percentage of the song's loudest.
function lane(samples, top) {
  const size = Math.floor(samples.length / LANE_BARS);
  return Array.from({ length: LANE_BARS }, (_, bar) => {
    let peak = 0;
    for (let i = bar * size; i < (bar + 1) * size; i += 8) peak = Math.max(peak, Math.abs(samples[i]));
    return Math.round((peak / top) * 99);
  });
}

The bars aren’t a waveformThe bars aren’t a waveform

This surprises people, because they look like one. A waveform shows loudness over time, so it needs to know the whole song. The bars show loudness over pitch, at one instant, so they’re redrawn from nothing on every frame. They’re from the same list of numbers, read the other way around.

The browser does the hard part. Hand an analyzer 1024 samples and it gives back 512 numbers, one per slice of pitch, from the lowest to the highest. I use the lower part and group them into 32 bands. Point at the bars below: the strip under them shows how many numbers each band takes in.

They’re not equal, and that’s on purpose. Going up an octave doubles the frequency, so the distance between two low notes is tiny in hertz, and the same distance up high is enormous. Equal slices would give the kick and the bass one bar between them, and hand half the screen to a hiss. So bands widen as they go up: the first ones hold a single number, and the last one holds seventeen.

// Bands widen with the pitch, the way we hear it.
const from = Math.floor(usable * (i / BANDS) ** 2);
const to = Math.max(from + 1, Math.floor(usable * ((i + 1) / BANDS) ** 2));

That wasn’t enough, and I found out late. For a long time the right end of the ruler barely moved, and neither did the right of the cursor’s bars. I assumed they were only getting the drums. They were getting everything: the song was fine, and my bars were wrong. High notes are simply much quieter than low ones. Switch to “As loud as it is” and you’ll see what I had shipped.

Now each band is measured against its own last few seconds, with its own loudest and quietest. A kick and a hi-hat both fill their bar. In the middle of the song, the right-hand bars went from traveling about a fifth of their height to about four fifths.

// Each band is measured against its own recent range: a loud kick and a faint hi-hat both fill their bar.
quietest[i] = Math.min(level, quietest[i] + (level - quietest[i]) * forget);
loudest[i] = Math.max(level, loudest[i] + (level - loudest[i]) * forget);
music.spectrum[i] = (level - quietest[i]) / Math.max(loudest[i] - quietest[i], LEVEL_MIN_RANGE);

The minimum range matters more than it looks. Without it, a note held perfectly still would have its tiny wobbles blown up to full height. The ticks at the bottom of that demo are the ruler from the home page, reading the same 32 bands from left to right. And the analyzer sits before the volume, so if you turn the song down, the page keeps dancing just as hard.

A cursor that playsA cursor that plays

While the song plays, the little island that follows your cursor stops being a label and turns into an equalizer. You’re probably looking at it now, if you moved your mouse. Here’s the same one, held still, with the two settings I went back and forth on.

I started with too many bars, and too tall. At the size of a cursor, a lot of thin lines is a gray smudge, and it pushes the label around for no reason. Twelve bars at 16 pixels is where it stopped looking like a chart and started looking like a thing that’s alive. They read twelve of the 32 bands, spread out from low to high, and every bar is a CSS scale of the same element, so nothing changes size and the island keeps its width.

“My first try” is that same bug from before, seen up close: a square root and a boost to help the high notes, and the result sits near the top and shivers.

const NOW_PLAYING: Live = {
  frames: () => eqMarkup((band) => 0.14 + music.spectrum[band] * 0.86),
  every: 50,
  widest: eqMarkup(() => 1),
};

const stopLow = onHit("low", (strength) => kick(0.05 + strength * 0.12));
const stopHigh = onHit("high", (strength) => kick(0.012 + strength * 0.022));

It’s redrawn twenty times a second, not sixty, since the island’s content already has a way of updating itself and nobody can see the difference at that size. And it doesn’t stop listening to the list either: every hit also gives the whole cursor a small push, through the same spring as a click. So your pointer hears both things, the bars and the beat.

A cord, not a buttonA cord, not a button

A play button says what it does. A cord makes you wonder, which is why it’s worth having for something nobody needs. It starts out hidden, with only its end showing. Point at it and you see what it is. Pull it and it unrolls, and your cursor becomes a paw that closes around the ball, with three small marks for the effort.

It’s a real rope, simulated at a fixed 120 steps a second. My first version pushed its links apart as well as pulling them together, and it couldn’t fall properly: a rope doesn’t resist being folded. Once the links could only pull, it hung like one.

Pause the song and the controls wait, then leave together. Nothing goes while your pointer is near it.

const PARK_AFTER_MS = 2600; // paused this long, the song's controls go back into hiding

// Nothing hides from under a pointer: whoever's being held off asks to wait.
export function holdParking(who: string, holding: boolean) {

What it costsWhat it costs

Very little, which matters for a joke nobody asked for. Nothing loads before you ask: no audio, no analyzer, not even the list. After that there’s one audio element, one analyzer and one list that everything shares, and the work stops when the song does. If you’ve asked your system for less motion, the song still plays and the page stays still, and it doesn’t even build the analyzer.

It’s only the one song for now. The player has room for more, and the scripts only need the next song’s stems. The cat will be consulted.