Your Brain on Music: The Neuroscience of Listening and Playing
Section 4 of 10

Your Brain Is Guessing the Next Note

13 min listen Updated

Sing a scale in your head, the plain one that goes do, re, mi. Climb it one step at a time, and stop on the seventh note, ti. Leave off the last one. You can feel it waiting. The tune leans toward that top do like a sentence missing its last word. You feel that pull whether or not you ever had a music lesson. Nobody sat you down and told you that ti leads to do. And a singer who ended the climb on a note from outside the scale would sound wrong to you.

The last lecture ended on a question about exactly this feeling. How does a brain that has never studied music know which note is wrong? Inside it sits a larger question. Is the wrongness of a wrong note in the sound, or in you? Some chords seem sweet and others sour, as if by nature. Does every ear on earth agree?

We begin with an old theory of musical feeling, and with babies who pick up the odds of sounds without trying. Between them, they explain where expectations come from. A computer program that bets on every next note puts numbers on those expectations. A CD mailed to new parents reveals how early the learning sets. And villages in the Bolivian rainforest give the question its hardest test.

Leonard Meyer had a theory about that pull. In 1956 he published a book called Emotion and Meaning in Music. His idea was that you feel something when a response you were leaning toward gets blocked. The scale that stops on ti is his kind of case. The top note you expect never comes, and what you feel is a small frustration. Meyer argued that the meaning music has within itself comes from expectation too. A note means what it leads you to expect.

Where do the expectations come from? Meyer named two sources. Some come from habits of perception that the mind brings to any pattern. One such habit is to expect a pattern to carry on as it began, so a line that climbs should keep climbing. Other expectations are learned from a particular style of music. Meyer was sure the learned kind mattered, and he doubted any account that left it out. The question under this whole lecture is how much is built in, and how much is learned.

The learned kind now has a name, statistical learning. It means picking up the odds of what you hear just by hearing it, without trying and without knowing you're doing it. Jenny Saffran is a psychologist who studies how babies learn. In 1996 Saffran and her colleagues played eight-month-old babies a stream of nonsense syllables, with no pauses. The syllables made up nonsense words, something like golabu, all run together. Inside a word, one syllable reliably led to the next. Across the join between two words, what came next was less predictable. That was the only clue to where the words were. Two minutes of listening was enough. Afterwards the babies could tell a made-up word from a string of the same syllables that ran across a join. Nobody taught them, and nobody told them when they were right. Three years later, the team ran the same test with short groups of musical tones in place of syllables. The babies learned those too.

Music gives this kind of counting plenty to work on. Take a key. A key is the family of notes a tune is built from, with one note as its home. In Western music, the notes that anchor a key turn up more often than the rest. They also tend to land on strong beats and at the ends of phrases. The do of your scale is the home note, and tunes tend to come to rest on it. Children probably learn keys from these odds. Babies of seven months already link particular pitches with strong beats. Western adults with no training spot sour notes that fall outside the key. Recordings of the brain's electrical activity show it flagging an unexpected chord even while they're busy with something else.

One set of odds can't serve every kind of music, though. The music scholar David Huron argues that we keep a separate set of rules for each style we know. One reviewer of his book gave the reason. Suppose your brain averaged everything you'd ever heard. Then Beethoven would set up much the same expectations as Britney Spears. He doesn't.

Meyer himself tied expectation to probability, in an article the year after his book. What was missing, until fairly recently, was a model that could learn the probabilities the way a listener does. In London, the cognitive scientist Marcus Pearce built a computer program that does. It learns the way Saffran's babies did, by counting. It was fed 903 folk songs and hymn tunes. From them it counted which notes tend to follow which, over stretches of one note, two notes or more. The program gives each possible next note a chance before it arrives.

Those chances give two numbers, and this course will keep both. Treat the program as a gambler, spreading chips across the possible notes. Uncertainty is how thinly the chips are spread before the note arrives. Uncertainty peaks when every note looks as likely as every other. It's low when one note is the obvious bet. Surprise comes after, when the note arrives. It measures how little was staked on the note that came. Uncertainty belongs to the moment before a note. Surprise belongs to the note itself.

Your scale shows both. Stop on ti, and a gambler who knew your music would pile the chips on do, so uncertainty is low. A singer who then lands outside the scale lands where almost nothing was staked. That's high surprise.

The gambler is more than a figure of speech. In one experiment, fifteen people played a real betting game with a hymn tune. Before each note, they staked play money on which of twenty pitches would come next. They kept betting until they got it right. A bigger stake on the right note won more, so their winnings showed how expected each note was. The program was tested on the same tune. Across studies of several kinds, from betting games to brain recordings, its surprise accounts for up to 83 percent of the differences in how strongly listeners expected notes.

The program can also be checked against Meyer's own ear, on a tune from Schubert's Octet. The case turns on an octave. Two notes are an octave apart when one repeats twice as fast as the other. The violin's A at 440 and the A at 880 above it are an octave apart. Western music gives both the same letter. In another book, Explaining Music, Meyer analyzed the tune. In its second bar, the melody moves through the notes of a chord, and the next note should be a high G. It is a G, but an octave too low. Meyer wrote that the continuation was "in the wrong register," the right note at the wrong height. The program has never read Meyer, and it agrees. It gave the high G nearly nineteen chances in a hundred. It gave the low G that Schubert wrote fewer than two.

The brain keeps the same score. One team recorded listeners through electrodes on the scalp while they heard 58 hymn tunes. About a tenth of a second after each note, the brain's electrical response dipped. The more surprising the program found the note, the deeper the dip.

Pearce himself gives the caution that goes with all of this. A program that fits Western listeners shows that counting could explain what they hear. It doesn't show that counting is how they learned it. That takes an experiment that changes the listening itself.

That answers the last lecture's question, at least in outline. A brain that never studied music has been studying all along. It counts what follows what in the music around it, and it bets on every next note. A wrong note is one it gave long odds. What the program can't tell us is when the counting happens, or whether it reaches what sounds pleasant. Does any of right and wrong come with the ear itself?

When does the counting start, and when does it set? On listening tests, the sense of key takes years. Babies of eight months notice when one note in a short tune is changed. To them, a note that breaks the key is just another change. Adults, with or without training, pick out the key-breaking note more easily. North American children do so by age five. By six or seven, they also catch a note that clashes with the chord beneath it. Brain recordings find some sense of harmony earlier still.

Rhythm narrows sooner on these tests, and the psychologists Erin Hannon and Sandra Trehub tracked it in babies. Western dance tunes mostly keep their beats evenly spaced. Many Balkan dance tunes don't. Their bars run to seven quick counts, grouped three, two, two. Earlier work had found that North American adults notice a broken Western rhythm but miss a broken Balkan one. Adults from Bulgaria and Macedonia notice both, and so do North American babies of six months. In 2005 Hannon and Trehub tested babies of twelve months. Each bar of the test tunes got one extra note. In one version the rhythm made room for it. In the other, the extra note broke the rhythm. Place your own bet before the answer. The twelve-month-olds noticed a break in the even, Western kind of rhythm, and not in the Balkan kind. All the test tunes were Balkan folk tunes, some with even rhythms and some with uneven ones. In six months, they had come to hear like North American adults.

Then Hannon and Trehub ran the kind of experiment Pearce's caution calls for. They changed what the babies heard. Two weeks before a lab visit, they mailed families a CD, with instructions and a log sheet. It held about ten minutes of dance music from Macedonia, Bulgaria and Bosnia, and none of the test tunes. Parents were asked to play it every morning and every afternoon, while the baby was alert and content. They were asked not to draw attention to it. After two weeks, the twelve-month-olds noticed a broken Balkan rhythm as well as other babies noticed a broken Western one.

Forty college students got their chance too. Half of them took the same kind of CD home for one or two weeks. They could earn up to ten dollars for recognizing its tunes afterwards. They improved a little, but not enough to beat guessing. Adults in the study even rated the broken Balkan version as closer to the original, because it fit a Western pattern. You might suspect the students just listened less. But how much each one listened made no difference to how much they improved. The researchers add that money was at stake for the students, so they were likely listening on purpose. The babies heard the music in the background.

Hannon and Trehub's reading is that years of Western music leave adults committed to its patterns. A foreign rhythm gets bent to fit a familiar one. A baby's patterns are still loose enough to reset. Two weeks were enough for a baby, and not for an adult. Rhythm and key were measured by different tests, though, so nobody can yet say which truly comes first.

Rhythm is one thing. Surely some notes just sound good together. The followers of Pythagoras thought so, and credited simple numbers. Modern theories look to the ear instead. An octave's two notes repeat at two to one, and a fifth's at three to two, like C with G. Western listeners call pairs like these consonant, meaning pleasing together. They call pairs without such simple ratios dissonant, and C with F-sharp is one of those. From here on, call them sweet pairs and grating pairs. If the liking were built into the ear, every ear should share it. The test needs listeners who have barely heard Western music. The last lecture introduced the MIT scientist Josh McDermott. He names the difficulty: "It's pretty hard to find people who don't have a lot of exposure to Western pop music."

An anthropologist who had long studied the Tsimane asked McDermott to help test how they respond to music. The Tsimane are farmers and foragers in the Bolivian Amazon, and most have heard little Western music. Their music is usually made by one person at a time, so they rarely hear two notes at once. In 2016 the team reported what Tsimane listeners made of sweet and grating pairs. On average, they liked the grating pairs as much as the sweet ones.

Bolivians in a nearby town and in La Paz showed a small preference for the sweet pairs, and Americans the largest. Did the Tsimane misunderstand the question? The checks suggest not. They rated laughter above gasps, like everyone else, and they disliked roughness as much as anyone. Roughness is the fast flutter in loudness when two tones are close in pitch but not the same.

Liking a sound and hearing it are different questions. In August 2019, McDermott's lab went back to ask what the Tsimane hear. They chose villages with no electricity or phone signal. Some were two days' walk from the nearest town, and others two or three days by motor canoe. Each listener wore closed headphones, with a translator alongside. Two notes sounded together. Was it one sound or two? And how much did the listener like it? Thirty-one Tsimane took part, along with twenty-eight Bostonians with no musical training.

Both groups heard the sweet pairs as one sound more often than the grating ones. Both heard the octave as one sound most often of all. Hearing two notes as one is called fusion, and this trip was the fusion study. But only the Bostonians liked the sweet pairs better. The lab also tested a hundred more North Americans online. Even in them, fusion and liking came apart. A person who fused the sweet pairs strongly was no more likely to like them strongly.

Why would sweet pairs fuse? The answer builds on the first lecture. Notes from voices and from instruments like the violin are stacks of harmonics, at whole-number multiples of the lowest frequency. The brain groups frequencies that fit one stack into one sound. That's how it tells two voices apart, in the rainforest as in Boston. Take the violin's A at 440 and the A an octave up, at 880. The harmonics of the higher A run 880, 1,760 and on up. All of them sit in the lower A's stack already. Together the two notes look like a single note. A fifth fits one stack nearly as well, and a grating pair hardly at all.

The octave has one more lesson. Played together, its two notes fused for listeners in both places. Played one after the other, they may not count as the same note. In another study, McDermott's lab asked people to sing back pairs of tones, some far above the range of any voice. Americans tended to land on the same letter, a whole number of octaves away. The Tsimane didn't, not even their most accurate singers. The team tried coaching, with a recorded voice saying "Excellent!", "Good" or "OK" after each try. They left out "Bad", so as not to discourage anyone. It changed little. The lab reads this as a sign that hearing an octave as the same note comes from experience. Singing back isn't the same as hearing, though, and an older study found young French infants treating octaves alike.

The music researcher Imre Lahdelma set out the critics' case against the 2016 study of liking. First, it hadn't tested the octave, the sweetest pair of all. The fusion study did, and the Tsimane still showed no preference for the sweet pairs. Second, the Tsimane did prefer a fifth to one pair, the harshest in Western music. The 2016 paper hadn't discussed that. Third, the critics point to babies. Several studies found infants looking longer toward sweet pairs. A later study found no such preference, only one for music the babies had heard before.

Lahdelma's own complaint was that the 2016 study left out truly rough chords. So Lahdelma's team tested two remote peoples in northwest Pakistan, and listeners in Britain, on chords. Everyone disliked a dense cluster of neighboring notes. Only the British preferred the major chord, a standard sweet chord of three notes.

To my mind, the critics win one point. Every group in these studies dislikes roughness. So far, they haven't won the larger one, that we're born liking the sweet pairs. McDermott's lab suggests why the liking varies: familiarity. On that account, we come to like the pairs we hear most, and Western music is full of sweet ones. The lab has looked for that among the Tsimane themselves. Villagers with more contact with towns, phones and churches showed a small preference for the sweet pairs, and the others none. The gap between the two groups was too small to be sure of, and most of this work comes from one lab.

A brain with no lessons still has an education. Meyer put musical feeling and meaning in expectation, the lean toward a note that may not come. Statistical learning supplies the lean. You absorb which notes follow which, as babies find words in a stream of syllables. A program that counts the same way scores every note twice. Uncertainty is how widely it spreads its bets before a note. Surprise is how little it put on the note that landed. On this evidence, a wrong note is mostly a long shot. Rhythm narrows to your own music within the first year. Key takes until about five, on the tests so far. And the ear's share is smaller than it feels. Fusion followed the same pattern in Boston and the Amazon, and so did a dislike of roughness. Liking the sweet pairs did not. Nor did singing an octave back as the same note, which Americans did and the Tsimane didn't.

The gambler still has to be paid. Huron argues that the brain rewards a correct prediction, so we feel good when we're proved right. If he's right, a good bet should pay out somewhere in the brain. The next lecture follows the payout, into what happens when music gives you chills.

How much of the liking is built in is still argued, and both camps say the chance to settle it is shrinking. Between 2019 and 2021, the Tsimane village of Mara got solar panels and phone signal. The Tsimane hear a sweet pair blend into one sound, as Bostonians do. What may be changing, McDermott's lab thinks, is whether they like it.