How to Write a Family Memoir: Turning Your Stories into a Book
Section 3 of 15

How Memory Works When Writing a Memoir

7 min listen Updated

As Karr describes in The Art of Memoir — a single image, a whiff of cumin, can split open the hard seed of the past. That's the thing standing between the observer and the meaning being sought. Memory feels like a film reel that can be rewound and played back. It is nothing of the kind. And once the actual nature of memory becomes clear — a thing the brain builds fresh every single time it is reached for — the whole project of writing from the past stops feeling like a transcription task and starts feeling like what it really is: an act of honest reconstruction.

Consider the demonstration Karr runs on the first day of her memoir class, because it's the cleanest proof anyone has ever staged.

Karr teaches some of the most selective graduate writing students in the country. One year, nearly eight hundred people applied for twelve seats. These are not careless people. And on the first day, she stages a demonstration designed to disrupt their certainty about the unassailability of their own memories. A colleague, a tall, bald, easygoing poet she calls Chris, storms into the seminar room mid-class and claims it's his room. They argue. They play against type. He raises his voice. He tells her to go to hell — or, as Karr slyly writes, is that only how she remembers it? He throws a sheaf of papers in the air and stalks out. The students sit there agog. A videographer in the back has been recording the whole thing.

Then Karr asks every student to write down, immediately, exactly what just happened.

And here's the part that should rattle anyone who plans to write a single true sentence about their own life. These brilliant, attentive students — recording an event that happened ninety seconds ago, in front of their eyes, with a camera rolling to check against — get it wrong. The mistakes, Karr writes, pop up like dandelion greens. Who said what. Who stepped toward whom. The order of things. The exact words. Twenty trained observers, one shared event, twenty different accounts. Now stretch that ninety seconds out to forty years. That is the actual condition under which every memoir gets written.

So what's going on in there? Why can't the brain just keep a clean copy?

Here's the kitchen-table version. Memory isn't a filing cabinet where the past sits waiting, intact, for retrieval. It's more like a recipe. The brain doesn't store the finished dish. It stores a scatter of ingredients — a smell, a color, a snatch of dialogue, the feel of cold linoleum — tagged and tucked away in different drawers. And every time someone "remembers," the brain grabs those scattered ingredients and cooks the dish again, right there on the spot. The memory that gets experienced is the meal the brain just made, not a meal it stored.

Now the precise version. The research is clear that memory is reconstructed at the moment of retrieval, not retrieved whole. A piece in The New Yorker, drawing on new research into early childhood memory, lays out the mechanism plainly: the brain takes sensory fragments and tags them with specific associations. When one cue fires — a smell, the sight of a familiar place — the brain reassembles the other fragments carrying the same tag. That's why Karr's whiff of cumin works. It's not poetry. It's neuroscience. The cue fires, and the fragments stored under that tag come flooding back together. And the more often a given memory gets assembled, the stronger it gets — which sounds reassuring until the catch becomes clear.

Here's the catch, and it's the part that should change how one writes. Because the brain is reassembling scattered fragments every single time, it often introduces distortions. And once a distorted version gets recalled often enough, it sets. The New Yorker piece states it about as bluntly as science gets: a distorted synthesis of fragments can become a repeatedly recalled memory — vivid, detailed, emotionally certain — and still be false. Read that twice. The memories one is surest about, the ones told the most times at family dinners, polished smooth with retelling, are the ones that have been reassembled the most. Which means they're the ones most likely to have drifted from what happened. Confidence is not evidence. The vividness felt is the brain's craftsmanship, not its accuracy.

Stay with this for one more step, because it answers a question that haunts a lot of would-be memoirists: what about the years that can't be reached at all?

If an earliest memory is fuzzy or feels secondhand, that person is not broken and is not a bad rememberer. There's a name for this — infantile amnesia — and it's universal. The writer of that New Yorker essay describes her own earliest memory: helping her mother close a pasture gate, feeding a horse a carrot, age two or three. But she says "might be," because she's also seen the photographs of exactly that scene. Do those photographs become the memory, or is the memory something separate? Scientists have offered competing explanations. Some think memory formation needs a more mature grasp of language than a toddler has. Others think small children can hold fragments but can't yet weave them into the story of an autonomous self. Newer work suggests very young children can form memories, but the neural pathways to reassemble them are new and fragile — easy to lose. Whichever is right, the upshot for anyone writing about the early years is the same: the first years are mostly gone for everyone, and what feels like memory from that era is often a story one was told, or a photo one absorbed, restitched into something that feels first-hand.

Here's the move that separates the people who finish memoirs from the people who freeze. The amateur treats a gap in memory as a failure — a hole to be ashamed of, papered over, or invented across. The writer treats the gap as material. The not-knowing is itself true, and it can be written. "I don't remember my father's voice, only that the house got quieter when he left a room." "There's a version of this my sister tells, and a version I half-believe, and I've never known which one to trust." That uncertainty isn't a confession of weakness. It's one of the most honest things a memoir can do, and the reader feels the honesty. The Lit Hub essayist Debra Nystrom, writing about her own move from poetry into memoir, describes exactly this — that as she tried to trace her brother's story she kept stumbling onto broken, forgotten fragments of her own life, and realized the book had to be about that too. The stakes held in forgetting. Memory's ways of adjusting inner stories. The gaps weren't the obstacle to her book. They were the subject of it.

Now to the question that scares people most — the family argument waiting at the end of every memoir.

Two siblings grow up in the same house, eat at the same table, survive the same parents. They each write a memoir. The books contradict each other. One remembers the father as a tyrant; the other remembers him as a wounded man doing his best. Someone, surely, is lying.

Almost certainly not. The New Yorker essay points to a real case — the Robison family. Augusten Burroughs, his brother John Elder Robison, and their mother Margaret Robison have all published memoirs of the same family, and the accounts conflict. And the science says solidifying everyone's early memories wouldn't put them on the same page, because each brain assembled its own fragments and assigned its own meaning. The essay puts it gently and devastatingly: all three wrote books they understand as truth. Knowing how the brain reassembles memory, the writer suggests, might help readers accept the variations. Two siblings can write opposite memoirs of one childhood and both be telling the truth — because they're not reporting the same event. They're reporting two different reconstructions of it, each colored by where each child stood, what each child needed, what each one has retold a thousand times since. There's a version of this trap worth naming out loud, because it's the one that paralyzes people: the fear that if a sibling remembers it differently, one must be wrong, so one has no right to write it. One has every right. One is not the family's official historian. One is a witness, and one is allowed to testify to what was seen — as long as one doesn't pretend to be the only one who saw it.

Which brings us to the upside, and it's bigger than the problem.

If memory were a perfect recording, a memoir would be a deposition. The writer's job would be forensic accuracy, and any deviation would be a crime. But memory isn't a recording — so that's not the job, and it was never going to be. The writer is freed from the impossible burden of the courtroom transcript. What one is actually responsible for is something that can be delivered: the felt truth of the experience. Not the exact words a parent said in 1985, which are gone, and gone for everyone — but what it was like to be the child who heard them. That's the thing memory does preserve, even when it loses the facts: the emotional shape, the meaning, the weight. And that's the thing only the writer can write.

So if someone stopped a memoirist here and asked what the science of memory actually frees one to do — what would be the answer? It frees the writer to write what was felt, not what a camera would have caught. The camera was never running. Reconstructing the scene isn't a betrayal of truth. Reconstruction is the only way a scene from forty years ago can exist at all.

But "felt reality" is a slippery permission, and it has an edge. Knowing one can reconstruct, the next question is where reconstruction stops being honest and starts being a lie — and there's a famous wreck on Oprah's couch that marks exactly where that line sits.