The urge to find out what will happen or had happened keeps us turning the pages, and, as I said, we sometimes form fairly definite expectations. That doesn’t mean we look to have our expectations confirmed in the end. There’s a different kind of satisfaction when the ultimate event or revelation completely overturns expectations. This is the shock of the narrative twist.
We’re very fond of having the rug pulled out from under our feet nowadays. Genres that thrive on twists—crime, spy, legal thrillers, conspiracy/political thrillers—have all become very popular over the last half-century. By the same token, we’re also more prepared for twists, and not so easily taken by surprise.
Guessing the twists can be a sort of game between author and reader. That is, the reader forms expectations on the basis of previous twists in previous books and movies, rather than on the basis of the actual characters and situations. You pick the least likely person as the killer not because of any reasons out in the fictional world, but because you know this is the way the detective genre usually works.
I’m not sure this is a good thing. As an author, I want to immerse the reader in the characters’ experience, not have the reader sitting off to the side, calculating the odds on a different level. Still, you have to reckon with this as something most readers will bring to their reading.
How much does it matter to take the reader by surprise? In some genres, like romance, the final conclusion is never in doubt. Instead, the interest likes in how things unfold along the way.
Fantasy, traditionally, has never been the most obvious place to find twists. Fantasy has tended to be a what-next kind of genre, whereas the most surprising twists tend to arise over what-has-happened. But there are plenty of exceptions … and who knows? maybe the Harry Potter books have changed the template forever.