Olivia Colman holidays strangely in the gripping The Lost Daughter

Olivia Colman can currently be seen in Landscapers, the HBO/Sky mini-series that has at last taken the obvious step with her cheery, sweet-centred persona and cast her as a murderer. In the jagged new psychodrama The Lost Daughter, the tweak is less extreme. No one is killed — only affronted. Somehow the effect is more unsettling.

Her character is Leda Caruso, a professor of Italian literature, originally from Yorkshire, now on a Greek island holiday. She is wry and playful. She can also be bad-tempered, brusque and creepily secretive. Mixed messages of this kind can be tricky for movies, a form in which people tend to be one thing or the other. For women — especially mothers — that usually goes double.

To complicate things further, Leda is played by two actresses. In flashbacks, Jessie Buckley is her younger self, before the kids grew up and left. But to start there is only Colman. Arriving on the unnamed island, her manner speaks of someone often alone in their study. Small talk does not go well. “Children are a crushing responsibility,” she flatly tells a pregnant woman on her birthday, grimacing through the cake. Scenes often end with her hurrying off, mortified. Truly, wherever we go on holiday, we’re stuck there with ourselves.

A gaggle of supporting characters enters. Among them is a sun-worn American ex-pat, played by Ed Harris. He comes connected to the vacationing New York family with whom Leda ends up sharing the resort, a noisy Hydra of strutting dudes, wailing toddlers and loutish teens who may just be the sort of people who always sit next to you on the beach — or something darker.

The film is directed by Maggie Gyllenhaal, adapting a novel by Elena Ferrante as her debut feature. The literary simmer of the source material can feel noodling on screen. More often, it grips. In place of splashy twists, the story jitters and wrongfoots. There are flirtations and outbursts. On the beach, Nina (Dakota Johnson), a young woman married into the nightmare family, realises her infant daughter is missing. The sequence ends with Leda taking a small, strange action that ripples all the way to the credits, one even she would struggle to explain.

A clue lies in the past. Having nipped at her through the first act, Leda’s history now erupts on-screen in scenes of her previous life as a frayed young academic struggling with the children. Here, debut nerves be damned, Gyllenhaal pulls off a coup. Where Martin Scorsese’s The Irishman used digital tech to de-age Robert De Niro, The Lost Daughter sticks with the analogue: another actor. The mesh of Colman and Buckley as one character — without particular physical resemblance — is uncanny. Between them, the mystery of Leda snaps into place, as much as people ever do.

And if the film is too subtle for statements, it broaches a genuine taboo — the strict standards we apply to motherhood, rarely discussed in cinema. (Psycho doesn’t count.) Gyllenhaal often uses intense close-ups. At first the result can be jarring. Then you realise how it blurs the line between Colman and Buckley; and makes us focus, too, concentrated on features that for a second might be anyone. The push to look harder sums up this bold, sharp-cornered film — one as admirably un-eager to please as the woman it portrays.

★★★★☆

In UK and US cinemas from December 17 and on Netflix from December 31