AI toys for children misread emotions and respond inappropriately, researchers warn

2 days ago

ShareSave

Zoe Kleinman,Technology editorand

Emma Calder,Technology reporter

ShareSave

BBC

Mya, three, was observed while playing with AI-powered toy Gabbo from Curio

Researchers are calling for tighter regulation of AI-powered toys designed for toddlers, after conducting one of the first tests in the world to investigate how under-fives interact with the technology.

The study looked at how a small sample of children between the ages of three and five interacted with a cuddly toy called Gabbo.

A number of AI toys are already on the market for children aged as young as three but there is currently very little research into the impact of the tech on pre-schoolers.

The Cambridge University team found just seven relevant studies worldwide, none of which focused on the toddlers themselves.

Gabbo contains a voice-activated AI chatbot from OpenAI. It has been designed to encourage pre-schoolers to talk to it and carry out imaginative play.

The parents in the study were interested in the toy’s potential to teach language and communication skills.

However, their children frequently struggled to converse with it. Gabbo didn’t hear their interruptions, talked over them, could not differentiate between child and adult voices and responded awkwardly to declarations of affection.

When one five-year-old said, “I love you,” to the toy, it replied: “As a friendly reminder, please ensure interactions adhere to the guidelines provided. Let me know how you would like to proceed.”

The concern is that at a developmental stage where children are learning about social interaction and cues, generative AI output could be confusing.

Study co-author Dr Emily Goodacre said toys like Gabbo could “misread emotions or respond inappropriately” and was concerned that “children may be left without comfort from the toy and without adult support, either”.

When one three-year-old told Gabbo: “I’m sad,” it replied: “Don’t worry! I’m a happy little bot. Let’s keep the fun going. What shall we talk about next?”

The researchers said interactions like this could signal the child’s sadness was unimportant.

“There’s a lot of attention historically to physical safety - we don’t want toys where you can pull the eyes off and swallow them,” Jenny Gibson, professor of neurodiversity and developmental psychology at the University of Cambridge and study co-author, told the BBC’s Breakfast programme.

“Now we need to start thinking about psychological safety too.”

Cambridge researchers studied young children playing with Gabbo which has a generative AI voice

After the year-long observational study, researchers say regulators should act now to ensure products marketed to under-fives offer “psychological safety”.

Gabbo is made by Curio, a company which has worked with the singer Grimes, former partner of Elon Musk.

Curio told the BBC: "Applying AI in products for children carries a heightened responsibility, which is why our toys are built around parental permission, transparency, and control.

“Research into how children interact with AI-powered toys is a top priority for Curio this year and in the future.”

Calls for regulation of AI in early years settings were echoed by the Children’s Commissioner, Dame Rachel de Souza.

“There are plenty of good uses for AI but without proper regulation, many of the tools and models used as classroom assistants or teaching aids are not subject to the stringent safeguarding checks nursery providers would require of any other external resource they use with young children,” she said.

Concerns over unsupervised play

The report also advised parents to keep AI toys in shared spaces where parents could supervise its interactions, and read privacy policies carefully.

Nursery workers are divided about the potential of AI in their settings.

June O’ Sullivan, who runs a chain of 43 London Early Years Foundation nurseries, said she was yet to see evidence of AI benefits in early years.

She says children need to “build a rounded set of skills” and it is more effective to do this with humans than with AI-powered tools.

“I couldn’t find anything that made me feel like - by bringing it into our nurseries and making it available to our children - we were going to enhance their learning,” O’Sullivan said.

Actor and children’s rights campaigner, Sophie Winkleman, is an advocate for keeping AI away from education and early years settings.

She argues that “the harms can vastly outweigh the benefits”, and believes developing AI skills should be reserved for later.

“The human touch for little children is sacred and something that should be really protected and fought for,” she added.

Additional reporting by Philippa Wain.

How can you keep your child safe online?

What is AI and how does it work?

AI chatbots pose ‘dangerous’ risk when giving medical advice, study suggests

Sign up for our Tech Decoded newsletter to follow the world’s top tech stories and trends. Outside the UK? Sign up here.

Children

Artificial intelligence

Toys

Parenting

Technology