Focus | Getting to grips with artificial intelligence

The Madonna della Rosa, a painting of the Virgin Mary thought to be by Raphael and now hanging in the Prado in Madrid, was recently revealed to have an unexpected extra dimension: an image of Joseph that the Renaissance master did not create himself.

It came to light thanks to analysis with facial recognition technology trained in Raphael’s style by experts at the University of Bradford.

This is just one eye-catching example of the role that artificial intelligence (AI) might play in art history.

While AI has been used behind the scenes in museums for some years, the revolution is gathering momentum. This is driven partly by generative AI and online tools such as OpenAI’s ChatGPT, which are igniting much wider interest in how the technology could improve efficiency and how it might open the door to new ways of working.

What do we mean by AI?

AI is an umbrella term for a range of sophisticated technologies that can perform tasks that traditionally require human intelligence, analytical skills or judgement, such as understanding speech or recognising images.

“Most AI is a form of machine learning – statistical models, usually created from data scraped from the internet – that tries to generate text, images, video or audio based on the ‘training data’ it has access to,” says Mia Ridge, a digital curator with the Western Heritage Collections digital research team at the British Library.

“It’s not ‘intelligent’ and doesn’t have knowledge or semantic understanding of what the words and images mean. AI is a great marketing term and a not-that-accurate shorthand name for machine learning.”

Generative AI is the form that has captured public attention since ChatGPT emerged in late 2022. Generative AI models, trained on data from across the internet, respond to prompts and can generate detailed answers in human-like terms.

They can replicate a specific tone of voice and can also adapt to format (for example, bullet-point summaries, social media posts or news item). Most significantly, they can do it in a matter of seconds.

ChatGPT was followed by other software including Anthropic’s Claude, Google’s Gemini (formerly Bard) and Microsoft’s Co-pilot.

Models such as Midjourney and DALL-E can produce visual content to a written text prompt, drawing on existing images. There are also tools that can produce audio and video to written instructions.

The latest version of Adobe Premiere, one of the most widely used video-editing applications, now includes a facility for writing a text instruction directing the AI to edit raw footage into a package: the human is still the director-editor, but the AI does the fiddly cutting and assembling.

A comprehensive report for the National Lottery Heritage Fund last year entitled Artificial Intelligence: A Digital Heritage Leadership Briefing explores how AI is already being used.

It identifies three key areas:

1. Heritage and collections management use and research

Heritage and collections management use and research This is the area in which AI is most deployed. It is helping to expedite some of the more labour-intensive, time-consuming tasks around the labelling of objects, indexing and generally making collections more searchable.

At the Science Museum, machine learning is used internally to help provide more relevant results in searches via the collections website, and to automatically identify elements of images that may be useful to users when returning results.

“We can use it to generate alternative spellings and synonyms – as well as related people, companies and subject areas – and add those entities to the underlying index to help improve discoverability,” says Jamie Unwin, the museum’s head of digital technology and a member of the Museums + AI Network, a 12-month research project that took place in 2019-20 and was funded by the Arts and Humanities Research Council.

The Science Museum also uses a technique called “entity recognition” to scan its internal catalogue records for identifiable people and places, which it then links to third-party sources such as Wikidata, making the content easier to find and enabling institutions to share data and perform other functions more efficiently.

“We’ve got over half a million objects in our database,” says Unwin. “Going through every one and trying to link everything to Wikidata or other institutions’ systems by hand would be impossible.”

Although machine learning works to a high degree of accuracy and can improve how searches work, Unwin says the museum does not “present as fact” anything that is produced with AI.

At the National Museum of the Royal Navy in Portsmouth, staff have taken textual collection descriptions written by curators and used AI technologies to generate the keywords.

AI is able to generate these 20 times faster than a human, according to Amy Adams, the museum’s collections information and access manager. “But the accuracies around some of them are still debatable,” she says.

2. Visitor experience

AI applications are already embedded in some museum functions. These include chatbots to interact with visitors online, automate translations for people for whom English is not a first language, and automated transcription of events such as conferences.

The Anne Frank House in Amsterdam incorporated a chatbot into Facebook Messenger to answer people’s questions about Anne Frank’s story, while the National Library of Scotland uses tools such as the transcription app Otter.ai to transcribe live events and write captions for videos, which are proof read by staff before publication.

3. Business and management

There is potential for AI to support functions such as forecasting ticket sales
or visitor attendance. The National Gallery, London, developed AI tools to predict how popular temporary exhibitions would be, drawing on two decades of its visitor data, which helped staff to manage room capacity at events in real time.

Departments such as marketing and comms seem ideal candidates for the use of ChatGPT and other generative tools to undertake tasks such as sending out emails, exhibition flyers or social media posts, but evidence of its use – at least institutionally approved use – seems limited.

Other uses of AI

Even if one shies away from using the generative tools for original writing, there are plenty of straightforward tasks that it could be assigned, always with the caveat that a human needs to check the final output. Such tasks include:

• Writing report summaries– feed in the 20-page document and prompt the AI to summarise it in 500 words.
• Email responses to common queries.
• Session planning outlines for schools and other workshops.
• One of the more creative applications popular in other sectors is brainstorming ideas. Ask ChatGPT for 10 ways to mark Black History Month as a museum and it might generate seven you’re aware of, a couple you had overlooked, and one you’d never even considered.

The importance of data

A cornerstone of the efficient use of AI tools is effective data. For collections purposes, for example, effective linkages in search depend on accurate data – and this can be a challenge.

“Whether you’re using tools that are tailored to you or those that are bought off the shelf, you need to have digitised data,” says Mathilde Pavis, author of the Heritage Fund report and an expert in intellectual property law.

“It needs to be properly labelled and structured, even before you start looking at AI tools. This is the bread and butter of digitisation – digital data management and rights management. Without that, it will be difficult to get the most out of your data and what you do with AI.”

Tools such as ChatGPT are by no means perfect and may “hallucinate” and produce an answer or text that sounds plausible but is factually flawed, with invented quotes and references. Even when generally accurate, it can be superficial and generic.

More insidious, still, is the risk of bias: a generative text tool is likely to have trained on biased text, meaning that bias – whether racial, colonial, historical or gender – will form part of the result.

It means human editing is vital; not just proof reading for spelling and grammar (which AI tools are actually pretty good at) but detailed checking of content, links and sources.

But some argue that the use of generative AI tools to perform even straightforward writing tasks will always be too great a risk for museums.

Writing object labels, for example, might be seen as a task that, in future, could be outsourced to an AI tool such as ChatGPT. But the result would inevitably lack depth, argues Rachel Coldicutt, founder of research consultancy Careful Industries and a member of the Museums + AI network.

“A subject matter expert in a museum will be bringing lots of separate contexts to include in the label – knowledge that doesn’t sit in an online database but is about how to engage specific audiences and what’s culturally relatable to visitors,” she says.

The Science Museum’s Unwin says: “ChatGPT works by guessing what the most likely next word or sentence should be in a text, based on what others have written.

"A system that just says the obvious probably isn’t what we should be doing for a museum website. Where we’ve got a vast wealth of information, we should be the primary sources for that information.”

Coldicutt also points to the economic impact on some parts of the workforce in that generative AI potentially poses a threat to jobs on the lower rungs of a career ladder, such as copywriting, which people do as freelancers or on an otherwise precarious basis to allow them to get a foothold in the creative industries.

Skills

AI is bringing a wave of disruption across the sector – as it is across society. The good news is you don’t need great technological aptitude to grasp at least some of the opportunities.

The British Library’s Ridge suggests people play around with AI to understand what might be coming.

“AI literacy is an important part of good governance,” she says. “People need a solid understanding of where biases are likely to appear, how to review and contest decisions made by algorithms and where sharing data might have privacy or legal implications, so that they can make good decisions about the products they buy or implement. It also helps people plan so that AI tools enhance jobs, rather than attempting to replace them.”

Pavis, an expert in intellectual property law, pinpoints training for staff and volunteers in areas such as data management, rights and compliance, AI systems and risks in all departments of the organisation, as part of workforce development.

One fundamental is to agree on a common language.

“Everyone needs to know that they are talking about the same thing when they use the term ‘AI’ in their institution – what does it mean, and when we are talking about ‘data’, do we all mean the same thing?” asks Pavis.

The future

The potential for even greater transformation is huge, says Dave Thomas, chief technology officer with Axiell, which provides specialised content management systems to museums worldwide.

The key to success will be greater interoperability, fuelled by AI, which will potentially allow museums to collaborate seamlessly for collections management and for public benefit.

The problem is people don’t share knowledge effectively and swathes of collections are yet to be digitised, says Thomas.

“My concern is around how we connect collections, because digital audiences don’t see institutional silos,” he says. “If I’m a digital visitor interested in the second world war, I’m not just interested in one collection in one place.”

That could, for example, mean a visitor starting with the Imperial War Museum online then following links to, perhaps, Tate Britain’s war artists and Cumbria’s Museum of Military Life.

“Museums have to be much more digitally inspirational when it comes to things like education, which they can’t do very well at the moment unless they work together,” says Thomas.

Coldicutt foresees museums following in the footsteps of some retailers and using AI-led tech to streamline areas such as customer service.

“It may be that we will get to a place where seeing an actual person front of house becomes unusual,” she says.