The Metropolitan Museum of Art in New York is one of the world's largest art museums, housing over 1.5 million works of art with over seven million physical visitors each year. In order to expand this reach to the internet-connected audience of 3.9 billion people worldwide, the museum must take on the herculean task of digitizing, classifying and tagging each work of art in a scalable way.

In 2017, the Met launched its Open Access platform, making all images and data relating to public-domain artworks in its vast collection available to everyone online to make the museum's collection one of the most accessible, discoverable and useful on the internet.

Now, The Met is collaborating with Microsoft to take this initiative to the next level, utilising artificial intelligence to help audiences better discover these artworks and develop meaningful connections with them.

The result is a proof of concept called, Art Explorer, which uses Microsoft's Cognitive Search to examine each artwork and automatically generate all the information needed to tag and classify the piece in a fraction of the time. Essentially, the project uses artificial intelligence to make its collection accessible to the 3.9 billion internet-connected people worldwide.

The classifying and tagging process is complex and labour-intensive because inherently, art is highly unstructured. Unlike with a document, there's no easy way to lift information. Instead, the visual elements need to be perceived, from the media and technique to the choice of colour palette and subject.

There's also a vivid story behind each piece: the artist's history, influences and style. In the past, gathering all this information had to be done manually for each piece ‒ a painstaking process, particularly when it involves millions of works of art.

"For our Open Access collection, we hired a team of people to examine each image and tag what they saw," says Jennie Choi, general manager of Collection Information at The Met. "However, this is very labour-intensive and requires careful inspection of each piece."

Art Explorer uses Microsoft's Cognitive Search to examine each artwork and automatically generate all the information needed to tag and classify the piece in a fraction of the time. In fact, the project was able to process hundreds of thousands of artworks this way in a single night!

But there's much more to the process than speed. When a digital image is loaded into Art Explorer, Cognitive Search also surfaces what objects are depicted in the piece, what other artworks in the collection are visually similar and what relevant information such as geography and artist's history it should pull in from the web.

This data is then organised into a searchable index that unlocks insights, uncovers relationships between pieces in the collection and grows the knowledge base around each piece online. As people navigate the Open Access collection, they can search for oil paintings or line drawings or all works of art depicting certain animals. In other words, they can find what interests or inspires them in ways that were never possible before.

Art Explorer is just one of the many ways The Met is experimenting with artificial intelligence. Earlier this year, they collaborated with Microsoft and MIT to host a hackathon encouraging developers to use AI to rethink the museum experience.

One such project is My Life, My Met which turns an Instagram feed into a work of art. Still in development, the concept is to have AI analyse Instagram posts and substitute the closest matching artworks from The Met's Open Access collection. In development by Gen Studio, the project uses AI to navigate the 5,000+ years of human creation represented in The Met collection. Viewers are placed in map-like model that organises the artwork in exciting new ways, pointing them towards artwork based on style, materials, subject or other characteristics.

Another project is Tag, That's It! ‒ a crowdsourcing game that enlists the global Wiki community, partnering humans with machines to fine-tune the keywords tags for a work of art. It requires identifying which pieces contain various objects: children, chess, calligraphy, and of course, cats.

Likewise, Storyteller uses AI to find pieces of art in The Met's collection to accompany any story. It uses voice recognition to follow a discussion and share artworks related to the stories being told. Afterwards, users can share the Storyteller thread of artworks on social media.

A very interesting project is Artwork of the Day, an app that helps individuals find a specific piece of art in the collection that will resonate with them on any given day. Each day, analysing data like location or news, the app offers an entry point into the collection based on world events and the user's current circumstances.

"Through AI, we can see things that we couldn't see before with the naked eye," says Maria Kessler, senior Programme manager for Digital Partnerships at The Met. "There are patterns ‒ there's information that's discoverable in the art, behind the art, and connecting the art from one piece to another."

This project may have big implications for museums overall. Instead of observing static works of art one-by-one, viewers will be able to dynamically curate their experience based on their unique interests. Understanding the innovative ways to harness AI, The Met is poised to create meaningful new ways people connect to art.