Can computers recognize beauty in photographs?

What makes the beautiful photo? While there’s no such thing as a perfect photo, programmers are working on developing machines that can choose aesthetically pleasing photos.

This is the topic of the panel “The Photographer’s Lens: How Do We Define Beauty?” On the panel were Danielle Strle, the director of culture and trends at Tumblr; Miriam Redi, a research scientist at Yahoo; and Ramzi Rizk, the co-founder of EyeEm, a sort of stock photo sharing and selling platform.

Both Rizk and Redi develop the sort of programming designed to choose beautiful photos. Strle works with programmers at Tumblr to help users find photos they are interested in.

“(It is) a deep learning machine modeled after how the human brain works – it’s basically using many layers of processing,” Redi said of the programming.

But having a computer recognize the content of a photo is more difficult than many would think. There are an infinite number of objects in the world, Rizk said, but there’s not always a way to teach a computer to recognize a new object.

Strle used an example where she saw a photo of a man in his backyard, wearing a hoodie and holding a beer. The tags, she said, were hilariously inaccurate, including tags like #Molotovcocktail and #protest. It was obvious the machine had never been taught what partying looked like, she said.

Aside from discerning the content, the machine has done a fair job at recognizing appealing photos, Rizk says. At EyeEm, photographers worked alongside the model programming, Rizk said, and from their perspective, the machine chose beautiful photos correctly about 90 percent of the time.

Even so, Rizk talked about iconic photos, and if they would ever be found if photos were sifted by a program instead of a human.

“Historically speaking, we talk about iconic photos and the reason why they were iconic…all of these photos were also very rare,” he said. “So the notion of iconic moments (in the digital age) becomes less about the rarity of it, but the rarity of the special moment.”

All three panelists agreed that beauty is subjective, though Rizk said he believes there is an objective baseline of what makes a photo pleasing.

Strle said at Tumblr there’s often a scarcity of information published with the photos on the site, and therefore the photo might not show up on a search. And while she says she believes this technology can be useful in recognizing the contents of the photo for the purposes of a search engine, it cannot replace human judgement.

“I just worry about a world where we trust computer vision to show us the best of the best stuff, and what’s going to get lost…if there’s not human elements occurring alongside,” she said.

The panelists talked about the kind of photos that they find appealing as well, usually ones they’ve never seen anything like before. It’s the kind of judgement a computer may not be able to fully make.

They were also concerned that over time, the computer would narrow the types of photos it finds appealing too narrowly.

“If you do that kind of same process continually, do you reach a point of ultimate basicness?” Strle said.

Other than curating content for sites, businesses, interests, etc., this technology may have the ability to teach photography.

“One of (my favorite uses) is an app that transfers the knowledge of the experts to the less expert,” Redi said. “We can code it to a phone app for amateur photographers. Some people would like to learn how to photograph, but don’t have time, money for classes.”

It’s a difficult endeavor to create this technology, Rizk says, but worth it to be able to share beautiful work to those who want to see it.

“To really understand the inner workings of beauty and make sure that all content that needs to be seen to all people who need to see it, I believe that’s our mission,” he said.

 

See our interview with Danielle Strle about the tech she encounters at Tumblr:

Leave a Reply