This morning, shark mega-enthusiast & PhD student David Shiffman (@WhySharksMatter) tweeted
I just uploaded a pic of me with a #Shark to Google Image reverse search and it IDENTIFIED THE SPECIES OF SHARK pic.twitter.com/fTpmHP7IVG
— David Shiffman (@WhySharksMatter) August 23, 2013
Here’s the screen cap image a little larger if you can’t make it out (click to embiggen further):
Is reverse Google Image search able to identify sharks to species? Yes, David included the search term “lemon shark” (David just let me know that Google included the text search terms itself… my mind is blown), but the fact that Google returned “Best guess for this image: lemon shark” might imply that they’re playing around with visual identification services, not just photo comparison. Considering how well reverse image search does at aggregating similar images, how many shark images are online & indexed by Google, and that many of those images are probably tagged with a species name nearby or in the metadata, I think the concept is entirely plausible.
Seeing how insect ID is kind of important to me, I tried it with a few of my insect photos, and got nothing. I even tried improving the odds by using search terms like David HAL 9000 Google did, and this is all I got:
I was beginning to get discouraged, but Marianne Alleyne (@Cotesia1) made a good point: perhaps it was the fact that David was sitting on the shark that mattered!
So, I reverse Google Image searched this photo
Apparently Google thinks this fly wrangler looks like a bride. Perhaps their search algorithm could use a little more work after all.
Clearly Google loves sharks and hates flies (and passenger pigeons). Not cool Google, not cool.
I guess we’ll just have to stick to other web-based tools for identifying flies for a little while longer. Darn.
It could just be pixel matching. The photo with the shark probably looks much like other pictures of blokes with sharks. We had to have Streetview remove our section of the street from public view as it pixel matches our front door with pictures on our blog. That wasn’t too much of a problem, but it wasn’t too big a step to go from there to photos of the inside of our house. The ease of linking all the information up was a bit disturbing.
What might be more interesting is can Google Image Search distinguish mimetic species from the species that they are trying to mimic…? Maybe some standard pinned specimen photos might yield a more standardised search (the algorithm seems to rely quite a lot on the background)? I just tried some hoverfly pictures with the search term “insect”, and it threw a lot of Hymenoptera back (along with lots of random stuff).
That’s a really interesting idea Chris. I’ll see if I can find some time to try a couple pinned specimens from our collection, both those that are mimics/models, and those that should be easily recognized. Will report back if I find anything interesting!
Google reverse image search works by making a mathematical model of the selected image and by reading the text associated with it (like:image name, attributes, etc.) to determine what the image is. Then Google compares that to other images in it’s database to find the best match.
I have also tried to other reverse image search engines (like: tineye) and found that they work pretty bad when compared to google.
Never make that face and pose again, Morgan. I have been scarred for life.
*enters cold shower with clothes on, sits down, and cries himself to sleep*