They Know Where You Live!

General description: In this project we implemented two crawlers for Flickr website. Our code is in Python and it runs in two levels.

Implementation period: Sep - Nov 2010.

flickr

Technical details:
This project includes the following two crawlers:

  1. Friendship graph crawler: this Python code runs a BFS on social graph of Flickr (the friendship graph) to extract the friendship connectivities among Flickr users.
  2. Photos crawler: This crawler uses the social graph collected by the first crawler. It examines every node in frienship graph and collects all possible information about users social profiles by using the Flickr API. It also crawls all photosets uploaded by each user and downloads different attributes of each photo. However, this code does not download the photos itself [1].

*** You can download the source code for photos crawler from its github repository: flickr-crawler.

Language: Python
No of lines: 143
Used technologies: Flickr API

In this research project my main goal was to test if I can estimate people's hometowns by using their uploaded geo-tagged photos in Flickr website. I used the Flickr API to crawl the geo-tagged photos posted by people. I implemented a clustering algorithm to group similar photos by using their geographical locations in order to estimate people hometown. You can find the detail of this work in our technical report listed below.

One example of running our hometown prediction algorithm by using people photos is shown below. The first figure shows the photo map of a user who travelled to different countries in North America, Europe, and East Asia where she has taken several photos. The second figure shows our clustering algorithm results. Our algorithm predicts the user's hometown somewhere close to Warsaw in Poland. This actually matches with where the user has listed as her hometown in Flickr website!


flickr user

prediction

Publication
K. Jahanbakhsh, V. King, G.C. Shoja, They Know Where You Live!, arxiv, 2010.

Press
This project has been covered by online media from users' privacy point of view:

1- MIT Technology Review - Algorithm Uses Photo Networks to Reveal Your Hometown
2- Digital Trends - Can posting pictures to Flickr reveal your hometown?
3- ACM TechNews - Algorithm Uses Photo Networks to Reveal Your Hometown
4- Hard Forum - Flickr Photos Reveal Your Hometown?
5- Science & Technology - Algorithm Uses Photo Networks To Reveal Your Hometown: They Know Where You Live! By Kazem Jahanbakhsh, Valerie King & Gholamali C. Shoja (from arXiv)
6- Technology Review - Heise Online (in Germany) - Data Mining in Picture Service
6- CiOL - Photo networks can reveal your hometown

Refs:
[1] Flickr API: http://www.flickr.com/services/api/



Tweet

You should follow Follow @kjahanbakhsh me on Twitter.