Image Hunt: A Game With A Purpose

General description: A web game for hunting image!

Implementation period: March 2013

Technology: LAMP

Event: Morbify Hackathon.

Contributors: Kazem Jahanbakhsh, Ajay Sridharan, Amir Moghaddam, Priyanka Gupta & AmirHossein Hajizadeh

Technical details:
I think many of us had the experienec that we searched for an image in google image search but we couldn't find the image that we were looking for easily! This is mostly due to the hardness of mapping images to texts. For instance, I picked a random asian girl photo from web (this image was returned by google for a query). The image is shown below:

asian girl image

By looking at the image, I tried to find the best set of keywords which describe the above image as accurate as possible. Using those keywords, I wanted to see if Google find the image for me when I search for those keywords. I came up with the following query:

"an asian girl with green shirt and white pants"

After trying google search image with above keywords, I got the following images back:

google image search results

Two main observations from above experiment:

  1. We were not able to find asian girl image in Google search results (recall=0).
  2. The output results from Google were not very well related to our query (low precision).
Above observation shows that image search is a hard problem. There is an interesting area in gamification called "game with a purpose". One of the pioneer in this area is Luis von Ahn (Luis homepage). He developed several games with purpose in order to solve hard AI problems by making use of human brain computation. One of his games was used by Google for image annotation (Google Image Labeler).

We decided to design and implement a GWAP in order to solve a hard problem. We wanted our game to have the following "properties": We found a game developed by Microsoft called Page Hunt. The main idea behind Page Hunt was to show a web page to a player and ask her to type a set of keywords in order to find that page in Bing search engine. If a player can successfully find a set of keywords such that the page appears between the first 5 results, she will go to the next page and continue until the game is over (after 3 mins). The purpose of Page Hunt is to understand people search behavior, tag web pages, and also identify Bing's ranking issues.

We adopted the same idea as Page Hunt for images by our designed game: "Image Hunt". In Image Hunt, we show one image at a time to player and ask her to type a set of keywords in order to find that image in google image search. If she's successful, she'll get score and we show the next image and continue until game is over. Our game has several purposes: it can be used for advertisement purpose, it tags images, identifies ranking issues for searching images, identifies searchable/non-searchables images. Note that Image Hunt game is a single player game.

Rules of the game:
  1. Player must find the best keyword/keywords which would bring an image to the list of the top 5 results in Google image search.
  2. The higher is the rank of the image within 5 first results, the more points the player gets.
  3. Achieving this without frequent queries earns a bonus.
  4. The game lasts for x minutes.
Players need to register to play the game. The game has leaderboard. Thus, players can compete with one another.

We came up with a simple FSM to describe the game as shown below:

image hunt fsm

Having the FSM of Image Hunt, we came up with a list of methods that we needed to implement:
  1. load_game(username): calls load_leaderboard(username), create_a_session(username), start_timer(), load_game_frame() to start the game
  2. load_leaderboard(): this method collects all records from leader_board table and returns the leaderboard as an associative array (so the caller can use the array to display the leaderboard to the player).
  3. update_leaderboard(username, score): this method takes username and score of the player at the end of the game and updates the corresponding entry in leader_board table. This method will be called when the game is over.
  4. start_timer(): start the timer
  5. create_a_session(username): create a game session for this player
  6. load_game_frame(): load all necessary data from model (image collection) and submit them to view layer for loading the game page
  7. verify_keywords(keywords): upon keyword submission by user hits google_api_call(keywords)
  8. google_api_call(keywords): send keywords to google search api and gets the first 5 images and then call verify_images()
  9. verify_images(): run the algorithm to compute the similarity of the image with the firs 5 images. if image was found, we call update_score(), update_leaderboard(), log_user_activity(), check_game_over() else call log_user_activity(), check_game_over(), display_try_again(), and load_next_image() if game is not over yet
  10. update_score(): compute the current score of the player
  11. log_user_activity(username, image_id, image_rank, keywords): This method takes username, image_id of the image that we are currently displaying to player, the rank of the image in the returned search results, and the keywords which were typed by user for the given image and store them into user_activity table. This method will be called in the end of every round when the user sends his keywords to the server for evaluation. So, we will record all activity of the user during the game.
  12. check_game_over(): check if the game is over, if not call load_next_image(), else call clean_up()
  13. load_next_image(): load the next image and show it to user
  14. display_try_again(): display try new keywords to player
  15. clean_up(): do clean up for user and show the final page

You can download the source code for this project from its github repo: ImageHunt.


You should follow Follow @kjahanbakhsh me on Twitter.