For this assignment, you will become a work requester on a platform like Amazon Mechanical Turk. The goal is to hire people to generate text on 50 images from at least 3 independent people on each image. You will create a Web document that illustrates the images and the corresponding text and reflect on this experience by answering a few short open-ended questions.
You will turn in your assignment using this form.
Become a Requester
Visit the following page to register as a requester: requester.mturk.com You may also have to register on the requester sandbox, at requestersandbox.mturk.com. We strongly recommend posting and testing your HITs (tasks) on the Mechanical Turk sandbox before posting on the real site and potentially using real money.
Go to "My Account" to enter money into your account. I recommend you deposit at least $20 into your account. You will need money for HITs, as well as for MTurk fees.
Note: If you earned money on HW1, you will not see it reflected here. To cash out on HW1 earnings, transfer your balance to your Amazon Payments account and then you can use it on Amazon.com or transfer to another bank account.
If you run into barriers with using Amazon Mechanical Turk for this assignment, you can use one of these platforms as a substitute: CrowdFlower or Microworkers.
Goal: Label 50 images
To get familiar with hiring crowd workers, we will create a classic HIT where workers label images. You have creative license over which images and what types of labels you want to create. For example, you might gather 50 personal images from a recent trip to the zoo and have people provide labels for the animals. Instead of labels, you could have people write a comedic tweet that could go along with each image. You might get 50 photos from a research project and have people generate hypotheses. It's up to you.
The goal is to general 3 independent pieces of text for each image. This can come from 3 different workers who do the task 50 times (one for each image), or from 150 unique workers who each do task.
Create a task
There are several strategies for accomplishing work on Mechanical Turk. I will review the two most common approaches: using MTurk templates for tasks and redirecting workers to an external site. For savvy programmers, you might consider experimenting with the MTurk API. For a primer, see project 2 here.
External survey:
This approach will be most familiar to people who have conducted online surveys. Use your favorite survey tool (e.g., Google forms, SurveyMonkey, Qualtrics) to create the specific task you want people to do. Make sure to ask for their MTurk ID in your survey, so that you only pay the people who fill out the survey. You might also want to collect a few demographics and other information from your workers, beyond the image text they provide. Finally, at the very end of the survey, show the worker a unique code that you could only get by completing the survey.
Once you have created and tested your survey, you can post your survey link on Mechanical Turk. In MTurk, go to "Create" and then "New Project". The very first template is one for Survey Link, which is exactly what you need. You can modify this template to accomplish what you want. When you're ready to launch, go to back to the Create menu and click on Publish Batch for the project you just created.
Results will be available in the survey tool. Make sure to pay your workers after you verify their data.
MTurk image-tagging template:
In this approach, you will work primarily with the MTurk templates to accomplish the HITs. Start in MTurk and go to "Create" and then "New Project". The third template is "Tagging of an Image" which you can modify to suit your needs. In this example, each HIT generates four labels for one image. You would modify this by reducing it to one text entry field.
This template uses a variable called "image_url" to show images from a .csv file. Basically you post your 50 images in a public dropbox folder and then load your .csv file with that list of URLs. When you go to Publish Batch, MTurk will prompt you to upload a .csv file (should be a single column with image_url in row 1 and URLs for your images in rows 2-51).
Results can be downloaded from MTurk directly. Make sure to pay your workers after you verify their data.
Turn in
For this assignment, create a page on the web (at least a Google doc) that shows your 50 images and the 3 independent text labels generated for each. At the very top of this document, put a screenshot of your task and answer the following questions:
Make sure your online document is viewable by everyone in the class and then use this form to provide a link to your page.