Project

Students will work on quarter-long research project that proposes a contribution to the fields of collective intelligence, human computation and crowdsourcing, or uses crowdsourcing in a creative way to make a contribution to another research domain.  Successful projects raise interesting and open research question(s) and create a prototype system or conduct a preliminary study that demonstrates a potential contribution to the academic literature.  This project will be completed solo or in groups of two.

Project goals:

Project timeline:

We will have interim project deadlines throughout the quarter:

Project details:

Weeks 1-3: Ideas and teams 

Everyone has ideas! Let's create a collective repository of possible research projects (link will be shared on first day of class).  Feel free to list anything related to the course topic.  For each idea, include a brief descriptive title, a paragraph that describes the potential contribution, and your name so that peers may reach out to you about collaborations through our Slack channel.  We encourage everyone to contribute ideas asynchronously outside of class. Start communicating with each other about potential collaborations for the final project. 

This project will be completed in groups of two. If we have an odd number of students, there can be one team of three. No teams with a single member. When discussing a potential partnership, you should strive to create a team with diverse backgrounds and skills (e.g., programming, design, writing, study design) but overlapping interests, availability, and motivation levels. It's important to communicate and follow through on commitments. Keep a clear, consistent line of communication open with your teammate.

You'll likely iterate on your ideas throughout the quarter -- try to avoid fixating on one idea early on. Collaboration with people not enrolled in the class is allowed as long as their contribution is clearly identified. A higher level of quality will be expected with additional collaborators.

Add your ideas to the Ideation tab in the Course Dashboard.  When you form a team, add your team to the ProjectTeams tab. 

Weeks 4-5: Pitches

During class in Week 4 or Week 5, teams will present a short “pitch” that describes 1-3 of your favorite ideas. Your talk and discussion should be relatively short (no more than 10 minutes per team) and should cover these points:



After your pitch, the instructor and peers will provide comments, ideas, and feedback. If your team presents more than one idea, we can all vote on which of your 2-3 ideas has the most promise.  Make sure to get the instructor's approval before moving on to the prototyping stage. 

When your "Pitch" slides are ready, add a link to the ProjectTeams tab (by W4 or W5 class). 

Weeks 6-8: Prototypes

After getting feedback and approval on your project pitch, spend weeks 6-8 creating an initial prototype of your proposed system or creating a draft of the materials you will need to conduct a study.  Your team's prototype should be a concrete representation of your project that people can actually try out. It could be a working prototype of some aspect of an envisioned system or it could study materials for an intervention you want to test empirically.  Whether it's a system or intervention, every team should develop an initial study protocol (e.g., desired participants, procedures, measures, etc.). 

During class in week 8, each team will demonstrate their "prototype" by either demoing the prototype, or even better, by conducting a short trial session with your peers as pilot participants. Use this opportunity to gather preliminary data on how people interact with the prototype (ie., behaviors) as well as how they feel about it (ie., impressions).  Teams should also create a short survey to collect comments and feedback directly after people try it.  We will end the session with a discussion about how to improve the prototype and study protocol. 

Build on your earlier slide deck. Include details about the prototype and instructions for how to interact with the prototype.  Additional written instructions will help insure that everything is consistent across participants. The slides should include links to the system or study prototype, as well as to any forms you create to collect feedback. 

When your "Prototype" slides are ready, add a link to the ProjectTeams tab (by W8 class). 

Weeks 9-11:  Data Collection, Presentation, and Report

Data collection. Iterate on your prototype system or study.  Then try to collect data from actual participants. You may recruit people by any means. This includes using UCSD's SONA system, posting a link to your social media or to online forums related to your topic, or by paying people from a crowdsourcing marketplace. From Week 8 to Week 10, collect data from at least 20 participants and conduct some preliminary analysis.  

Presentation. Each team will present during week 10 (no paper discussions that week) for up to 15 minutes (~10 min for talk and 5 min for Q&A).  Please be sure to include all teammates' names on the first and last slide. The goal should be to discuss all aspects of your research process and results, as if you were giving a conference research talk or a departmental research talk. Cover these points:

When your "Final" slides are ready, add a link to the ProjectTeams tab (by W10 class). 

Report. For the final team report, your goal is to write a 4-6 page paper that would serve as a good first draft of an actual submission to CHI, HCOMP, CI or any academic conference.  Please use the ACM SIGCHI paper format. All of the text and figures/tables should be included within the 6-page limit, while any number of references/citations can go beyond the six-page limit. Appendices are acceptable and optional (they don't count towards the page limit). Include details an interested reader might want to see (e.g., survey questions, interview questions, ChatGPT prompt engineering, etc.).

The research paper should include the motivation for the research, background literature, method, results, discussion, and conclusion. The goal is to draft a paper worthy of publication in a top-tier research venue.

Some advise: 

When your Final Report is ready, add a link to the ProjectTeams tab (by Friday of W11). 

IRB Review:

In general, when students do research as part of a class project, IRB approval is not required. You can pilot your study and collect data; it counts as your educational experience. However, this means that the data you collect may not be published in an academic research venue. This is not something you can change later.  If you anticipate that you might want to publish your data later, we encourage you to apply for an exempt IRB protocol BEFORE you start collecting data. If you would like to apply for IRB approval do the following:

Even for class projects that are not subject to IRB jurisdiction, you still have a responsibility to protect the welfare of those participating in your class project. That means, creating a consent form that spells out what you are asking from each participant in terms of information, time, and effort. Specify what data will be collected from each participant and how you will protect this data. 

Expectations and Grading:

The expectation is that every student will present one paper per week, participate actively in discussions around all papers, and make significant contributions to a team research project as described above.  If you are fully engaged, you can expect to get an A. Prof. Dow will not provide interim grades on any deliverables, but will happily provide feedback throughout the quarter.  Prof. Dow will also let you know if he thinks your research paper has the potential to make a contribution to the research community.  After the class ends, you will have a good start on a full paper submission with detailed feedback and you will have collected at least some of your data.  It's your team's choice about whether to carry forward to submit your work to an academic conference.