KAIST Fall 2018

CS473: Introduction to Social Computing

Human-computer interaction no longer only concerns a single user in front of their computer. An increasing number of modern systems are inherently social, involving a large group of users to collaborate, discuss, ideate, solve problems, and make decisions together via social interaction. This course focuses on crowdsourcing and social computing, two of the most important concepts in the era of interaction at scale. This course will cover major design issues and computational techniques in building crowdsourcing and social computing systems.

This is a highly interactive class: You’ll be expected to actively participate in activities, projects, assignments, and discussions. There will be no lectures or exams. Major course activities include:

  • Reading Response: You'll read and discuss important papers and articles in the field. Each week, there will be 1-2 reading assignments, for which you'll write a short response to.
  • Topic Presentation: Once a semester, you'll lead the class by summarizing the readings and peer students' critiques, and spurring the in-class discussion. You’ll also get a chance to design your own activity that best captures the topic of a class.
  • Design Project: In a semester-long team project, you'll design, build, and test your own crowdsourcing / social computing system. If you have an ongoing research project that might benefit from having a crowdsourcing / social computing component, connecting to your research is encouraged.
  • In-class Activities: Each class will feature activities that will help you experience and practice the core concepts introduced in the course.

Course Staff

Instructor: Prof. Juho Kim
    Office Hours: by appointment

TA: Seoyoung Kim & Eunyoung Ko
    Office Hours: Thu 3-4pm @ N1-625

Staff Mailing List: cs473@kixlab.org

Time & Location

When: 4:00-5:15pm Tue/Thu
Where: N1-201


Course Website: https://kixlab.org/courses/cs473-fall-2018/
Submission & Grading: KLMS
Discussion Forum: Classum
Reading Groups: bit.ly/cs473-reading
Topic Presentations: bit.ly/cs473-signup



Week Date Topic Reading (response indicates a reading response is required for the article.) Due
1 8/28 Introduction & Course Overview (backchannel)
1 8/30 Social Computing Overview (backchannel) (1) Grudin, Jonathan. "Why CSCW applications fail: problems in the design and evaluation of organizational interfaces." CSCW 1988.
(2) Ackerman, Mark. "The Intellectual Challenge of CSCW: The Gap Between Social Requirements and Technical Feasibility". Human-computer interaction 15.2 (2000): 179-203.
2 9/4 Communication Tools (backchannel)
Led by Jisu, Narae, Yujin (slides)
(1) response Hollan & Stornetta. "Beyond Being There." CHI 1992.
(2) Venolia, Tang, & Inkpen. "SeeSaw: I See You Saw My Video Message." MobileHCI 2015.
RR by A
2 9/6 Collaboration Tools (backchannel)
Led by Oğuz, Thu, Mingyeong (slides)
(1) response Kittur & Kraut. "Harnessing the wisdom of crowds in wikipedia: quality through coordination." CSCW 2008.
(2) Vaish et al. "Crowd Research: Open and Scalable University Laboratories." UIST 2017.
Project 0: Team Formation
RR by B
3 9/11 Peer Production (backchannel)
Led by Wasachon & Peerapon (slides)
(1) response Michael Nielsen. "Open Science Now!" TEDxWaterloo video, 2011 (16 mins). + Tim Gowers, "Is massively collaborative mathematics possible?" 2009.
(2) Benkler, Yochai, and Helen Nissenbaum. "Commons‐based peer production and virtue." Journal of political philosophy 14.4 (2006): 394-419. (read pages 394-409)
RR by A
3 9/13 Discussion and Q&A (backchannel)
Led by Kinam & Seyeon (slides)
(1) response Mamykina et al. "Design Lessons from the Fastest Q&A Site in the West." CHI 2011.
(2) Zhang et al. "Wikum: Bridging Discussion Forums and Wikis Using Recursive Summarization." CSCW 2017.
RR by B
4 9/18 Collective Action & Civic Participation (backchannel)
Led by Sungwoo, Juan, Jibon (slides)
(1) response Wael Ghonim. "Inside the Egyption Revolution". TED video, 2011 (10 mins) + Wael Ghonim. "Let's Design Social Media that Drives Real Change". TEDGlobal>Geneva video, 2015 (13 mins).
(2) Starbird, Kate, and Leysia Palen. "Voluntweeters: Self-organizing by digital volunteers in times of crisis." CHI 2011.
RR by A
4 9/20 Social Networks 1 (backchannel)
Led by Dongmin & Peter (slides)
(1) response Eslami, Motahhare, et al. "I always assumed that I wasn't really that close to [her]: Reasoning about Invisible Algorithms in News Feeds." CHI 2015.
(2) Gilbert, Eric, and Karrie Karahalios. "Predicting tie strength with social media." CHI 2009.
Project 1: Ideation
RR by B
5 9/25 No class (Chuseok)
5 9/27 Social Networks 2 (backchannel)
Led by Youngin, Yixuan, Willmer
(1) response Manjoo, Farhad. "How Twitter Is Being Gamed to Feed Misinformation." The New York Times, 2017.05.31.
(2) Marcus, Adam, et al. "TwitInfo: Aggregating and Visualizing Microblogs for Event Exploration." CHI 2011.
RR by B
6 10/2 Project Pitches (schedule/order spreadsheet | backchannel)
6 10/4 Project Pitches (schedule/order spreadsheet | backchannel)
Project 2: Pitch
7 10/9 No class (Hangul Proclamation Day)
7 10/11 Project Feedback Meetings (signup link)
Assignment 1: Social Computing HOFS
8 10/16 No class (Midterms week)
8 10/18 No class (Midterms week)
9 10/23 Crowdsourcing & Human Computation Overview (backchannel)
Led by Yong-Jun & Chang-Su (slides)
(1) response Von Ahn, Luis, and Laura Dabbish. "Labeling images with a computer game." CHI 2004.
(2) Doan, Anhai, et al. "Crowdsourcing systems on the World-Wide Web." Communications of the ACM, April 2011.
RR by B
9 10/25 Platforms & Workers (backchannel)
Led by Sihyun, Dukhwa, Minji (slides)
(1) response Irani, Lilly C., and M. Silberman. "Turkopticon: interrupting worker invisibility in amazon mechanical turk." CHI 2013.
(2) Ipeirotis, Panagiotis G. "Analyzing the amazon mechanical turk marketplace." XRDS: Crossroads 17.2 (2010): 16-21.
(3) Bigham, Jeffrey P. "My MTurk (half) Workday."
Project 3: Low-fi Prototype
RR by A
10 10/30 Programming Paradigms (backchannel)
Led by Yeonsu, Jeongho, Seonju (slides)
(1) response Little, Greg, et al. "Turkit: human computation algorithms on mechanical turk." UIST 2010.
(2) Kittur, Aniket, et al. "Crowdforge: Crowdsourcing complex work." UIST 2011.
RR by B
10 11/1 Workflows (backchannel)
Led by Viktoriya & Zelalem (slides)
(1) response Bernstein, Michael S., et al. "Soylent: a word processor with a crowd inside." UIST 2010.
(2) Bigham, Jeffrey P., et al. "VizWiz: nearly real-time answers to visual questions." UIST 2010.
RR by A
11 11/6 Project Feedback Meetings (signup link)
11 11/8 Quality Control 1 (backchannel)
Led by Viet, Saelyne, & Junyong (slides)
(1) response Snow, Rion, et al. "Cheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks." EMNLP 2008.
(2) Ipeirotis, Panos. "Worker Evaluation in Crowdsourcing: Gold Data or Multiple Workers?" 2010.
RR by A
12 11/13 Quality Control 2 (backchannel)
Led by Gabriel, Viet, & Ern Chern (slides)
(1) response Harris, Mark. "How a lone hacker shredded the myth of crowdsourcing." Wired, 2015.
(2) Harper. "Crowdsourcing isn't broken." Wired, 2015.
RR by B
12 11/15 Human-AI Interaction (backchannel)
Led by Arjun, Dohyeong, & Yueru (slides)
(1) response Marti Hearst, James Allen, Curry I. Guinn, and Eric Horvitz. Mixed-initiative interaction. IEEE Intelligent Systems and their Applications 14.5 (1999): 14-23.
(2) Sankar, Shyam. The Rise of Human Computer Cooperation. TED Talk Video, 2012 (12 mins).
RR by A
13 11/20 Incentives & Gamification (backchannel)
Led by Yunho, Nyoungwoo, & Keon (slides)
(1) response Beenan, Gerard, et al. "Using social psychology to motivate contributions to online communities." CSCW 2004.
(2) Chris Lintott. "How to discover a planet from your sofa." TEDxCERN Video, 2013 (12 mins).
(3) Thorin Klosowski, "The Psychology of Gamification: Can Apps Keep You Motivated?" LifeHacker, February 2014.
Project 4: High-fi Prototype
RR by B
13 11/22 Experimentation (backchannel)
Led by (slides)
(1) response Reinecke, Katharina, and Krzysztof Z. Gajos. "LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples." CSCW 2015.
(2) Kohavi, Ron, et al. "Online controlled experiments at large scale." KDD 2013.
RR by A
14 11/27 Ethics & FAT (Fairness, Accountability, & Transparency) (backchannel)
Led by Do, Saebom, & Mathias (slides)
(1) Cathy O'Neil. response "The era of blind faith in big data must end." TED Video, 2017 (13 mins).
(2) Carole Cadwalladr and Emma Graham-Harrison. "How Cambridge Analytica turned Facebook ‘likes’ into a lucrative political tool." The Guardian, Mar. 2018.
(3) Nicholas Diakopoulos et al. "Principles for Accountable Algorithms and a Social Impact Statement for Algorithms."
RR by A+B
14 11/29 No class (Undergrad admission interview day)
15 12/4 Final Presentations
15 12/6 Final Presentations Project 5: Final Presentations
16 12/11 No class (Finals week)
16 12/13 No class (Finals week) Project 6: Final Report

Topics (tentative)

Major topics include:
  • Crowdsourcing and human computation
  • Social computing platforms
  • Social networks and online communities
  • Computer supported collaborative work
  • Privacy and ethics
  • Incentives and gamification
  • Online experimentation and analysis
  • Programming paradigms
  • Quality control and fighting malicious behaviors


  • Design project: 40%
  • Topic Presentation: 20%
  • Reading responses: 20%
  • Assignments: 10%
  • Class participation: 10%
Late policy: Three lowest reading response grades will be removed. No late submissions are allowed for the reading responses. For assignments and project milestones, you'll lose 10% for each late day. Submissions will be accepted until three days after the deadline.


There are no official course prerequisites. But assignments and the final project will require building features of a crowdsourcing / social computing system, so programming skills are needed. Knowledge or research experience in HCI (CS374 or equivalent) or social computing is useful, but not required.