Interaction Design (IxD) S t u d i o
Keith Evan Green, RA, PhD
Teaching Assistant: Bill Xu, tx66@cornell.edu
Tuesday/Thursday, 1:30-4:30pm, HEB 2L32 - "Assembly Studio," nextdoor to D2FS

C O U R S E   D E S C R I P T I O N   |   H C D   5 2 1 0
The built environment made interactive and adaptive by embedded computation has great promise to support and augment us at work, school, and home, as we roam, play, interconnect, and age. Students will iteratively design, prototype, and evaluate artful, meticulous, physical objects and environments responsive to specific challenges of an increasingly digital society.

P R E R E Q U I S I T E S   |   E N R O L L M E N T
• All students require professor's permission.
• Preference is given to student-majors in HCD, IS, and MAE, and students enrolled in the Robotics Minor. Students from other majors are welcomed as space permits.
• Enrollment is limited to twelve students to make full use of the D2FS, the digital and manual fabrication shop and staff located across the corridor from our teaching space.
• This course is for 4 credits, for letter grade only. There is no final exam.

S Y L L A B U S    |   S E E   A L S O   M Y  D E A  6 2 1 0   a n d   D E A  2 7 3 0
---------------------------------------------------------------------------------------------------------------

L E A R N I N G    O U T C O M E S - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Interaction Design Studio aims to cultivate new vocabularies of design and new, complex realms of understanding towards realizing artifacts and systems responsive to people and the planet. Three learning outcomes are expected of this course.

Outcome 1: To grapple with under-constrained, “wicked” problems and opportunities of living today and in the future.

Outcome 2: To identify design opportunities that may prove responsive to these problems and opportunities.

Outcome 3: To demonstrate an ability to realize interactive and adaptive devices and physical environments as working prototypes responsive to these problems and opportunities.

S T U D I O   C U L T U R E - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

DEA 5210 is not a formal course in mechatronics (the integration of electronics, coding, digital fabrication, interaction design, .... We do these things in this course in an applied way.

DEA 5210 is instead a design studio. In a design studio, class time is dedicated to:

(a) students working in class and outside class on assigned projects with guidance from the instructors.

(b) students presenting periodic status reports on and demos of their developing projects.

(c) students offering their design critiques of developing projects accomplished by other students.

This course is foremost focused on "thinking big" and "thinking different" about designing interactive and adaptive systems that enable and augment people in their everyday lives.

Students with no coding or electronics experience have done well in this course. The Grove hardware kit components make "physical computing" accessible to the novice who is willing to "roll up the sleeves" and work at it.

If you are new to mechatronics but want to try prototyping with an electronics breadboard rather than the Grove kit, you can use circuito.io that let's you drag-and-drop Arduino and Raspberry Pi components you select from a menu, which then produces for you the breadboard prototype (wiring, circuits, components) and the code for that system (more on this, below). If you want to go deeper yet, this publication is a great resource for electronics prototyping with a breadboard.

If you would like to create an app that interacts with or controls your physical object, you can develop the app rather easily using MIT App inventor.

More than anything else, DEA 5210 Interaction Design Studio expects that you - with the best information you can acquire, and with your most intensive and exploratory effort - develop a compelling, meticulously conceived and crafted artifact responsive to challenges and opportunities of living. Given this ambition, I ask my students to “take responsibility" for their education by attending class, participating, and submitting and presenting assignments that demonstrate learning and (even) advance thinking in the field. I firmly believe I have something to offer you in class.

C L A S S   O R G A N I Z A T I O N - - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - -

A key objective for this Intereaction Design (IxD) Studio course is to have every student in the class design not only their project but every project in the class. How? By benefiting from various inputs:

• Once per week, you will present a brief update on your developing project, noting what you did in the past week, and why you did it. You can upload slides to our shared folder, ahead of class, and/or show us a developing artifact.
• Once per week, you will select a peer who will be your critique partner for the class session. Confer with your partner at least once during that class.
• You will benefit from informal exchanges with peers.
• You will deliver formal presentations at designated milestones throughout the semester.
• You will advance your project through "desk crits" with the professor and TA.
• You will consult with D2FS staff on fabricating your project.
• You wil review and respond to critique and grades.
  
  —and perhaps most critically—
  
  
• You will offer design critiques of developing projects accomplished by other students, for which you will be graded as part of your participation grade (see below).

M Y   F O U N D A T I O N A L   P U B L I C A T I O N S - - - - - - - - - - - - - - - - - - - - - -

• Keith Evan Green. Rethinking the Machines in Which We Live. IEEE RAS.
• Keith Evan Green. Architectural Robotics. MIT Press, 2016.
• Keith Evan Green. Architectural Robotics, Inevitably. ACM Interactions.

I N T E R A C T I V E   A R T I F A C T S    /   E X A M P L E S - - - - - - - - - - - - - - - - -

• ixD examples from the ACM "Demo Hour" of Interactions.

• A classic example: musicBottles (Hiroshi Isshi, Tangible Media Lab, MIT).

• The lovebox, a simple, meaningful, whimsical product.

• Student examples from DEA 5210 on this page, left-column, DEA 6210, DEA 2730.

• Projects from my lab - the ARL (Architectural Robotics Lab):
> SocialStools CHI
> Growbot DIS BEST DEMO PAPER
> SORT RO-MAN
> AWE CHI
> pheB 1 TEI, RO-MAN, and 2 IROS

• Projects by Bill Gaver:
> Drift Table (video).
> History Table Cloth
> Indoor Weather Station
> Key Table
> The Prayer Companion

• Projects by Wendy Ju:
> Mechanical Ottoman: Engaging and Taking Leave.
> Collaboration with Robotic Drawers.

• You might also find inspiration in projects presented in MAKE and Instructables.

V I D E O S   /   E X A M P L E S   &   P R O D U C T I O N   G U I D E - - - - - - - - - - - -

• My guide for making videos (this applies less-so to the first assignment).

R E A D I N G S - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

• Richard McGuire. Here. Pantheon Press, 2014.
• Paul Dourish. Embodied Interaction. MIT Press, 2001.
• Steve Dow. Wizard of Oz Interfaces [WOZ].
• Jen Golbeck. Back off, man. I’m a scientist.’Using Fiction to Teach Beginners HCI.
• E. Grönvall, et al. Causing commotion with a shape-changing bench.
• Kristina Höök. Affective Interaction.
• Hiroshi Ishii. Tangilbe Bits and Radical Atoms: Beyond Tangible Bits.
• Bruce Mau. An Incomplete Manifesto for Growth.
• John McCarthy and Peter Wright. Technology as Experience. MIT Press, 2004.
• William J. Mitchell. Computers for Living In, from e-topia. MIT Press, 1999.
• Nicholas Negroponte. Soft Architecture Machines. MIT Press, 1970.
• Richard Picard. Affective Computing. MIT Press, 1997.
• Terry Winograd. From Computing Machinery to Interaction Design.

M A T E R I A L S   N E E D E D - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

You will be provided the following in class (as part of the course fee of $60):

- 1 Grove Beginner Kit for Arduino ($23.88 from Mouser).
- 1 Grove Servo ($6.90 from Mouser).
- 1 Grove Ultrasonic Sensor ($4.30 from Mouser).
- 1 Grove Gesture Sensor ($10.99 from Mouser).
- 1 Grove RGB LED Stick (15-WS2813 Mini; $5.40 from Mouser).
- 1 Grove Switch ($3.20 from Mouser).
- Panels (pre-cut) to make an enclosure for your assignment-1. If you need a different size enclosure, you can work with D2FS on laser-cutting panels to-size from digital files you generate using CaseMaker.

Optional: Additional Grove components are available from Mouser electronics with competitive pricing and quick shipping. Grove components are also available from Amazon (often at a higher price), from DigiKey, from other vendors listed on this page, below, and from its manufacturer, Seeed Studio, which ships from its US warehouse.

You will also need:

• 1 sketchbook like this one or a comparable one found in our bookstore.
  
  
• Aluminum foil - 1 roll, any brand, as used in your kitchen (example).
  
  
• Lithium battery pack (this inexpensive one works and will charge your phone!).
  
  
• Your laptop. You need your laptop in class, every class session. If your laptop is not equipped with a port to plug in a USB-A cable (that comes with the Grove Kit), then you need a USB-C hub (here's one) that plugs into your laptop's port and provides a USB-A port. Newer Mac laptops need this USB-C hub, as Macs no longer have USB-A ports on them.
  

• Fabrication materials for early, rapid prototyping. These include: cardboard from shipping boxes, plastic from fruit and vegetable containers, and craft materials needed to construct your prototypes. Many of these materials you have already, at no-cost; other materials are avaiable from the Cornell Bookstore, Michael's at the Ithaca Mall, and online at Utrecht, Blick, and Amazon.
  
  
• Fabrication materials (as needed) for high-fidelity prototyping
  • Some fabrication materials are available to you at no cost from our D2FS.
  • Coroplast corrugated plastic is easy to work with and low-cost. I like the colorless, translucent finish; small quantities can be found on eBay via Duco Plastics (your best vendor), on Amazon from various vendors, and from Home Depot (white in large, single sheets; clear by case of 10).
  • Honeycomb cardboard is inexpensive and rigid enough to build furniture from it.
  • Acrylic sheets and other plastics are available online from TAP Plastics (cut to your size with reasonable precision) and from ePlastics (cut to size, less expensive than TAP, but less control over cutting dimensions and multiple cuts).
  • Cut2Size Metals.
  • Aluminum Composite Material (ACM) as in the red entry of HEB as well as my LIT ROOM project; available locally (in Syracuse) from Polymershapes, contact Kevin Passerell.
  
  
• Optional: an inexpensive book on human-centered design methods, The Pocket Universal Methods of Design: 100 Ways to Research Complex Problems,... Available for about $13 from the Cornell Bookstore and from Amazon. This inexpensive book may be useful to those who are new to human-centered design: Do not purchase the similarly titled, The Pocket Universal Principles of Design: 150 Essential....
  
  
• Tip: If you don't have access to video editing software (e.g., iMovie from Apple), try Open Shop, a free video editing app for windows https://www.openshot.org/. And if you don't have Adobe suite, try Canva (free 30-day trial) and Paint.net.

S C H E D U L E   B Y   W E E K - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

NOTE: During the second and third class meetings, we will also do D2FS shop training to learn how to use the basic power tools safely. Please come to class with close-toe shoes (no sandals).

Week 01.23 | Course Intro | Assignment-1 Intro; Proposals as a GIF (see GIFs below)

Week 01.30 | Ideation: Rapid Prototyping and Scenarios; Paper•Mech & Mechanisms

Week 02.06 | Grove-Prototyping; Morphological Chart; Precedents (e.g. ACM DL)

Week 02.13 | Prototype Development; Role Play with a Think Aloud; Video Production

Week 02.20 | Video Iterative Development > preliminary DEMO + draft video screening

Week 02.27 | DEMOS and SCREENINGS  | Cornell Break Tuesday

Week 03.06 | Assignment-2 Intro; Individual proposals using DALL-E2; Form Teams

Week 03.13 | Storyboard; Prototype Development as a GIF or ? (Team Effort hereon)

Week 03.20 | Prototype Development > early DEMO and Scenario

Week 03.27 | Prototype Development > Role Playing with a Think Aloud

Week 04.03 | Cornell Break

Week 04.10 | Prototype & Video Iteration > hardware system DEMO

Week 04.17 | Prototype & Video Iteration > Think Aloud > DEMO

Week 04.24 | Prototype & Video Iteration > UX Survey / SUS (how to score) > DEMO

Week 05.01 | Prototype & video iteration > DEMO; Course and Peer Evaluation

Final Class 05.09 | DEMOS and SCREENINGS

DEADLINE: tbd
Upload all requirements to our shared folder for grading by date/time announced ear semester by Cornell U. registrar here.

A S S I G N M E N T S - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

There are two assignments for this course.

Both assignments are framed by Here, the graphic novel by Richard McGuire.

Here is "the story of a corner of a room and of the events that have occurred in that space over the course of hundreds of thousands of years," from the distant past to the far future.

More specifically, you are asked to design two interactive prototypes: a "small" one (assignment-1, an individual project) and a "larger" one (assignment-2, a group project). These two interactive prototypes must enable or augment one or more of the novel's characters, in that room, at a single instance shown in one pair of pages (one image of the room) from Here. In designing your prototypes, assume that you are transporting your interactive prototypes through time, from today to the room in the year shown in the image. This means that what you design today, with today's means, will enter the space of another time in our planet's existence.

The first assignment, undertaken by each student, is intended to be a fast-paced engagement. The second assignment is undertaken by teams of 2-4 students and provides a longer, deeper development of the design following a trajectory like this:

• Conceptualization
• Prototyping / Rapid, low-fidelity
• Concept Generation: GIF, Scenario, Morphological Chart, Storyboard
• Prototyping / Hardware/Software
• User Testing
• Video making and Reporting
• Documentation

Team composition for the second assignment will be formed by the instructor(s) based partly on proposals pitched in class by class members.

Your development of both prototypes may beneft, intellectually, from:

• Gopnik, Adam. 2018 (July 30). "What Can We Learn from Utopians of the Past?
  Four nineteenth-century authors offered blueprints for a better world...." The New Yorker.

Assignment-1 | Scale: "Small" (45% of the course grade; individual effort)

Using the Grove kit (or an Arduino and breadboard, if you wish), develop an interactive, physical device at small scale that enables or augments one or more of Here's characters, in that room, at a single instance shown in one pair of pages (one image of the room) from Here. You may integrate any manner of input device, actuator, hacked device (e.g. a toy, a camera) and any technological approach (e.g., machine learning, computer vision, AR).

To get us started quickly (while you are waiting for your copy of Here to arrive), use one of the following five instances of the room from the book:

Here, 1933, holding hands and hanging a picture
Here, 1957, trying to remember
Here, 1964, dancing
Here, 2005, with the pull-out bed
Here, 2213, in the future

Practically, your box enclosure must be constructed from the panels provided in class. If you need a different size enclosure, you can work with D2FS on laser-cutting panels to-size from digital files you generate using CaseMaker.

Intellectually, your "small" assignment may take inspiration from the boxes of artist Joseph Cornell (one such box is in Cornell's Johnson Museum of Art), the words of Brian Eno (producer of, e.g., Coldplay's "Viva La Vida", ambient music pioneer, and member or the band Roxy Music), and three perspectives on affective computing / designing for emotion:

• Affective Computing (emotion as biologically-based cognition), by Roz Picard (her TEDx Talk).
  
  
• Affective Interaction (emotion as a culturally-determined construct), by Phoebe Sengers of Cornell IS, Paul Dourish, Kristina Höök.
  
  
• Technology as Experience (emotion as part of a larger interactive experience), by John McCarthy and Peter Wright, Don Norman, and Bill Gaver.

The kind of artifact we are striving for is small in scale, whimsical/poetic, beautifully crafted, interactive in simple ways, and meaningful/purposeful.

Assignment-2 | Scale: "Larger" (45% of your course grade; team effort)

Assignment-2 is the same as assignment-1, except that:

(a) the prototype will be of larger scale.
(b) the prototype will be developed in groups of 2-4 students.
(b) you will use a room from the book of your group's choice (that is not one of the 5 instances linked above for assignment-1).

G R A D I N G   /   G R A D I N G   R U B R I C - - - - - - -- - - - - - - - - - - - - - - - - - - - - -

Throughout this course—an intimate and intensive “conversation” across students and the professor— students will have ample opportunity to receive feedback on their work. Grading is based on a 100-point scale.

Review my grading rubric for course deliverables.

(10 points) attendance, participation, and your assigned reading reviews / uploaded before each class. An attendance sheet must be signed by you in the first ten minutes of class for you to be counted as present. If you expect to be later than 10 minutes on a given day, or if you are absent, email both the professor and the TA (if any) with the cause for your late arrival or absence; these will be considered as a valid excuse or not. To assess participation during class, names may be pulled "from a hat" to identify student critics who will then peer-review the developing work accomplished by other students; the quality of the student critique will form part of the 10 point assessment.

The deliverables are the same for both assignments, except as noted:

(45 points for each assignment) Review my grading rubric

• A working, refined prototype at 1:1 scale (full-sized), made interactive by way of sensors and actuators to create combinations of movement, lighting, displays, and/or sound. You may integrate any manner of input device, actuator, hacked device (e.g., a toy, a camera) and any technological approach (e.g. machine learning, computer vision, AR, ...).
  
  
• Two "new pages" (i.e., one complete image of the room) for Here, adhering to the drawing and color style of the novel. These pages will illustrate how your prototype would ideally look, operate, and impact the lives of one or more characters, in that room, at a single instance shown in one pair of pages (one image of the room) from Here.
  
  
• The video [my guide] communicates the full, cohesive story of the designed artifact your team produced, answering why, for whom, and how it was developed, including an overview of the methods used to design and evaluate it. The video will adhere to the requirements for the Video Showcase submission to the benchmark HCI conference, CHI (Human Factors in Computing Systems). Here are videos from previous CHI Video Showcases: 2018For the video, include in your documentation a URL link to your video in Vimeo or YouTube; and upload to our shared folder an MP4 file reduced to < 30MB using, e.g., Handbrake (see my video guide). The video will otherwise adhere to the requirements for the Video Showcase submission to the ACM conference CHI (a benchmark for design research), where you will also find example videos.
  

  Tip: For your demo video, you may want to add a remote environment (e.g., a dorm lobby, the Cornell campus, a Parisian café, an assistive living unit, a museum lobby) as a preferred physical context for your design; however, such an environment is not always readily accessible to you. An easy strategy for adding this physical context is as follows: video record your working prototype (with “actors” or scale figures of people if your prototype is to-scale) on a white background (e.g. in front of a white wall); then, add your background context photo (e.g. a photo of the café) as a virtual background in Zoom and record your screen.

• Documentation of your design, in the form of a report, includes the following: (a) a unique name for your prototype, (b) jpg, 300dpi, of your "money shot" (i.e. best photo), (c) an abstract, (d) a scenario that describes how people interact with your design, (e) a brief description of how your prototype operates in technical terms, (f) a photo of your prototype with its components labeled, (g) the design process, including outcomes from your Role Playing experiment and how these outcomes informed your design iterations, (h) a discussion (of what worked and didn't work), (i) proposal of future work (as a response to your discussion), (i) a link to your video uploaded to Vimeo, and (k) the code. Upload your documentation to our shared drive as a print quality pdf document. This example from a previous class provides a model for this documentation, but it does not contain every requirement listed above.

U S I N G   G R O V E   &   A R D U I N O   C O D E - - - - - - - - - - - - - - - - - - - - - - - - -

G E T T I N G   S T A R T E D
• Grove Beginner Kit for Arduino Wiki.
• Follow the Wiki to install Arduino IDE and the CP2102 USB Driver for your OS.
• Two videos (1 and 2) introduce the Grove Kit: how you work with it and code it
• An article, a video, and my guide on how to use ChatGPT to code Arduino for you!
• My useful notes on how to correct for common code Errors and what the code means.

A R D U I N O   C O D E  Y O U   C A N   C O P Y   &   P A S T E  
Paste these codes in Arduino! Change values in the code! Modify with ChatGPT!
• A Sound Sensor controls a single LED [code].           
• A Button controls a single LED and a Buzzer [code].
• A Potentiometer (i.e., Rotary Angle Sensor) controls a Servo Motor [code].
• An Ultrasonic Sensor controls a Servo Motor and an RGB LED Stick [code].
• A Gesture Sensor controls a Servo Motor and a single LED [code].
• A Light Sensor controls a Servo Motor and an RGB LED Stick; a Switch turns the whole system on/off [code].

F I N D I N G   A R D U I N O   C O D E   F O R   G R O V E
• You can find lots of code already built into the Arduino software (IDE): Open up Arduino, select File > Examples, select an example and it will open in an Arduino window, ready to upload to your Arduino board!
• All of the built-in examples are thoroughly described here. Follow their logic to
construct the code for your project.
• An Arduino Library LIst for Grove components. Find code for lots of modules here!

M O R E   A R D U I N O   P R O J E C T S   W I T H   C O D E
• Project Examples from Grove
• Project examples from "Instructables" with code and documentation.
• Project examples from Arduino Project Hub
• Grove tutorial that has 9 simple codes you can copy-and-paste.
• Numerous Grove Tutorials.
• Many more Grove components are available than found in the kit.

A   D E E P E R   D I V E   I N T O   G R O V E   &   A R D U I N O   GITHUB
• Seeed Studio's open source community (here and here) and Help Forum.
• GitHub is the open-source repository of code, including code for Grove Arduino.
• 15-Video Tutorial for Arduino (free) from Jeremy Blum, Cornell alumn! - great!

A B O U T   H A R D W A R E  
• How to work with wire: stripping, soldering, crimping, braiding it.
• How to extend (chain together) Grove kit wires.
• How to use an LED strip: solder it to a Grove connector and code it.
• How to convert your servo for continuous rotation (how to guide).
• How to breadboard prototype; about basic electronics.
• How to make two Arduinos communicate wirelessly using Blynk or NRF.
• About mechanical movements - 507 mechanisms to inspire you.
• About Interactive Paper Craft.
• About LiDAR sensors - measure distance and generate/identify objects and gestures.
• About using relays - "electronic switches."
• About making Arduino robots.