Human-Machine Virtuosity
Spring 2020 Syllabus

Summary

16-455/48-530: Human-Machine Virtuosity: Hybrid Skill, Fabrication, and Design
MW 9:30-11:20AM
MMH C4 (dFab Lab)
Instructors: Dr. Garth Zeglin, Prof. Joshua Bard
https://courses.ideate.cmu.edu/16-455

Offered by the School of Architecture and Robotics Institute for the IDeATe program at Carnegie Mellon University.

Course Description

Human dexterous skill embodies a wealth of physical understanding which complements computer-based design and machine fabrication. This project-oriented course explores the duality between hand and machine through the practical development of innovative design and fabrication systems. These systems fluidly combine the expressivity and intuition of physical tools with the scalability and precision of the digital realm. Students will develop novel hybrid design and production workflows combining analog and digital processes to support the design and fabrication of their chosen projects. Specific skills covered include 3D modeling (CAD), 3D scanning, algorithmic geometric modeling, digital fabrication (additive and subtractive manufacturing), motion capture and computer based sensing, and human-robot interaction design. Areas of interest include architecture, art, and product design.

This course is part of the Integrative Design, Arts, and Technology (IDeATe) program at Carnegie Mellon University. The course is an elective offered under the Intelligent Environments and Physical Computing concentrations. The prerequisite is one of the appropriate IDeATe portal courses or instructor permission.

Enrollment is limited to 25 students, drawn from all departments. The central IDeATe principle is to bring together students from multiple disciplines to develop hybrid skills. Students will be expected to learn skills from outside their home discipline, but more importantly, to develop their abilities to collaborate in diverse groups.

Learning Objectives

Upon completion of this course the student will be able to:

  1. negotiate the strengths and weaknesses in the following: digital design representation tools; digitally controlled additive and subtractive fabrication processes; and dexterous human skill and physical intuition, including handcraft and everyday manipulation.
  2. develop hybrid physical and digital design processes customized for a particular domain.
  3. enable human-machine collaboration by transferring human skill to machine movement and designing interaction systems for fluid cooperation.
  4. expand design thinking to customize the means and methods of production.
  5. create innovative physical artifacts enabled by their novel hybrid production systems.
  6. use Rhino, Grasshopper, and Python to implement a custom design workflow.
  7. use a digitally controlled additive or subtractive fabrication process to realize tangible artifacts designed using a custom workflow.

Grading Rubric

Each assignment is graded spanning three categories: concept, execution, and documentation. The final grade is weighted with 40% for each of the two projects, and 20% for all exercises combined.

Please note also that much of the feedback on your work will come in the form of critique and commentary rather than simply your numerical scores. Please attend to this; the commentary will be a much more substantive guide to your personal learning process than the scoring.

Each project will also include a peer evaluation component. The purpose of this element is to identify the specific contributions of each group member to the project outcome. Individual scores for a project may vary from the group score based on peer reports and instructor observations.

Course Goals

The essential focus of this course is to understand the nature of dexterous skill. This idea exists at the intersection of traditional craft, design process, manufacturing, and robotics. By understanding dexterous skills we can develop systems to enable hybrid skills. The following sections elaborate on this focus through a discussion of the underlying questions of human-machine virtuosity.

Hybrid Human-Digital Workflow

The following sketch represents a template for how human skill can be combined with digital manipulation and output to create a hybrid design workflow. Each element of this abstract workflow may be elaborated in specific ways related to the overall design goals.

Hybrid Skill Diagram

On the left is represented human skill, which is expressed in the physical world through the manipulation of tools and material. This manipulation can be captured and measured via several means of motion capture or scanning and integrated into a digital model in Rhino. This model can be transformed both to provide feedback to the human designer and to produce artifacts. The real-time feedback pathway is represented at the right as ‘Visual’ and ‘Audio’ transformations, and the fabrication pathways are represented by ‘CNC Router’, etc.. As in a normal iterative design process, these prototype outputs are provided to the human for critique and design revision.

Dexterous Skill

Our particular concern is with skills which apply human physical intuition to fabrication and manipulation. For practical reasons we focus on tool-based skills as they are both easier to measure and frequently reflect practiced learning, especially in handcraft. This is distinct from non-tool-based fabrication skills such as bare-hand clay forming, or non-fabrication skills such as musical performance. The following questions may help inspire inquiry into the nature of these skills.

  1. What are the fundamental material processes performed by the tool?
  2. What are the dynamic processes involved in the tool actions? Examples include momentum transfer in hammering; viscous friction in clay cutting; asymmetric compliance in wood scraping.
  3. How do variations in natural source materials affect the process? How has this been incorporated into traditional design techniques?
  4. What is the characteristic amount of practice required to become an expert human user? What is the nature of that practice? How does expert use differ from novice use? Is the tool use characterized as handcraft?
  5. What are the fundamental perception processes required by a human user?
  6. How many degrees of freedom are intrinsic to the required movements?
  7. Is the tool control process more characterized by force feedback or kinematics?

Hybrid Human-Digital Processes

The human body is a marvelous instrument, one which we are far from replicating in robotics. Instead we create tools which augment and extend our skills, then learn to apply those tools.

A central thesis of the course is that human skill wielding a physical tool can be deeply expressive in a way which is complementary to the precision and scalability of digital design tools. This is a middle path between pure traditional craft and industrial manufacturing which uses entirely different processes. Perhaps in the future we will invent new robots which can dextrously wield tools in a way which responds to material variation, or new industrial processes which more closely simulate handcraft. But even then the question remains of creating design processes which can retain the richness of handcraft.

There are several means we will explore for creating system for hybrid skills:

  1. Measurement and simulation of natural processes. Natural objects are information-dense and so require great effort to model by hand with CAD systems. However, digital systems are suited to simulation of natural processes, so this can be applied as a means for algorithmic design.
  2. Algorithmically generated prompts for humans. Digital representations are strong means for creating precision, replication, variations in scale, and communicating intent over time and space. The immediacy of a skill-based process may be enhanced by providing real-time graphical or auditory cues to a human.
  3. Pure digital manipulation of human process data. Algorithmic transformations can be applied to measurements of human gesture, even if based on non-physical principles such as arbitrary scaling. This combines the strengths of physical process and digital transformation with the goal of producing designs which would not result from either alone.

The space of possible outcomes is vast, so the following questions may help to resolve which student projects fit within the scope of the course.

  1. Are the transformations motivated by the objective?
  2. Are digital transforms used which cannot be replicated physically?
  3. Does the project highlight the differing constraints of physical and digital manipulation?
  4. Is the entire pipeline coherent and justifiable?
  5. Where is the locus of intent?
  6. What is the immediacy?
  7. Are the digital transformations and physical transformations balanced effectively?
  8. Where is the autonomy of the author?
  9. What is the conceit of the digital feedback system?
  10. Does the digital system bring outside information to bear in useful ways, e.g. images?
  11. Does data generated from physical simulation reflect natural physics or artificial physics? E.g., particle systems can generate consistent dynamics inconsistent with the natural world, but useful as trajectory references.
  12. Is the perfection of digital memory used effectively?

Material Considerations

Hand tools and related skills are already optimized for particular materials. For cost and practicality reasons, the course will focus on relatively low-cost materials such as wood, wax, foam, sand, clay, sheet metal, and plaster.

The essential material questions related to hybrid skills are as follows:

  1. How do the specific physical properties govern the dynamics of the related skills? E.g., plaster has a curing time profile which determines the workability, sequence of forming steps, and job pace and scale.
  2. Can digital transformation help simulate material constraints in a different medium? I.e., can a material be used as a faux material in a novel way via simulation and prompting?
  3. Are physical transformations incorporated which utilize material properties in expressive ways? E.g., sheet metal can be cut and scored prior to bending to achieve composite effects; plaster can be molded to duplicate surface textures. Multiple physical transformations can provide opportunities complementary to digital transformations.

Design Interaction

The purpose of the course is to develop novel design tools which combine human skill and digital means. The success of the projects will be gauged along two trajectories: 1. whether they create an enjoyable and robust design experience, encouraging sustained practice 2. whether they create potential for the creation of compelling and novel design artifacts.

One metric is whether the tools encourage a sketching approach: can the tool enable a designer to rapidly explore an idea with approximation and abstraction, and then iteratively refine it?

Temporality and Design History

Designed artifacts are often considered as static entities but computational and fabrication processes are both intrinsically temporal. The usual resolution of this duality is that a design represents an outcome of a process, with a choice of the degree of history legible in the form.

The hybrid nature of digital transformations offers other resolutions. For example, a non-physical process simulation could produce an artificial history unrelated to the actual material processes so that the legible result would combine the actual and the fictional in a novel way.

Geometric Considerations

A strength of digital representation is mathematical description of form. Hybrid systems offer opportunities for superposition of mathematical and natural form, e.g. for geometric discretization and rationalization, or for combining closed-form features such as catenary curves, circles, ruled surfaces, or topographic isocontours with organic variation. An example of a prompting system supporting this would be a dynamic French curve that projects geometry extrapolated from gesture in real time.