Carnegie Mellon University
Twisted Signals: Multimedia Processing for the Arts
Instructor: Professor Jesse Stiles (jessestiles@cmu.edu)
TA: Jeena Yin (qyin@andrew.cmu.edu)
Time: M/W 9:30AM – 11:20AM
Location: Hunt Library Media Lab, HLA10A
Class website: https://courses.ideate.cmu.edu/18-090/f2019/
Instructor office hours: Tuesdays, 3:00PM to 5:00PM, HLA10A.
TA office hours: Thursdays, 12:00PM to 1:00PM, HLA10A or by appointment.
COURSE DESCRIPTION
This course presents an overview on manipulating and synthesizing sound, video, and control signals. Signals are the raw materials used in many forms of electronic art and design – electronic music, interactive art, video art, kinetic sculpture, and more. In these fields, signals are used to represent information about sound, images, sensors, and movement. By transforming and manipulating these types of signals, we are able to create powerful new tools for digital art, multimedia applications, music, responsive environments, video and sound installation, smart products, and beyond.
In this course we will study Signal Processing from a practical point-of-view, developing tools that can be easily integrated into art-making using the graphical programming environment Max (a.k.a. Max/MSP/Jitter). We will present a survey of Signal Processing techniques used in the sonic and visual arts, and will discuss the mathematical theories underlying these techniques. Students will be encouraged to combine, modify, and extend working examples of software to create original digital artworks.
COURSE CALENDAR
The course calendar is here.
LEARNING GOALS
- Present a survey of Signal Processing techniques that are used in the sonic and visual arts.
- Develop a working knowledge of how to implement, combine, and modify Signal Processing techniques, and to integrate these techniques into one’s art-making.
- Develop a familiarity with the mathematical concepts underlying Signal Processing techniques.
- Get strong at Max.
ASSIGNMENTS & PROJECTS
There are 4 short-term assignments that will be completed over the course of the semester as well as 2 longer-term projects. Due dates for every assignment are listed on the course calendar. Assignments are due before the start of the class on the date indicated.
Submission of assignments/projects is a two step process.
- The student creates a post on the course website with documentation and code/media. The documentation will include video and/or audio recordings as well as a written description of the work. Code and media/dependencies will be uploaded to the shared Google Drive and a link to the uploaded files will be included in the post. For details on the proper formatting and submission of work see this page. Work that is submitted with broken code or improperly formatted documentation will be considered incomplete.
- The student turns in the assignment via Canvas by delivering a link to your post on the course website.
Assignments that are delivered late will be penalized. See the grading section of the syllabus for further details.
ONLINE DISCUSSION/ANNOUNCEMENTS
Please join the class Facebook Group to participate in online discussion and receive course-related announcements.
Students are expected to post to the Facebook Group at least once per week. Your post can be about anything even tangentially related to the course. Posts can be as simple as a link to a youtube video that you find interesting – for example documentation of a work of electronic art, or a Max tutorial you found useful. Your posts could also just be questions or commentary that you might have about course-related topics throughout the semester. Posting to the Facebook Group is a great way of sharing your knowledge and learning experience with your fellow students. Please post the group by the end of the day every Sunday so that we can discuss posts in class the following Monday.
QUIZZES
There will be four short quizzes to test your progress on both the theory side of things (principles of Signal Processing) and the practical side of things (creating working patches using Max. Dates for the quizzes are on indicated on the course calendar.
SOFTWARE
The majority of the work we do in class will be done in Max, a graphical programming environment that is optimized for artists, musicians, and other creative practitioners. For access to the software you have three options:
You may work on the Mac Pro that is permanently installed in the Media Lab. Select the “Media Lab User” user account. When you are using this account create a folder with your Andrew ID and store all your files in this folder. It is very important that you back up your work every time you work on the computer. It is entirely possible that files on a communally used machine can be unexpectedly moved, erased, or modified. Get an external drive (physical or in the cloud) and back up your work constantly.
You may also work on any of the 40 MacBook Pro laptops that are available to you at the IDeATe lending desk. Similarly, you must be diligent in backing up your work when using these machines. The laptops and the Mac Pro all have Max 7, Logic Pro, Audacity, Arduino, Final Cut, Photoshop, Illustrator, and many other useful pieces of software.
Thirdly, you may work on your own computer. A one-year license of Max is available for $59. If you work on your own computer you will be responsible for acquiring and maintaining any of the software you choose to use. Your instructor cannot provide technical support on your computer for you.
Max is available from Cycling 74 at:
https://cycling74.com/shop/. Make sure you take advantage of the student discounts.
THE GOOGLE DRIVE
A Google Drive folder with course-related patches, media, and other ephemera can be found at: bit.ly/TS-F19. This folder will be updated throughout the semester as new patches and concepts are presented in class.
EQUIPMENT & FACILITIES
Please read and become familiar with the IDeATe lending and purchasing policies, which can be accessed at resources.ideate.cmu.edu. The IDeATe facilities are shared student resources and spaces. As such, all members of the IDeATe community are expected to be respectful of the equipment, the spaces, and fellow students and their projects. Always clean up after completing your work, put things back in their correct place, and leave the lab in better condition than you found it.
Students may be required to purchase materials to complete class projects. For convenience, some materials are available for borrowing and for purchase at IDeATe Lending (Hunt A29).
If you experience any issues with IDeATe facilities such as swipe card access, access to the room reservations system, non-functioning or damaged equipment, etc., send an email describing your issue to: help@ideate.cmu.edu.
ONLINE RESOURCES
A list of excellent online resources has been posted on the course website here. An essential resource is the Cycling 74 discussion forums, it is highly recommended that you join the forums and the read discussions that arise – you will learn a lot. This is also a great place to get questions answered, and to seek out advice for problem-solving. If you post questions clearly and politely you are likely to get some great responses very quickly.
ON TAKING NOTES
You should take notes.
USING THE MEDIA LAB
You will have access to the Media Lab outside of class hours – it is an excellent environment for you to work on your projects. To use the Media Lab outside of class time you must make a reservation using the online reservation system. When using the Media Lab outside of class you must comply with the policies indicated on the IDeATe website.
ACADEMIC INTEGRITY
When we are designing new software it is perfectly acceptable to use sections of code from examples found on the web, in help files, in tutorials, etc. Indeed, this is not only acceptable but is totally necessary if one wants to work efficiently.
Furthermore, when we are creating new works of electronic art is perfectly acceptable to make use of found materials (video files, sound files, images, etc.) to use as raw material in creating new works of art/music/design.
When using found code/images/sounds in your own work there are two requirements:
- Attribution. You must clearly identify where the code/images/sound came from.
- Transformation. You must significantly transform the materials you are using. You should extend the material, modify it into something new, offer new insight into the concepts underlying the material, etc. Work that uses borrowed code or other materials without significantly transforming those materials will result in a low grade.
More information on CMU’s Academic Integrity policy can be found at:
http://www.cmu.edu/academic-integrity/
ATTENDANCE & CLASS PARTICIPATION
Attendance: Unexcused absences are disruptive and disrespectful. If you are unable to attend class for any reason you must email me in advance. Failure to contact me before the start of class will result in an unexcused absence. Three or more unexcused absences will result in the drop of one letter grade per absence. This means that if you have three unexcused absences and would otherwise receive an A in the class, you will receive a B. If you have four unexcused absences and would otherwise receive an A, you will receive a C, etc.
Absences: You are responsible for what happens in class whether you’re here or not. Organize with your classmates to get class information and material that you have missed. We cannot repeat course material in class just for you.
Participation: You are invited, encouraged, and expected to engage actively in discussion, reflection and activities. Our class time is precious and limited. Please refrain from distracting electronic behavior such as texting during class time. Failure to follow this request will negatively affect your grade for class participation. Your contributions towards the class website will also count towards your class participation.
GRADING
Grading for the course is based on the assignments, projects, quizzes, and class participation.
Each of the short-term assignment will graded on a scale from 0 to 4 points. Of these 4 points, 2 shall be awarded to the quality and originality of the concept in the work. Another 2 points shall be awarded to the quality of the technical execution of the work. For both criteria, concept and execution, the determination shall be made accordingly:
0: Incomplete. For example: the code does not work, the work does not address the goal of the assignment, the work was not delivered on-time.
1: Satisfactory. The work was delivered on time and addresses the goal of the assignment.
2: Excellent. The work demonstrates an outstanding concept or execution. The work demonstrates great insight.
Your projects will be evaluated in the same way, but P1 will be worth 6 points and P2 will be worth 10 points. So the same as the Assignments, but then multiplied by 1.5 and 2.5 respectively.
Each of the four quizzes will be worth 2 points.
Class participation will be worth 10 points. This includes your participation in discussions and activities during class-time, as well as your online participation (posting to the Facebook Group, authoring high-quality documentation of your own work, and commenting on posts authored by your classmates and instructor).
Work that is delivered late will have the potential score reduced by 10 percent for every day of lateness. For example, work that is delivered one day late will only receive 90 percent of the points that would be received if it was delivered on time. Work that is delivered three days late will only receive 70 percent of the potential points, etc. Work will not be accepted if it is more than one week late.
To summarize:
Short term assignments: 4 x 4 = 16 points
Projects: 6 + 10 = 16 points
Quizzes: 4 x 2 = 8 points
Class participation: 10 points
Total: 50 points
Your final grade will be determined by multiplying the total by two and applying a curve, if appropriate.