-
Notifications
You must be signed in to change notification settings - Fork 0
Homework Ellen
Section 06 Info | |
---|---|
Meeting Time | Tues 9:30am - 12:00pm ET |
Location | Room 412 |
Contact |
ellen.nickles@nyu.edu On weekdays I aim to respond within 24 hours |
Office Hours | Mon 2:30pm – 4:30pm Tue 1:00pm – 3:00pm Sign up here for Zoom or in person (near Design Lab) |
Support |
Resident office hours (schedule) The Coding Lab (schedule or drop-in help) How to ask code-related questions: examples |
Additional Materials |
Submit creative assignments Assignment responses Our Miro board Course syllabus p5.js Reference / Web Editor The Coding Train's Intro to p5.js Allison Parrish’s Creative Coding Notes xin xin's Intro to p5.js Qtv's Creative Coding tutorials in Chinese (also on bilibili) HappyCoding.io |
- About Our Course
- Week 8 • Nov 1 • Images, Pixels, Color
- Week 9 • Nov 8 • Video and Body Point Estimation
- Week 10 • Nov 15 • Sound
- Week 11 • Nov 22 • Some Text, Mostly Data
- Week 12 • Nov 29 • Final Project Proposals
- Week 13 • Dec 6 • Final Project User Testing
- Week 14 • Dec 13 • Final Project Presentations
Upon completion of the second half of the ICM course, Media, we will be able to:
- apply fundamental programming concepts to generate and manipulate media including images, video, sound, text, and data.
- experiment with media-specific, computational techniques and apply them to our creative work.
ICM Media is a seven-week course that meets once a week in person at the scheduled time. We will have assignments to do before each class, and these assignments will build towards a final exercise to share at the end of the seven weeks.
This course embraces an active-learning approach with the belief that programming is best understood by doing. In keeping with this approach, class time will be structured as a workshop and seminar instead of a lecture, relying heavily on group discussion and participation. Weekly tutorials before class sessions will introduce computational media topics. During class, supplemental code examples will be presented and discussed before hands-on work together. Time will also be allotted to share our weekly assignments. Examples and assignments will iteratively build upon lessons learned in preceding classes.
I encourage you to reach out to me outside of class to ask questions, share ideas/feedback, and discuss topics in detail.
All assignments will be posted on this page, and all are required. Unless otherwise stated, assignments are due the night before class so that I have a chance to review your questions before we meet. All assignments are expected to be submitted by the stated deadline. If you anticipate any challenges meeting the deadline, please reach out to me so that we can consider your options together.
Before class, read and/or watch the assigned tutorials about computational media topics and programming techniques to practice together when we meet in class.
There are worksheets* (linked below) to practice techniques introduced in this course. You are expected to add links to your p5 sketches directly in these documents.
There are also open-ended exercises to explore the topics for creative expression. You are expected to document your process, include a link to your p5 sketch, and answer these questions:
- What did you do?
- What worked?
- What didn’t work, and what steps did you take to try to solve the issue?
- Did any code-related questions come up for you?
- Be sure to cite (link to) any sources that you used.
Submit these assignments using our homework form*.
Here is the spreadsheet with everyone's responses*.
A big part of learning at ITP is learning from each other. Share your work and in return you'll see everyone else's!
*Log in to your NYU account to access the document.
**If you are struggling with the weekly assignments, put your energy into writing about what didn't work and how you tried to solve it. It is totally okay to vent your frustrations. Document your work to reference later and to help you synthesize what you did and tried to do.
- Welcome new students!
- Overview of ICM Media
- Image files
- Objects with images
- Pixel regions
- Pixel manipulation
- Syllabus resources
- Media resources
- Supported media type and formats
- Pxhere | Unsplash | Wikimedia Commons or your own?
- p5 examples
- All the ways to create images
- Getting started with image files
- Animating images
- Objects with the same image
- Working with many images
- Pixel and pixel region info
- Objects of pixel regions
- Demonstrating alpha
- Canvas pixel array
- Image pixel array - change pixel color
- Pixelated image using pixel array - just get pixel color
- Pixelated image with
get()
- Related projects
- gämma by p1xelfool
- Joiri Minaya’s #dominicanwomangooglesearch
- Photomontage by Sohei Nishino*
- Four Artworks by Kensuke Koike, Photographs by Tommy Kha for The Myth of Asian American Identity*
- The Constructed Self by Karen Navarro
- The Moment We Meet by Hsin-Chien Huang
- Joiners by David Hockney
- Garbage Selfie by Jenny Odell
- Pushin' Pusheen Pixels by August Luhrs
- 2D Water Ripple Coding Challenge (p5 version by Lisa Jamhoury)
- First 1/3 of Coding Train Coding Challenge: Slitscan Time Displacement Effect using the
copy()
function with images - *Free NYTimes subscription via NYU
- PRACTICE • Week 8 Worksheet
- CREATE • PARTS OF A WHOLE
- Create and/or manipulate an image (or images) using pixel regions and/or individual pixels to create an alternative reality than depicted in the source image(s). Describe in 1-3 keywords how the result feels different from the source image(s). Write a blog post documenting your process and result. Work with any kind of imagery you want. Whatever you choose, you might consider these elements of a photograph in your computational approach:
- Techniques and attributes: Light, focus, time, motion, vantage point/point of view, framing, cropping, composition
- Composition: Shape, line, angle, color and tone, pattern, depth, overall composition
- Content: Subject, background, foreground, people, story, mood/feeling, symbol/metaphor, style, genre, artist's intention/purpose, meaning
- Submit on our homework form.
- Create and/or manipulate an image (or images) using pixel regions and/or individual pixels to create an alternative reality than depicted in the source image(s). Describe in 1-3 keywords how the result feels different from the source image(s). Write a blog post documenting your process and result. Work with any kind of imagery you want. Whatever you choose, you might consider these elements of a photograph in your computational approach:
- READ
- Working with video tutorial by Allison Parrish
- WATCH
- Coding Train Video 11.1 on capturing your webcam ~10min
- Coding Train videos Beginner's Guide to Machine Learning with ml5.js and Pose Estimation with PoseNet ~40min
- Coding Train video Teachable Machine: Images and Snake ~30min
- A visual and intuitive understanding of deep learning from Octavio Good (0:00 - 9:40) ~10min
- (OPTIONAL) Videos 11.2, 11.4 - 11.7 on video photobooths, pixel mirrors, painting with pixels, and slit-scan video ~1hr 20min
- Share our homework
- Color functions and HSB mode
- Video files
- Working locally with VS Code
- Webcam
- Body point estimation with PoseNet, Handpose, and Facemesh
- Related projects
- Syllabus resources
- Media resources
- Supported media type and formats
- Videvo | Videezy or your own
- Option to downsize video files: VLC (free) > File > Convert / Stream > Customize > Video codec
- p5.MediaElement methods (.play(), .stop(), .onended(), and more!)
- p5 examples
- Color
- Video
- Webcam
- PoseNet
- Handpose
- Facemesh
- Work locally with Visual Studio Code (VS Code)
- All uploaded media files to the p5 web editor, must be < 5MB
- For working with media files that are > 5MB, use a local text editor. There are many options! VS Code provides several p5 add-ons to get started quickly.
- PART 1: Set up VS Code
- Download and install VS Code (not the full Visual Studio suite)
- Once installed, click on the Extensions icon from the left sidebar menu and search for: p5.vscode by Sam Lavigne
- PART 2: Create a p5 project
- Open the Command Palette (with command-shift-p on Mac, or ctrl-shift-p on Windows) and then start typing and select "Create p5.js Project".
- Select “New Folder” to create a folder in which to put your project’s files, and then click, “Open”
- Drag your media files into your new project folder.
- Start coding on the sketch.js file. Make sure to save as you go!
- Click "Go Live" in the bottom status bar to open your sketch in a browser. If you don’t see “Go Live”, click on the port number (something like Port : 5500) to dispose of the local server on your computer, and then start a new one by clicking “Go Live”. A browser window should open with your project.
- Return to VS Code to change code. Save the file before reloading the browser window to see the updates.
- Open your browser’s Console to see any error messages:
- Chrome: View > Developer > JavaScript Console
- Firefox: Tools > Browser Tools > Web Developer Tools > “Console” tab
- Related projects
- Doppelgänger by NUUM Collective
- Online Live Performance (OLP) by Yuguang Zhang and Nuntinee Tansrisakul
- Coding Traing Videos on video photobooths, pixel mirrors, painting with pixels, and slit-scan video
- p5VideoKit - a dashboard for mixing video in the browser by John Henry Thompson
- Flickering Existence by Liu Chang
- Interactive Art Works by Danny Rozin
- Más Que la Cara by Zach Lieberman
- Aspire Mirror by Joy Buolamwini
- Maya Man PoseNet Sketchbook
- Text Rain by Romy Achituv & Camille Utterback
- Body, Movement, Language: AI Sketches With Bill T. Jones
- Pose estimation examples listed on TensorFlow.org (scroll down)
- Charlie Gerard Quaruntine and Beat pose
- Vibert Thio ITP Web Olympic
- Richard Yee Touch Type: Interactive Typography
- Casher's Drawing with Body
- Finger Talk by Chelsi Cocking, Lingdong Huang, and Zach Lieberman
- AI ‘Emotion Recognition’ can’t be Trusted by James Vincent
- No worksheet this week!
- CREATE • BODY
- The idea this week is to center an experience around the body--yours, someone else's, or a specific audience. Choose the tools and techniques that make the most sense for the concept, keeping in mind that body point estimation extends choices for interactive design beyond the mouse and keyboard, which prioritize the hands and fingers for communication and expression in a very specific way. Some questions to start brainstorming:
- What are the private, public, and performative aspects of bodies?
- How are bodies imagined, idealized, reflected, altered?
- What is the personal and cultural significance of masks for ritual, to explore identity, hide, entertain, frighten...
- In whay ways can you use the body for expressive input? Consider choices for the part(s), position, direction of movement, and speed.
- Submit on our homework form.
- The idea this week is to center an experience around the body--yours, someone else's, or a specific audience. Choose the tools and techniques that make the most sense for the concept, keeping in mind that body point estimation extends choices for interactive design beyond the mouse and keyboard, which prioritize the hands and fingers for communication and expression in a very specific way. Some questions to start brainstorming:
- WATCH • Coding Train Videos 17.1 - 17.3, 17.5 - 17.8 on sound (code) ~90min
- Share our homework
- Mic input
- Sound output (samples and oscillators)
- Making notes
- Timing and patterns
- Related projects
- Syllabus Resources
- Media resources
- Record or generate your own!
- Orchestral Sample Library
- Another Orchestral Sample Library
- Free Music Archive
- Freesound
- BBC Sound Effects
- Sound of Picture
- Blackhole for routing computer sound
- Supported media type and formats
- Audacity for audio editing
- media.io for converting media files
- p5 examples
- Mic input
- Sound output
- Making notes
- Timing and patterns
- Related projects
- NYTimes Magazine Sonic Voyages (New York: Laughing rats!) browser version*
- Can Brown Noise Turn Off Your Brain?*
- The Handbook for Sonic Happiness
- jazz.computer
- Plink by DinahMoe
- Patatap and Typatone
- Writing Music in Light
- Music Eclipticalis by Brian Foo
- Troupe-like in p5.js by Cassie Tarakajian
- ...based on Trope app by Brian Eno and Peter Chilvers
- Symphonies of the Planets by Joohyun Park
- Chrome Music Lab
- PoseNet-related Sidewalk Orchestra by Cristobal Valenzuela, Code Music by Tero Parviainen, Body Beat by Vibert Thio, Semi-Conductor, and Body Synth
- *Free NYTimes subscription via NYU
- PRACTICE • Week 10 Worksheet
- COMPOSE • Create a 30-60s sound composition or generative music system. Option to work with partner. No visuals. Just sound. Some ideas to get started:
- Design a melody using this process and figure out an algorithmic way to generate it. Or, How Generative Music Works by Tero Parviainen.
- Record bits of spoken word and loop them to create music. See "Record and loop" example above. Why repeating words sound like music to your brain.
- Use sound samples and manipulate their playback rate to control pitch instead of the oscillator.
- Try implementing a different scale with different pitch ratios. More about scales and Music Theory: The TL;DR Version 2022.
- Play with Timbre and make use of p5 Sound's post-processing features: Delay, Filter, Reverb, Convolver, etc.
- Maybe the composition tells a symphonic story with several characters.
- Compose a sonic piece that is not music.
- Submit on our homework form.
- READ
- Allison Parrish's tutorial on Text and Type
- Review these syllabus examples and tutorials for Text and Data. We'll cover a couple topics, but you might want to go further with the materials posted here.
- WATCH
- Coding Train videos 17.9-17.11 on sound analysis (code) ~36min
- Listen to homework
- Data
- of Sound
- of Text
- as JSON
- from API
- Final project overview
- Sound
- p5 examples
- Related projects
- Text
- Syllabus resources
- Font resources
- p5 examples
- Array vs string
- Dancing characters
- Random story starters
- Draw with lines from a poem
- Book word and keyword count
- Carrie Wang's Scrolling Text and Drawing the Text Array
- Alllison Parrish's Interactive Type Examples
- Generative Design P.3. Type Examples
- Related projects
- Data & APIs
- Syllabus resources
- JSON resources
- Working with JSON
- JSON Formatter Chrome Extension
- Formatting and Validating JSON
- Coding Train Intro to Data and APIs (includes working with Wordnik, NYTimes, Giphy, and Wikipedia APIs)
- p5 examples
- Data sources
- Related projects
- Shirley Wu
- Jason Snell
- The Library of Missing Datasets by Mimi Onuoha
- Listen to Wikipedia by Stephen LaPorte and Mahmoud Hashemi
- See, also: A collection of our favorite visualizations built on Wikipedia data, curated by Hatnote
- Numbers Series: performance and sound installations by Mendi + Keith Obadike, related article: Reading the Numbers of Stop-and-Frisk
- We Need Us by Julie Freeman
- The True Colors of America’s Political Spectrum Are Gray and Green*
- Dear Data by Giorgia Lupi and Stefanie Posavec
- Sleep Drawings by Laurie Frick
- Visual Essays by The Pudding
- Heartmonic (heart+harmonic) and Eunoia (Version 2) by Lisa Park
- Engaging the Data Science Community with Met Open Access API
- 45 Ways to Communicate Two Quantities
- *Free NYTimes subscription via NYU
- DESCRIPTION
- Your final project is an open-ended creative project that builds off or is inspired by the concepts in this class. It is also an opportunity to push your abilities to produce something that utilizes what you have learned. The final project includes:
- presenting a proposal to our class in Week 12,
- user testing a prototype of the project with our class in Week 13,
- and documenting your project in a blog post (or Notion, Google Doc, etc) including your code contributions to the project, when the project is shared during final presentations in Week 14.
- There is no requirement to use a particular aspect of programming120+30. Focus on an idea that excites you and choose the best programming concepts and tools to help you realize it.
- You can take something you've already made and develop it further or create something entirely new.
- Final projects can be one part of a larger project integrated with a different class.
- Final projects can be collaborations with anyone in any class. Group projects are welcome and encouraged, and it is expected that everyone contribute their own code to the project.
- It’s okay to keep things simple and small in scope. If your project idea is a big one, consider documenting the larger idea but implementing just a small piece of it.
- Also consider making a final project for a small audience such as our class or for a single person like a family member, friend, or yourself as a way to focus your idea and design process.
- Your final project is an open-ended creative project that builds off or is inspired by the concepts in this class. It is also an opportunity to push your abilities to produce something that utilizes what you have learned. The final project includes:
- EXAMPLES from ITP/IMA/IMA Low Res Spring Show 2020 | Winter Show 2020 | Spring Show 2021
- FINAL PROJECT PROPOSAL
- PART 1 • Research your project and collect some notes, such as:
- Inspirations • How did you become interested in this idea? quotes, photographs, products, projects, people, music, political events, social ills
- Source material • Drawings, images, videos, sounds, text
- Code • Your own sketches and/or other people's sketches
- Ideas for title • And maybe a 1-sentence description?
- Context • What is the story? Who's it for? How will people experience it? Is it interactive? Is it practical? Is it for fun? Is it emotional? Is it to provoke something?
- Questions for your classmates • What are you unsure of, conceptually and technically? On what aspect(s) would you like feedback?
- PART 2 • Add two (2) to three (3) slides to this deck to share your proposal. Consider using the list above as an outline to illustrate what’s in your head(s), e.g. include some keywords, images, or embed video, etc. Everyone will have ~4-5 minutes each to share in our next class.
- PART 1 • Research your project and collect some notes, such as:
- Final project proposals
- Discuss final project user testing
- FINAL PROJECT PROTOTYPE
- PART 1 • Create a working prototype of your final project to user test in class next week.
- If your project is screen-based, be prepared to share a live sketch, full-screen if possible.
- If your project involves some type of physical interaction, ideally bring it to class or show a short video (~2min)
- PART 2 • Consider the kind of feedback that will be most useful for your project's development. Prepare some open-ended questions for your testers that focus on specific aspects of your project that you’re unsure of or want to know more about, e.g. “What did you take away from this project?” instead of “Do you get that my project is about XYZ?”
- Submit on our homework form.
- PART 1 • Create a working prototype of your final project to user test in class next week.
- User testing
- Discuss final project documentation and presentations
- FINAL PROJECT
- PART 1 • Complete your final project.
- PART 2 • Write your final project documentation in the form of a blog post (or Notion, Google Doc, etc). It's up to you to figure out how to best document your project. Here are some guidelines if you aren't sure what to include:
- Title and a brief written description
- Visual documentation such as a sketch running online, images, video, etc.
- References such as links to related projects, code samples, etc.
- What were some of the successes and challenges? What did you learn?
- How do you feel about WHY you want to use code in your work now compared with the beginning of the semester?
- If you are struggling with your project and can't get things to work, you should feel free to put your energy into writing about what didn't work (and vent any frustrations).
- Next steps to further develop the project.
- If the project is a collaboration, each person needs to contribute code and submit a post documenting their contributions., to the project.
- Be sure to cite the sources of any material / code you use with the exception of examples provided by the instructors or demonstrated for the course.
- PART 3 • Prepare a ~4-5 minute presentation to demonstrate what your project does that emphasizes its computational aspects. Feel free to include any questions to direct feedback after your presentation.
- If your project is interactive, be prepared to share a live sketch online. Aim to share a project running full-screen.
- If your project involves physical interaction, as a backup, prepare a short video (~2min) of the experience.
- Submit on our homework form.
- Welcome and general notes
- Final project presentations
- Course evaluation