Vague Terrain

[Daniel Shiffman / Arrivals and Departures / 2007]

Daniel Shiffman works as a researcher, teacher, and sometimes artist at the Interactive Telecommunications Program (ITP) at NYU’s Tisch School of the Arts. His digital art explores emergence and interactivity and has been widely exhibited. Jeremy Rotsztain recently interviewed Daniel about art and programming on behalf of Vague Terrain.

Vague Terrain: Your background is in mathematics. How did you make the transition into computer science and graphics?

Daniel Shiffman: Mostly by accident. As an undergraduate, I studied mathematics and philosophy, focusing on logic and paradoxes. After that I moved to NY and bumbled around from job to job. I worked in the theatre, for a couple music companies, and moonlighted as a freelance tech consultant (tutoring people on how to set up home networks, etc.) At some point, I wandered into ITP and thought it would be a good place to explore creativity with technology. There (or I should say here?) I discovered how all that logic I had studied as an undergrad fed into programming and I can't seem to stop.

VT: In some ways, it's akin to an addiction. I personally love the logical problem solving involved in programming. If I ever make myself an "I'd rather be programming" t-shirt, I'll make a second one for you.

DS: Maybe we could sell these t-shirts and make millions!

VT: I was just watching Golan Levin's video from the TED conference. He begins his presentation by stating how commercial applications are a "great homogenizing force". Does the act of programming have any political connotations for you?

DS: Interesting, I hadn't thought of what I do as "political" in nature. However, ever since I graduated from ITP in 2003, I have spent the majority of my time thinking about how to teach programming as well as build open-source programming tools and libraries. So for me what’s important is finding ways of helping people who thought they could never write their own software, who might have felt beholden to behemoth software companies, the chance to break molds and discover new forms by writing their own code.

VT: I definitely see it as political: learning how to program is learning how to communicate (in this day and age). It can also lead to a better understanding of biases in software and has its own DIY connotations.

What have you discovered about teaching programming? I see your job as rather challenging, as you're working with students with no prior technical experience and others who have computer science degrees within the same class. How do you keep humbled beginners enthusiastic about their practice when the attitude is often "oh, I can just find somebody else to program this for me"?

DS: This is definitely one of the great challenges of teaching programming at ITP. But it's also what makes it so much fun and interesting. Students with comp-sci experience have a lot to offer the class in terms of helping others and pushing the material forward. However, those of us who program all the time can easily get stuck in a rut, using the same techniques and ideas over and over again (I have, like, 12 projects that implement flocking rules). Beginner students can offer a fresh perspective and ask totally unexpected questions that can really take a class in new directions.

In terms of teaching, it's a very delicate balance. Let's say I have an hour of material. I like to make sure that there is 1/2 hour of extremely detailed, step-by-step instruction, from a very basic and beginner perspective. After that, the second half-hour might move more quickly through advanced examples, discussing concepts in greater detail, but leaving out the specifics of implementation steps required. I’m not sure if I really succeed at this, but that's the idea.

VT: MIT Media Lab graduate Jared Schiffman started off his class at ITP by telling students that copying code would be considered plagiarism and grounds for academic punishment. Clearly, he wanted them to learn through doing it themselves instead of copying. What do you think of his approach?

DS: Certainly, copying code without proper citation is plagiarism. You can’t use someone else's code and claim it as your own, turning it in for an assignment, publishing it in a paper, or selling it in commercial software. However, when learning with an open-source programming environment like Processing, in my view, there's nothing wrong at all with using freely available examples in your work (with proper citation). That's what all the code I put out online is for, after all!

The more interesting question here is "What is the best way to learn?" Should I write all my code from scratch? Should I copy examples and try to make changes to those examples? Personally, I like the cop-out answer. Do both! You definitely need to feel comfortable writing code from scratch, and you should practice doing so. For a beginner, it's even useful to simply re-type code (rather than copy/paste) from an example and practice making and correcting syntax errors, just getting used to the feeling of writing code.

However, take the Craig Reynolds boids example (back to flocking again). I don't expect students to recreate this example entirely from scratch, at least not initially. In order to learn and understand how the algorithm works, I suggest that students write code that changes certain parameters in the system (such as the weights for rules: separation, alignment, cohesion). By playing with these weights, the students will hopefully gain a deeper understanding of the system and how it works.

When it comes to my process however, I like to write everything from scratch. For me, this is the fun of programming, understanding and knowing how every single little bitty line of code works.

VT: When you create visual treatments for your work, do you give much consideration to old techniques of image production or think more about the development of a new visual language? How do you see your work in a history of painting?

DS: I do think about old techniques quite a bit. My projects develop as experiments with an arbitrary end point. However, they need to start from somewhere. Old techniques from painting, photography and sculpture tend to serve as this starting point for me. I spent sometime looking into simulating Jackson Pollock's action paintings, and the end result was the piece Swarm. For Timeframe, I was looking at the photographer Edward Muybridge's motion studies and wondered, "Could I recreate this effect in motion?"

VT: At the same time, there's so much more to it than that - the algorithms play just as important a role in the piece than the inspiration. Audiences respond more favourably to a work that simulates a famous artist like Jackson Pollock than one that uses Craig Reynolds’ Boids algorithm. Interestingly enough, Pollock's work was done just before scientists and mathematicians explored chaos theory and many simulations of his work were made using chaos simulations.

Your work comes from a lot of experimentation and the outcome is always something very poetic (in its movement and colour). You must spend a tremendous amount of time tweaking and modifying your work in order to make it ready.

DS: Yes, this is definitely true. If you ask any of my students, they know that I have some obsessive tendencies with code. For example, I can barely stand to look at code that isn't indented properly. It tears at the bottom of my soul. With my work, I tend to make stuff very quickly, but then spend large amounts of time tweaking and fine-tuning. With the project I made for the Toronto airport, I had the basic system up running in a day or two, but fine-tuned for several weeks. Every little force variable or Perlin noise incrementation or angle of rotation was changed maybe hundreds of times. In these cases, it’s often useful to build an interface for yourself to dynamically tweak aspects of a project while it is running. I think I could do a better job of this in the future.

VT: What role and consideration do you give to colour in the development of your work?

DS: It's funny. I think I try to avoid thinking about colour too much. This perhaps comes from my sheepishness as a designer. I prefer not to make too many decisions, letting my algorithms make those decisions for me. Since most of the projects I've created use imagery from a video camera, I let the real world give colour to the work. Whatever the camera sees, those are the colours I use.

VT: How important is it for you that your work involve user interaction and play?

DS: I have to admit, in the past, I've started without considering interaction. I become obsessed with solving the puzzle that is whatever algorithm I'm working on. However, I recently installed some of my work in Savannah, Georgia (at the Jepson Center for the Arts) in a part of the museum designed for kids. It's been tremendous fun to see the wonder and enjoyment of kids as they jump and dance in front of the camera, I'd like to try to keep this quality in my work.

VT: I don't think that's a sin. One challenge that still exists for me is how to create a deeper level of interaction and engagement for these screen-based works. Golan Levin's idea that his systems are "instantly playable and infinitely masterable" is still very exciting to me.

DS: Yes, that's a terrific quote. I think I often err on the "instantly playable" side and aspire to make things that are "infinitely masterable".

VT: Recently, you've been working on a number of different libraries for Processing. Do you see this as a natural translation from software programmer to open source developer? How does teaching fit into this for you?

DS: Aha, yes! Actually, this has really been my focus for the last year. I really enjoy building libraries and tools for other programmers and watching these bits of code propagate out into the world. The problem solving aspect of programming is what attracts me the most and when I solve a problem, I enjoy sharing that solution. For me, the transition is natural and it fits into teaching perfectly. My "Nature of Code" class, for example, teaches how to do everything that lives in the first three artworks I created (Swarm, Mosaic, and Reactive).  When I became interested in creating some text-themed works, I developed a class called "Programming from A to Z". My own projects are sitting unfinished on the shelf, unfortunately, but the code from the class is finding its way into student projects, which is great.

VT: For each of your classes, you make websites that are incredibly useful, and they're used by a larger community. Did being around ITP encourage an open-source mentality? And do you get a chance to see how your ideas make it out into the world?

DS: Yes, the spirit of ITP is democratization of media and tools for making media. It's an environment of play and collaboration. I think being there as a student, a researcher, and a teacher has encouraged an open-source mentality and a desire to share code and ideas. The web sites began out of neuroses. When I first started teaching, I couldn't fathom what I could possibly talk about for two hours, so I would prepare as much as possible and put the material online.

When I was in Hong Kong two summers ago teaching a workshop on Processing, I met several students who were using some of my examples in some projects. That was really fun and I hope to have more chances to travel and see where some of the code ends up!

VT: This past summer, when Red Burns was discussing the appeal of working on big screens, she spoke of the need for narration to create meaningful work. Your biography states that you're involved in a theatre company. Have you ever integrated narrativity into your work?

DS: Honestly, I haven't. I came to ITP originally with the assumption that this would be my focus (theatre + computers = fun!), but my experiments have led me elsewhere. In following the work of the students in the "Big Screens" class, however, I am seeing the extraordinary potential of bringing narrative and story to a "big" public space screen in new ways.  I can't wait to see the projects at the end of the semester and perhaps I'll be inspired to move in that direction.

VT: Red also asked what more there was to working on large displays than the "wow factor". Is that criticism a concern for you?

DS: Well, I am only now beginning to grow out of the "wow" stage. It’s important not to deny the wow factor, it's part of the process. Wow, look at that! Wow, that screen is giant! Wow, that looks cool! When developing the work, we have to realize that people first experiencing the screen will have these feelings and we should feel free to embrace that. Nevertheless, for me, what is exciting about the screen is not the size itself, but what we get from its size. We have the opportunity to create a shared experience (think of watching a movie in a theatre versus on your iPod) that is interactive.  We also have a screen here where in order to see what is happening on one part, we have to physically walk from one end of the room to the other.  How many displays can you say that about? This idea of a large-scale, digital display integrated into a building’s architecture provides an opportunity for ITP students to experiment with and develop new forms, whether they be narrative, interactive, video, algorithmic, etc. That's the real "wow" factor.

VT: The concept for your project for Pearson International Airport reminds me of the ideas presented in the Living Skins paper, that screen design and custom content in architectural spaces could be used to give a better understanding of the functionality and behavior of the space. Can you talk a little bit about how you came up with that idea?

DS: I think it's fun to play with people's expectations when they look at their computer screen. I had this idea for a project which would take whatever you have on your desktop and rearrange it as a mosaic of your image (as seen by a video camera). So you would walk up to your computer, which would look just like it did on any normal day, but as you moved towards it, it would react, respond, and mimic you. So when the project for the Toronto Airport rolled around, I thought I could try a similar technique. Looking at an arrivals or departures screen in the airport is such a ubiquitous experience, we are numb to it. The idea of playing with a traveller's expectation seemed like something fun to try. I hope to have a chance to play with this technique with other screens we are used to seeing in our daily lives.

Please visit shiffman.net for more information on Daniel Shiffman's work.