Information Arts, Critical Making

Golan Levin
School of Art, School of Design, School of Computer Science Carnegie Mellon University

Good afternoon, I am truly honored to be here at Maloej #22 in Pamplona. In my talk today I will discuss information graphics and information visualization, with a focus on some possibly unexpected people who are making it, and some possibly unexpected contexts in which we might encounter it.

Information Arts, Critical Making

Golan Levin
School of Art, School of Design, School of Computer Science Carnegie Mellon University

In particular, I will show some insights from how artists are using principles from information visualization to yield new insights, and I will discuss some disruptive new contexts for visualization -- outside of the familiar frame of the newspaper page. I will show a few works of my own, and many works by others, including a few by my students at Carnegie Mellon University.

As a warmup I would like you to consider this very large picture. It is a logarithmic map of the universe, produced by the Princeton physicists, Richard Gott and Mario Juric (http://www.astro.princeton.edu/universe/). It shows the known universe all the way back to the Big Bang. It is a picture of everything we know.

And here is (artist) Tom Friedmans cheeky Everything: a sheet of paper on which he has painstakingly written every word in the dictionary. It took him three years to make. It too is a picture of everything we can imagine. The arts and sciences may di!erent truths to reveal. And both are good.

So when creating visualizations, this triangle, which I learned from Martin Wattenberg, delineates a possible scope of objectives -- beautiful, interesting, useful -- of what we might hope to achieve with information graphics. Martin jokes that we are usually fortunate if we can achieve two out of three.

($)

Most of the worlds attention, it seems to me, is focused on the apex of this triangle: it is where the money is, after all. In my talk today, and in my work generally, however, I focus on the bottom axis, which is much less trammeled. I submit that there are many more undiscovered possibilities there :)

I consider information visualization as a kind of self-examination. We get a new mirror, and we can then look at ourselves from a new angle for the rst time -- and say, Oh! I didnt know I looked like that. This could be at the level of a single person, or it could be of all society reecting on itself. Often these analyses begin with a simple question, which acts as a lter on all possible information. Ill give some examples.

This strange image is Blindspot by the artist Tim Hawkinson. It is a completely inverse self-portrait, which answers the question, what are all of the parts of his body, that he himself cannot see? Hawkinson sets about the typical artistic project of a self-portrait... but he quickly turns it into a research investigation of terra incognita. It is as if he says, I already know what I look like in a mirror, and therefore, thats uninteresting to pursue further. And I can already see most of my own body directly. All... except.. here.

Here is a closer view. He cannot see the top of his own head, nor his back, etcetera. [http://www.acegallery.net/ artwork.php?pageNum_ACE=54&Artist=1]

Here is another inverse self-portrait by Tim Hawkinson. He describes this sculpture as all of the dentalimpression compound my mouth could contain, cast in lead. You can see the material beginning to slip down the back of his throat as it solidied. The positive space reveals the limits of the negative. It is a tool (and display) for coming to know his own limits.

Here are Lipsticks by New York photographer Stacy Greene, in a small-multiple series that dates to the late 1980s. She documented the lipsticks owned, and unconsciously sculpted, by her friends. Although there is no quantitative data here, these objects o!er richly multidimensional qualitative information about their owners: hints about their complexion, their fastidiousness, and the 6-dimensional rotations of a repetitive gesture in space. [http://www.stacygreene.com/lipstick.html]

Kim Dingle asked Las Vegas teenagers to draw the United States from memory. She then compiled these into a single painting, which reveals as much about the underlying ur-gestalt of the United States, As it does about its citizens lack of geographic knowledge. [http://www.kimdingle.net/united-shapes-of-america.html]

We can also use ourselves as an apparatus for developing a representation of the physical world around us. Our otherwise subjective experience acquires intersubjective validity - again through proceduralization schemes like small multiples. Here are drawings made by Meaghan Kombol, which document her attempts to draw straight lines while riding various subway lines in New York City. Despite its simple mode of collection, this is true data. [From the book, Speck: A Curious Collection of Uncommon Things by Peter Buchanan-Smith.]

Nancy Burson is known for having invented the technique of morphing while working at MIT in the 1970s. In this crowdsourcing artwork, she placed an advertisement asking for guys who look like Jesus. She then created a composite image, at lower right, by computationally averaging their faces. Her premise is that we can use the law of averages to compute the appearance of Jesus -- or perhaps more accurately, the appearance of Jesus in our popular imagination. We all gasp: Wow -- that one really looks like Jesus! ...a person whom nobody has actually ever seen. [http://nancyburson.com/guys-who-look-like-jesuswomen-who-look-like-mary/]

Heather Dewey-Hagborg
Stranger Visions, 2012-

Id now like to show you some astonishing visualizations by an artist who works in very unusual new ways. You may have heard of hackerspaces: places with 3D printers and so forth. Well, Heather Dewey-Hagborg is an artist who works in one of the worlds few DIY genetics laboratories - hackerspaces for synthetic biology. Shes producing visualizations of found genomes. [http://deweyhagborg.com/strangervisions/]

Dewey-Hagborg is concerned with the topic of genetic surveillance. And here is a typical starting point for her work... some discarded chewing gum. It contains the saliva of... who?

Here is another: some hair, inadvertently left in a public restroom. The thing is... these objects contain traces of the DNA of their owners. DNA that describes how they look.

In her project Stranger Visions, Dewey-Hagborg creates portrait sculptures from analyses of genetic material collected in public places. Working with the traces strangers unwittingly leave behind, she calls attention to the impulse toward genetic determinism and the potential for a culture of genetic surveillance.

She extracts the DNA from these discarded fragments, and sequences the genome. She then looks for more than 60 genetic markers, or SNPS, which are known to contain all sorts of descriptors -- for example, for whether the person has blue or brown eyes, a big nose, wide forehead, etcetera. (These SNPs are used by police, incidentally, when creating proles of criminal suspects.) In Heathers case, she then alters a computer model of a face according to the extracted properties, and 3D prints a colored model.

The resulting works are surprisingly accurate portraits. Heres a video. [http://deweyhagborg.com/strangervisions/about.html]

Sometimes, new ways of understanding a collection can be achieved simply through re-arrangement of an existing set. This could mean sorting or grouping it in an unexpected way. Here is SkyMall Liberation by Evan Roth. It is an interrogation of advertising culture: a visualization of who matters, and who matters less. Male and female faces are extracted from an in-ight shopping magazine, and sorted: men on the left, women on the right. [http://www.evan-roth.com/work/skymall-liberation/]

Here: white, and non-white faces from Skymall have been sorted right and left. I particularly like this visualization because it was made completely without technology. Being on an airplane, Evan didnt even have a pair of scissors!

The artist Fred Wilson also uses reorganization of databases as a operation to create meaning. He works by reorganizing the collections of museums, revealing their ideological agendas. For example, here, he collides a collection of metal objects which are all from Baltimore in the 1780s: silver vessels and slave shackles. Although they come from the same place and time, ordinarily these objects do not appear in the same museum case. Hmm... [http://beautifultrouble.org/case/mining-the-museum/]

Visualization can sometimes be accomplished through a process of removal or ltering, rather than accumulation. Rutherford Chang has blacked out everything but the faces from the front page of the New York Times --

...and presented as a 30-day calendar, the result tells exclusively of the people who made the news over the course of a month. It is a new view of the news. (How would your newspapers look, transformed like this?) [http:// joseebienvenugallery.com/exhibitions/2005-02-03_dating-data/]

This project, Most Wanted Paintings by the artists Komar & Melamid, produces an unexpected form of visualization from survey data. They used internet survey forms and professional telephone surveys to collect data from thousands of people, about what sorts of paintings they liked best. [http://awp.diaart.org/km/index.php/ intro.html]

They asked all sorts of questions, such as, do you prefer outdoor or indoor scenes?, do you prefer city or country scenes?, etcetera.

And they asked questions like, what is your favorite color? and what size painting do you like best?

They then synthesized this data into a single painting that met all of the preferred criteria. It is a holistic visualization of visual art preferences in the Unites States.

They also created the Least Wanted Painting. It is small, orange, and full of pointy triangles.

And they did this for ten di!erent countries. The Dutch are the only group that preferred abstract art, for example, while the Italians dont prefer paintings with classical or contemporary icons.

Id now like to show a project of mine, an older project in fact, which dates to 1997 and which I completed in 2002. It concerns the popularity of search terms, as indicated by the number of results which is always returned with every search. Here you can see that there are 84 million pages that contain the number 910.

Its a little odd that this single number is the only feedback which search engines provide about the quality of ones search. But its a good indicator of what we might call the popularity of a search term. Here, we see that some 69 million pages contain the number 912. (Incidentally: Google wont actually allow you to view all 69 million links; they cut you o! after the rst thousand.)

Here is the same search for the number 911, and we can see that theres signicantly more pages that contain this number. There are plenty of reasons for this: September 11th; the American emergency phone number; the Porsche 911 car, etcetera. We can see that the number 911 plays a more signicant role in our culture than either of its neighbors, 910 or 912.

So I wrote a program, The Secret Lives of Numbers [1997-2002] to scrape and visualize the popularity of every integer from 1 to 1 million. This simple query has fascinating results, revealing all sorts of information about our biology, history, and culture. Here you can see the popularity of the number 911 and its neighbors. On the right side, these results are organized into rows of 100. [http://www.turbulence.org/Works/nums/]

Heres the popularity of the numbers from 1 to 2000 or so. You can see regular spikes on every hundred, and the broad spike of numbers peaking at 2002 -- the year this screenshot was made. On zooming in, we can see that we literally care twice as much about 1970 as we do about 1930. The slope of that spike is our rate of forgetting: the extent to which we cease to care about the past.

Two student projects

Jesse Kriss, Sampling History (2005) Shan Huang, Favicon History (2014)

Ill now show two projects by former and current students, who work in areas like computational interactive information visualization.

This is a website called the-breaks.com, screenshot circa 2005. Its an early crowd-sourcing site, in which everyday people contribute information about which songs are sampled by which other songs. For example, the song Back in Black by AC/DC was sampled by both the Beastie Boys and Eminem.

My former student Jesse Kriss scraped The-Breaks for his project, History of Sampling. Here Jesse has visualized, on two parallel timelines, albums which have been musically sampled (on the bottom), and albums which contain sampled fragments of those (on the top). You can see the 20-year lag. This is not merely a histogram; in the interactive version, you can see how a given song connects to the songs which sample it, and vice-versa. [http:// stu!.jklabs.net/post/7284206698/the-history-of-sampling-is-a-visualization-i]

More recently, my student Shan Huang visualized her browsing history as a list of favicons. She released this tool as a free Chrome Plug-in, that anyone can use.

[http://shan-huang.com/browserdatavis/]

So it is now a well-known story that, over the past 30 years there has been a dramatic consolidation of news providers. Whereas there used to be hundreds of independent voices, more than 90% of todays media are owned by just 6 companies. I am concerned about the way in which this homogenizes --

... the diversity of perspectives available to us.

For this reason, Im interested in infographic messages which - though perhaps no less biased - originate from outside of corporate and governmental messaging structures. I want to know: what happens when Tufte comes HERE? [Photo: Darren Barefoot]

Im interested in exploring the impact which infographics can obtain when they are freed from the predictable frame of the business section, into the realm of the commons: When information visualization meets gra"ti. Evan Roth has said, When I look at gra"ti artists, I see people who are making their own tools and subverting systems to tell stories. These stories are not sanctioned, and this kind of unapologetic subversion is often neither pretty nor legal.

To be clear, Im not merely talking about the look of infoviz - i.e., the expressive or evocative use of visual elements that resemble information graphics. Certainly, this is something people do: whether with sparklines,

...or with maps, for example.

No. At the very least, Im interested in the potential for graphics whose content is tightly coupled to the location where it is deployed. Let us call this situated infographics, or in-situ visualization. Here is a poetic work by Jose Duarte, for example, which documents how rain dissolves chalk over a half-hour period.

My fellow Maloej juror, Scott Klein, snapped this photo on the upper west side in New York. A familiar spatial unit -- a common sidewalk square -- is transformed into an index of the citys skyrocketing rents. [This slide was added subsequent to the presentation in Pamplona :)]

Here is a project, Painting Reality in Berlins Rosenthaler Platz by Aziz Ikon. Lets watch this video for a moment. (The paint being dumped by the bicyclists is water-soluble and environmentally degradable.) [ https:// www.youtube.com/watch?v=tXCnWUzUw_E ]

The nal result is a perfect visualization of tra"c in that locale.

Another image some time later. This may be the largest Sankey diagram ever created.

Beyond this, I want to explore the possibility of combining the visual language of data visualization, with the communication strategies of urban markup. Why dont we see infographics in public space? These two languages dont usually collide, but they should. Messages of protest should be backed by data. This is especially important in an era when people have such a weak grasp of numbers that matter. So here Ill present some new works in the area of critical making, or what is sometimes called tactical media, which achieve this combination. [These typologies above are from Ben Fry and Richard Brett.]

Heres a small example, found in the wild. This bar chart suggests that the US military exists to protect our access to countries with large oil reserves, and that the viewer -- a young African-American, it is suggested -will be sent o! by the military to protect our interests there. [Photo: Jerakeen / Tom Insam]

Of course, spray paint is just one of several e!ective media for infographic markup. Another is wheatpasting, as used in the kits produced by infobombing.org.

Heres another example.

But public space is not the only kind of commons we all share; there are others. This project, created by an anonymous artist collective, occupies US currency with rubber-stamped infographics, in order to widely disseminate information about the problem of income inequality -- one of the most signicant social issues of our time. [http://occupygeorge.com/]

Its called Occupy George, and it reaches people precisely where and when they are thinking about money. At the top, you can see how the richest 400 people in the USA earn the same as the poorest 150 million. The bottom graphic is a comparison of typical worker pay (the small red patch in the upper left of the bill) to CEO pay.

At the bottom, you can see income inequality in the 1920s, the 1960s, and in the 2000s. So here is this kind of infographic gra"ti, which overlays the portable architecture of our economy. The stamp designs are freely available online.

Next I will show a few projects which I've made and released through an international group of professional pranksters called the Free Art and Technology (F.A.T.) Lab, which is dedicated to the enrichment of the public domain through the development of open-source technologies and media. The motto of our group is: release early, often, & with with rap music. We hope to provide people with open-source tools that allow them to engage directly as creative actors.

All of the projects Ill show have been made possible through new modes of digital fabrication, such as lasercutting. This project is an adjustable pie-chart stencil -- a laser-cut pie chart machine, which can be easily recongured and reused for di!erent messages. [http://!!f.at/infoviz-gra"ti-stencil/]

Everything is cut from a single sheet of material. Here is the template for lasercutting: a free, downloadable design. It contains a complete set of letter tiles, for making nearly any message.

The only other necessary item is a nut and bolt.

Heres an example of a possible design. The letters are held in place with Scotch tape.

This tool can be used to create situated information visualizations, in which the visualization relates, in a thought-provoking way, to the architectural program of the location where it is deployed. Here, it is deployed on the side of a universitys power plant building.

Here is another laser-cut visualization instrument released through the F.A.T. Lab: a set of stencils, called 5-Star Gra"ti, created by my student Madeleine Varner. She asked questions like: How do I know which bathroom stall is super-gross? Does this water fountain taste bad? Will I get splashed if I stand at this bus stop? Which park bench has the best view? Which tree is the best for naps? [http://!!f.at/5-star-gra"ti/]

Her system of free, downloadable stencils is designed for in-situ rating visualizations on a classic 5-star scale. Its particularly good at adapting the visual language of Yelp to all sorts of public services and natural phenomena, such as billboards, park benches, drinking fountains, skate parks, access ramps, playground equipment, and trees.

[A close-up of one of Varners 5-Star Gra"ti stencils.]

In the next examples Ill show infoviz gra"ti systems made possible through custom software.

Weve always been confronted with corporate messaging in cities. But theyve recently taken a particularly dehumanizing turn -- intended for reading by machines. There seems no place for everyday people to make their voices heard in this, amidst the robot barf of QR codes.

In addressing this problem, I was inspired by hobo codes, which were widely used by vagabonds and migrant workers in the United States in the early 20th century. These are a form of civic markup intended to guide vulnerable people, warning them about danger and clueing them into good situations. These codes indicate things like Talk religion, get food here, work available etc. And theyre still in use today; these on the right are from sometime in the past decade in New York City.

Love them or hate them, I wanted to put large-scale QR codes into the hands of everyday people. Now, stencils are a fast way that an everyday person can make highly accurate images at any scale. But QR codes present unique technical problems for stencils, because of the islands of isolated material that pop up in unpredictable places.

So I developed some free software that makes it easy for everyday people to create their own QR code stencils. My software uses simple image processing techniques to automatically bridge the islands that would otherwise make stencils so tricky to design and fabricate.

This software is released with a set of 100 free QR HOBO CODES, a set of urban, covert-markup tools for digital nomads. These ready-to-cut designs port a number of classic hobo annotations to the QR format ("turn right here", "dangerous dog", "food for work") as well as some new ones, inspired by warchalking, that are specic to contemporary conditions: "insecure wi", "hidden cameras", "vegans beware"... [http://!!f.at/qr-stenciler-andqr-hobo-codes/]

The next project Ill show blends 3D printing and urban markup. But before I begin, I have to clarify, since its possible that some of you may not be aware, that its recently become possible to 3D-print rubber, as shown in the image above.

I was prompted by my 5-year-old son, who observed that he was able to make his own form of urban markup with his sneakers. He asked me if I could use my 3D printer to make shoe soles with his name.

My sons request seemed as good a motivation as any to develop a new piece of free software, developed in collaboration with Randy Sarafan and my student Jordan Parsons, which generates 3D models for shoe soles, rubber stamps, toy wheels and car tires.

The result is another system for civic markup. Heres the current state.

People have been using rolling printers to make messages in public for some time; for example, this one is from the 1895 Paris worlds fair.

(Heres a cross-section of the bicycle tire.)

This is the Gra"ti-Writer (2000) by the Institute for Applied Autonomy, an artist collective. This small robot printed phrases that were controllable via text message.

Similarly, Joshua Kinberg deployed this protest bike at the 2004 Republican National Convention. He painted facts and gures, as well as messages submitted through his web site.

Banksy has remarked that if gra"ti changed anything, it would be illegal. And of course there are large forces lining up to prevent it. Germany is now spending hundreds of thousands of euros on anti-gra"ti drones.

They look like this.

But it will be an endless game of cat-and-mouse. [Gra"ti drone developed at Ban! by Cameron Macleod.]

Some Civic Markup Projects

Sandy Claes et al., Street Infographics (2013) Andrew van de Moere et al., Neighbourhood Scorecards (2010) John Bird et al., the Tidy Street Project (2011)

I dont necessarily wish to leave you with the impression that I am a proponent of wanton destruction or befouling of the commons. Rather, the opposite: I feel that public spaces have already been befouled -- against our will and without our consent -- by advertisers and propaganda. For this reason Id like to conclude with three civicallyminded urban visualization projects, by others, which orchestrate close collaborations with everyday people in order to educate and actively improve various communities.

Sandy Claes is a graduate student of Andrew van de Moeres in Leuven, Belgium. In her street infographics, she designed a provocative augmentation to existing street signs with socially- and locally-relevant information. [http://sandy.deect.be/?portfolio=street-infographics]

The project presented facts about the demographics of the neighborhood: for example, the proportion of students, and the proportion of persons not from Belgium. The visualization encourages people to gain local knowledge, reect on their assumptions about the neighborhood and even foster social interaction.

The project brings up a lot of questions. How does one know this data is current? In calling out the proportion of local residents who are foreigners, does this project unwittingly serve a (right-wing) nationalist political agenda, singling out a vulnerable minority? Nonetheless, Claes demonstrated that citizens pay attention to such augmentations, and nd them provocative in several respects.

In this 2010 project by Andrew van de Moere, Neighbourhood Scoreboards, ve chalkboards as feedback displays were installed in a Sydney neighborhood and manually updated each day. The visualization included the change of electricity consumption compared to the previous day, and associated rankings. [http:// neighbourhoodscoreboards.com/]

Quantitative measurements showed that households that received a public display decreased their energy usage on average by 2% more per week compared to a control group with no feedback. The presence of the display also led to a more sustained conservation behavior.

Related to this is the Tidy Street Project by John Bird [et al.]. The neighborhoods cumulative energy consumption was plotted at regular intervals on a timeline painted on the street. [http://collabcubed.com/2011/11/01/thetidy-street-project/]

The artists wrote that the neighbors learned which appliances used the most electricity, as well as a general awareness on how to lower their consumption. The result was a 15% usage reduction in a 2-month period.

Thank You!
@golan