Está en la página 1de 5

Algorithm Visualization: A Report on the State of the Field

Clifford A. Shaffer Matthew Cooper Stephen H. Edwards
Department of Computer Department of Computer Department of Computer
Science Science Science
Virginia Tech Virginia Tech Virginia Tech
Blacksburg, VA 24061 Blacksburg, VA 24061 Blacksburg, VA 24061

ABSTRACT Baeker was the first well-known visualization, ad hoc visu-
We present our findings on the state of the field of algo- alizations existed long before. The first recognized system
rithm visualization, based on extensive search and analysis for creating algorithm animations was BALSA [2] in 1984.
of links to hundreds of visualizations. We seek to answer Since then, hundreds of algorithm visualizations have been
questions such as how content is distributed among topics, implemented and provided freely to educators, and scores
who created algorithm visualizations and when, the overall (or hundreds) of papers have been written about them.
quality of available visualizations, and how visualizations It is widely perceived that algorithm visualizations can
are disseminated. We have built a wiki that currently cat- provide a powerful alternative to static written presentations
alogs over 350 algorithm visualizations, contains the begin- (from textbooks) or verbal descriptions supported by illus-
nings of an annotated bibliography on algorithm visualiza- trations (from lectures). There has been some debate in the
tion literature, and provides information about researchers literature as to whether algorithm visualizations are effective
and projects. Unfortunately, we found that most existing in practice. Some studies have shown the classic dismissal
algorithm visualizations are of low quality, and the content that is the downfall of most technological interventions in
coverage is skewed heavily toward easier topics. There are education: “no significant difference” [6, 9, 11]. Other stud-
no effective repositories or organized collections of algorithm ies have shown that algorithm visualizations can indeed im-
visualizations currently available. Thus, the field appears in prove understanding of the fundamental data structures and
need of improvement in dissemination of materials, inform- algorithms that are part of a traditional computer science
ing potential developers about what is needed, and propa- curriculum [13, 3, 7]. Certainly, many visualizations exist
gating known best practices for creating new visualizations. and are widely (and freely) available via the Internet. Un-
fortunately, the vast majority of those currently available
serve no useful pedagogical purpose.
Categories and Subject Descriptors So we see that (a) many algorithm visualizations exist,
E.1 [Data Structures]; E.2 [Data Storage Represen- yet relatively few are of true value, and (b) algorithm visu-
tations]; K.3.2 [Computers and Education]: Computer alizations can be demonstrated to have pedagogical value,
and Information Science Education yet it is also quite possible to use them in ways that have no
pedagogical effect. These facts seem to imply that creating
General Terms and deploying effective algorithm visualizations is difficult.
There is a small body of literature that investigates how to
Algorithms, Measurement, Design
create pedagogically useful algorithm visualizations (for ex-
ample, [10, 15]). Yet, there is still much to be done before we
Keywords are at the point where good quality visualizations on most
Data Structure and Algorithm Visualizations, Algorithm topics of interest are widely available.
Animation, Courseware The purpose of this paper is to provide a summary of
preliminary findings resulting from our efforts to survey the
1. INTRODUCTION state of the field of algorithm visualization. To bound the
content area, we focused our attention on topics commonly
Data structure and algorithm visualizations and anima-
taught in undergraduate courses on data structures and al-
tions (hereafter referred to generically as algorithm visu-
gorithms. We seek an understanding of the overall health
alizations) have a long history in computer science educa-
of the field, and present a number of open research ques-
tion. While the 1981 video “Sorting out Sorting” by Ronald
tions that we and others can work on in the future. Some
examples of the questions we seek to address are:
• What visualizations are available?
Permission to make digital or hard copies of all or part of this work for • What is their general quality?
personal or classroom use is granted without fee provided that copies are • Is there adequate coverage of the major topic areas
not made or distributed for profit or commercial advantage and that copies covered in data structures and algorithms courses?
bear this notice and the full citation on the first page. To copy otherwise, to • How do educators find effective visualizations?
republish, to post on servers or to redistribute to lists, requires prior specific • Is the field active, and improving?
permission and/or a fee.
SIGCSE’07, March 7–10, 2007, Covington, Kentucky, USA. • Is there adequate infrastructure for storing and dis-
Copyright 2007 ACM 1-59593-361-1/07/0003 ...$5.00. seminating visualizations?


and most These might have each been created by the same individual likely we have captured the vast majority of the visualiza. or they might have been de- actively collecting new links. our wiki) attempt to link to collections of visualizations that bled across a page that had links to collections of visualiza. We are still collecting visualizations. the Hope College We have collected links to over 350 visualizations. We catalog such systems in addition to cataloging kits have been implemented in Java. sure that our data collection process does not focus unfairly ternet searches (this is discussed further in Section 3). but a signifi. it does serve as a representative sample of the 20 14 13 12 total population of visualizations accessible from the Inter. Many visu- pages we already had.6 What is the content distribution? applications that must be downloaded and opened locally. on our topic list. 100 The results can be found at our Data Structure and Algo- 80 rithm Visualization Wiki [18]. since they need not go through dynamic programming. While it is by no means 40 29 complete. Many collection [8] in 2001). we have made a significant effort to 120 catalog as many existing algorithm animations as we could. We considered what search terms collections that we find (as described in Section 2. Perhaps 50% of visualizations are provided 2. We have restricted our study to topics that are typically These numbers are somewhat biased. visualizations. 60 veloped the most extensive collection of links to algorithm 60 46 38 visualizations currently available. a noticeable minority are Java 2. a single Java haps 10-20% of algorithm visualizations are essentially sin- applet that embodies separate visualizations for both stacks gle efforts by their respective authors. the site managers have considered worthy. While this concerns mostly lower division materi- to do (as explained in Section 3). categories for grouping visualizations. We do a significant amount rent population counts. However.2 How many are available? sive catalogs of visualizations (for example. Sometimes we could status and effect of various courseware repositories on the find these other visualizations from direct references on the algorithm visualization community in Section 3. and string matching. organized repositories. Once we had generated a base 2. we would follow those links to capture any new ones of these link sites are small in scale. Visualizations that are als. algorithms for N P-complete problems. We began with a list of all visualization systems that we were aware of from our general knowledge of the Figure 1: Histogram of major categories in Table 1 field. 4 0 net.4 How are they disseminated? of visualization links. There have been a small number of efforts to produce comprehen- 2. once for each provided by “small shops” that have created typically 5-15 distinct visualization. We speculate that we have so far cap. 151 .2. along with their cur- ization or visualization system. we have de. most tions. individual applications that include 5-20 2. veloped by a small number of students working for a given faculty member. we also considered some upper division topics like com- directly available in web pages will typically get more at. of available visualizations are provided as applets directly from web pages. In this time. we then examined these pages to try Almost none of the visualizations that we have found are to locate other visualizations. WHAT’S OUT THERE? 140 134 Since Spring 2006. system). and many of the Compression Algs Sorting Algorithms Graph Algorithms Other Algorithms Search Algorithms Memory Management Search Structures Linear Structures Other Data Structures following assessments are based on preliminary data. and other times we could deconstruct alizations are cataloged by link sites. Whenever we stum. There is a tendency covered in undergraduate courses in data structures and al- for us to search for applets. mostly by hand as individual Java applets. If a given program veloped a picture of who is providing visualizations. 2. tention from potential users. only effort to do this that is currently active. We are still is teaching relevant courses). but our own wiki appears to be the of these are individual applets or programs.5 Who makes them? distinct visualizations. Based on individual applets. or toolkits that distribute a collection While these numbers are quite preliminary. we have de- of 5-20 distinct visualizations as a unit). over some number of years (typically a faculty member who tions that are easily found and better known.1 How did we find them? We used a number of techniques to locate algorithm visu- alizations.3 How are they implemented? by teams that have also created visualization systems of one Since Java’s introduction in the mid-1990s. virtually all sort or another (and the visualizations are related to the algorithm visualizations and algorithm visualization tool. meaning sites that (like the URLs to find more visualizations. tured roughly half of what is publicly available. However. available from large. We developed a topic list based on our experiences teaching relevant courses. We discuss the sualization often have others available. Perhaps 30-40% are and queues). putational geometry.1) to in- would be most productive for locating visualizations via In. are shown in Table 1. favorite visualizations for some course or textbook. since this turns out to be easier gorithms. Probably over half the individual visualizations. Our top-level the additional step of downloading and unpacking a visual. perhaps linking to 20 not yet in our collection. we then performed searches using Google to find whatever we could. since developers of a given vi. cant fraction appear as parts of integrated visualization col- lections (typically. then we count it multiple times. of cross checking by capturing links from visualization link As Figure 1 shows. Per- contains multiple visualizations (for example. there is wide variation in coverage.

These counts are “by project” rather than by individ- algorithms. with no explanation of how the result was obtained. particularly as stacks and queues. which are popular subjects for visualiza- Network Flow 3 tion.e.. Similar deficiencies occur in many Other Algorithms 28 visualizations for simple structures such as lists. as opposed to a recent rise in activity. Linear Structures 46 9 8 8 Lists 13 8 7 Stacks & Queues 32 7 Search Structures 60 6 Binary Search Trees 13 5 5 5 5 AVL Trees 7 4 4 Splay Trees 9 3 Red-Black Trees 8 3 B-Trees and variants 10 2 1 Skiplist 3 1 0 0 Other Data Structures 14 0 Heap/Priority Queue 8 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 Search Algorithms 12 Figure 2: Last-change dates for sorting visualiza- Linear/Binary Search 2 tions. In contrast. most specialized and advanced previously. algorithms. or for self-study of a topic. Shortest Paths 11 A different type of deficiency often occurs with visualizations Spanning Trees 9 of tree structures. the better numbers at the end visualizations. Sorting is an important topic for undergraduate ual visualization. Counts are by project. many of the sorting visualizations are variations on We can see that. Ongoing projects appear no more attuned value. of the histogram merely indicate that there still exist active projects. Since we are measuring the “last change” date topics are poorly covered. and significant sub- operating systems. Further. Only the outcome of an insertion or deletion is dis- N P-complete Problems 3 played. 2. When we say “have no pedagogical are sadly lacking. users are relegated to being passive observers Graph Algorithms 38 with no control over pacing (aside from animation speed). Over a dates for the sorting visualizations currently in our collec- third of all visualizations that we have found are on sorting tion. However. or appear to have tion of either topic coverage or content quality. how the insert or delete Memory Management 4 operation actually works. most of these are now no longer available. Even the better vi- Mergesort 28 sualizations tend to have serious deficiencies.8 When were they made? Mathematical Algorithms 4 A significant number of systems were developed in the early 1990s for creating algorithm animations and visualiza- Table 1: Major categories for visualizations and tions.” we mean that they give the user no understanding to the topical gaps in coverage. were readily available to create visualizations during the sualized” works. either as a lecture Insertion Sort 18 aid. Java). or the operations being conducted. we have observed a decline Nearly 15% of all visualizations are on linear structures such over time in the creation of new visualizations. these will show the tree that results from a Compression Algorithms 13 user-selected insert or delete operation. in that if a given visualization or visual- data structures and algorithms courses. But rarely do they Huffman Coding 10 illustrate at all. Selection Sort 18 Another quarter of visualizations fall in between: they are Quicksort 23 potentially useful but severely limited. stacks and Computational Geometry 6 queues. Figure 2 shows a histogram of the last-change less difficulty to students than many other topics. but it is certainly ization system provided visualizations for multiple sorting being given too much attention by creators of visualizations. If we consider the development of visualizations since the mid 1990s (i. the recent decline in the creation of A majority of the visualizations that we have encountered new visualizations certainly cannot be explained by satura- either cannot be made to work easily. Hashing 10 Sorting Algorithms 134 Only a small fraction (certainly less than a quarter) would Sorting Overviews 6 we feel comfortable recommending for use. and so will be of little use in the classroom. education. overall the classic “Sorting out Sorting” video. while there are still active projects. Java boom of the mid to late 1990s. Algorithmic Techniques 7 String Matching 8 2. Each or so difficult to install and run due to changes in computer major category shows its total. even though these probably present after 2002. since both no pedagogical value. and just show bars activity does not appear to be as extensive today as it was being swapped. We speculate that students whatsoever of how the data structure or algorithm being “vi. Typically. For Radix Sort 10 many of these. as the basis for a lab exercise. Traversals 10 the data being processed. then we only count it once in the histogram. when Java and applets 152 . There is certainly room for new for the various visualizations.7 What is their quality? As discussed above. their counts for 352 visualizations collected. Roughly half Heapsort 9 of all algorithm visualizations are actually animations. let alone effectively. that they are not currently a factor in categories are also shown.

By browsing a tion. the instructor “lists” or “queues” is likely to return information unrelated might come upon different treatments and approaches to computer science. and two these make up a significant fraction of the total (they are with permanent redirection pages (indicating that our links still written in Java. need to be updated in the wiki or eventually we will lose The main alternative to keyword-based Internet search is them). everyday names. Some keywords. available on October 31. loss of access. ate and maintain specialized repositories or link collections and none have large collections of visualizations. and the CITIDEL repository [4]. Neither ware repository. 3. turnover of material. sired (visualizations as opposed to explanations). for at least four reasons. In that case. “applet” is often a successful choice of keyword. The Internet is well known for high terials or tutorials) or projects or exercises (assignments). however). this 2. or a curated link collection of which you supports browsing for courseware separate from the (over- might be aware. since non-applets were 46. collections. and the topic or content area desired. ing explicitly for them. we have word also results in generating a self-fulfilling prophecy that been tracking the status of the visualization links on our if you search for applets. not a single visualization within any of sults. the balance of the materials found in our wiki away from 2006. But now search terms will not find them. The other materials are papers. None of tors to find it. or Thus. But we have since made no longer on the Internet. 2006. we way of good repositories for data structures and algorithm had 219 unique working links. “animations” often do not label them as either. whether any given instructor will be able to presence” with respect to algorithm visualization. a course- topic (CITIDEL) or publication venue (ACM DL). While the ACM DL and There are essentially two distinct ways to go about find- CITIDEL are both huge collections. On July 31. structor who is searching for good ideas. Further. and visualizations in particular. and Java is considered old hat. 15 were no longer visualizations. we can hope number of visualizations [16]. In all the find an existing artifact depends a lot on the instructor’s hours that we have spent conducting google searches for var- ability to supply the right keywords to yield successful re- ious visualizations. and they tend to organize by content is to look in an algorithm visualization collection. The nation-wide drop typical visualization. Since it turns out that the vast major- in computer science enrollments might also mean that there ity of visualizations written since the mid 1990s were written are fewer students available who want to do such projects. Each week. For example. none of the repositories mentioned here have much “web Unfortunately. since they are not typically presented dividuals. To get some measure of this. Within the computer science community. If so. Like any Internet search. Of these. Nor is there any standard there are other “in things” competing for students’ atten. Both keywords are necessary for searches on most topics. Far more artifacts exist • Visualizations maintained by individuals are prone to that can be described as content presentations (lecture ma. then the only presentation mech- wiki. One is to large amounts of courseware in general or visualizations in “google for it”—that is. ture the desired artifacts. keywords need both to cap- these repositories was discovered by that means.were new and the “in thing” for students to learn. In other words. looking for “Huffman Trees” is likely to list of visualizations and courseware for lower-division give results related to the data structure. in Java. Unfortunately. so these Only curated repositories hosted by stable providers 153 . alternative synonym that we know of that can capture the tion. algorithm visualizations have a high turnover rate to the world labeled as “applets. while techni. use your favorite Internet search particular. some sort of restrictive because the material is no longer made available. to find a given visualization. either because URLs change. She might even discover visu- Visualizations constitute only a small part of the course.9 Will we find them again? will tend to leave out those visualizations that exist within a Like everything on the Internet that is maintained by in. keywords must identify both the type of material de- tions. Of those projects with integrated applications. three which returned “page not a conscious effort to catalog non-applets. alizations that encourage her to present new topics. visualization system. over a span of three months. There- perior to a basic Internet search when looking for visualiza- fore. and Connexions [5]. while looking for data structures and algorithms topics. some data structures than she might have found by listing topics and search- have common. neither provides good search engine to search for whatever keywords you believe will best tools for courseware or visualizations. might lead to a wealth of non-related informa. for courseware in general. the most likely candidates are the collection of materials submitted to JERIC [12]. SIGCSE main- Let us assume that there exists on the Internet a suitable tains a collection of courseware links which includes a small visualization on a specific topic.” Using “applet” as a key- in their accessibility. 6 out to look in visualization or courseware repositories or link of 46 links were either lost to us or moved. Equally important. and many of these are delivered as applets. Unfortunately. we had 46 active (working) links in our wiki. • Internet-based keyword searches do not help the in- cally specific. One was sub. ware materials available on the web. keyword-based search of the Internet and collections fortunately. 2006. The bulk of their find a useful visualization on the topic you need. and found that found” (two of these three were at the same site). we take a snapshot of how many links on anism you will find will be applets. whelming) body of non-courseware content. Unfortunately. Broader courseware reposi- that standard Internet search technology will allow educa- tories include SMETE [17]. this might alleviate the need to cre- these are well known within the CS education community. As of May 29. this skewed any of our wiki pages are still accessible. Initially. and to avoid burying the user in Courseware repositories have the potential to be far su- overwhelming numbers of false-positive responses. there currently is little in the sequently found at a new location. after three months we had one whose host machine was intrinsically harder for us to find. SEARCHING FOR VISUALIZATIONS JERIC and its contributed courseware are indexed as part of the ACM Digital Library [1]. the providers of algorithm “visualizations” and of visualization links are susceptible to this problem. neither appears to have ing an algorithm visualization on a given topic. Un.

2006. pages 21–28. T. 2000. 1996. S. Hansen. Stasko. H. or view an expert’s? In Proceedings. IEEE has begun to form regarding how to develop effective visu- Symposium on Visual Languages. http:// one repository moving in this direction. T. Some Symposium on Computer Science Education. 2000. [6] J. REFERENCES • A large fraction of existing visualizations are we also simply need more implementors to produce Journal of Visual Languages and Computing. resource that can provide value to the community. C. http://www. and Effective features of algorithm visualizations. By allowing interactive digital educational library. 154 . cal foundations for creating effective visualizations appear to [9] C. N. allow us and the CSE community to data mine our within the past ten years. If we can enlist significant community help. Visualization Research Group. H. New York. 2002.cs. McCrickard. and J.acm. Feldman. and cross-link or they might choose which visualizations to provide. B. CONCLUSIONS AND FUTURE PLANS [7] S. can make progress toward the day when computer science http://web-cat. a [15] P. 1994. and R. IEEE Symposium on Visual Languages. Yet.icio. [3] M. 2006. and ratings. http://www. they are poorly conceived and implemented. [1] Association for Computing Machinery.vt. alizations in the first place.hope. Shaffer. izations are available. Stasko. Gurka and W. NY. or user and tools for capturing public comments topics. Testing effectiveness of algorithm animation. collecting them. W. We Online Teaching. Lawrence. using social bookmarking schemes. Complete collection of algorithm terms of creating new visualizations now than at any time essary. 2. a Computing. New York. Schrimpsher. 2006. North. or a curated site of visualization links. pages 177–186. educators would be able to add value to the entire col- lection over time just by using it. Byrne. Assessing to provide visualizations on under-represented topics. In Proceedings. pages 382–386. www. pages 182–189. we algorithm visualization wiki. http:// users to comment on. the benefits of interactive prediction using web-based Given the existence of effective visualizations. http://www. supply ratings information. Citrin. March 2004. pages 48–54. In these ways. and a body of work own. ∼dershem/ccaa/ccaa. D. and [18] Virginia Tech Data Structures and Algorithm thereby better pinpoint what is available and what is lack. And we need to encourage them [11] Douglas.merlot. 2006. Much of the data reported here SIGCSE ’04: Proceedings of the 35th SIGCSE was hand generated. J. could thm animation. there is still algorithm animation courseware. IEEE Symposium on Visual Languages Our own future plans include developing our wiki into a 1994. are developing a bibliography of the research literature. A on how to develop and use algorithm visualizations is nec. 5. Thus. Sedgewick. While more fundamental research [10] C. Do • A repository or curated site that developed its web algorithm animations aid learning? Technical Report presence could allow educators to actively participate GIT-GVU-96-18. 2006. IEEE Computer Society. 2006. In SIGGRAPH ’84: Proceedings of the provide editorial guidance to educators regarding the 11th Annual Conference on Computer Graphics and quality of the visualizations. and A. More papers are appearing regarding to learn algorithms: should students construct their effective use of visualization in courses. NY.cs. like Amazon. del. While many good algorithm visualizations are available. D. the need for more and higher quality visualizations contin. The ACM digi- either because they simply do not work. educators have a rich collection of high-quality visualizations that span all the traditional topics that we teach. In a directory of researchers. ACM Press. A system for algori- itory. Jarc. and J. we [17] SMETE digital library. Catrambone. Such sites might pick and Interactive Techniques. Data structures and ing. Hundhausen and S. for example). In Proceedings. S. Saraiya. A repos. there seems to be less activity in [8] Hope College. rate. or because tal library. [2] M. Narayanan.smete. major concern for computer science educators should be how [13] A. We con. ACM Press. 2006. A. catalog of projects and algorithm visualization systems. but they need to mature before they can claim [12] JERIC: Journal on Educational Resources in to provide an adequate solution to this problem. Stasko. On the other hand. R. [14] Multimedia Educational Resource for Learning and tinue to expand our collection of links to visualizations. Helping learners visualize and comprehend algorithms. and D. Douglas. We are creating data collection tools to Technical Symposium on Computer Science Education. more quality visualizations. 377–381. or fea- [4] CITIDEL: Computing and information technology tures of other social networking tools. 2006. We have developed evaluation forms and a mechanism for [16] SIGCSE educational links. There are many topics for which no satisfactory 1984. M. Empirically the gains made in knowledge and artifacts can be dissemi. meta-study of algorithm visualization effectiveness. J. http://www. can give any assurance for long-term sustainability. both in terms of quantity and quality. evaluating the use of animations to teach algorithms. in the site. In SIGCSE ’00: a major gap in terms of a credible national or international Proceedings of the Thirty-First SIGCSE Technical repository for visualizations and other courseware. VA. Brown and R.citdel. Computer-Enhanced Learning. 2006. Collectively. Badre. like 1996.acm. the theoreti. Using visualizations be steadily improving. nated effectively so as to insure continued improvements. R. and C. Hundhausen. Norfolk. http://sigcse. http://portal. pages repositories exist now (CITIDEL and the JERIC/ACM DL. Heller. Georgia Institute of Technology. the community is learning how to cnx. 2000. 4. Interactive Multimedia Electronic Journal of ues. plan to provide evaluations of existing visualizations. MERLOT [14] is [5] Connexions scholarly content repository.