This action might not be possible to undo. Are you sure you want to continue?
Knight Foundation Final Report
Organization name: Tides Center Report due: November 30, 2009 Grant amount: $249,529 (over two years) Grant #: 2008-0142 Grant date: October 27, 2008
Project description: To create and launch two social media applications on Facebook to test strategies that leverage social media environments to engage youth in news and information.
Please report your project outcomes clearly, briefly and honestly. Include setbacks as well as successes. Knowing why something didn’t work can be as helpful as knowing why something worked. In describing your relationship with Knight Foundation, please be candid and provide constructive criticism whenever appropriate. Honest feedback can help both of us improve our organizations. To get started, please read the following anticipated outcomes paragraphs carefully. The anticipated outcomes sum up what we were expecting from this grant.
Anticipated Outcomes: The two Facebook publications will create communities of interest that engage young adults with news and information. The project will track the evolution of those communities to assess the impact of content types, outreach efforts, and marketing strategies in order to inform news organization about their relative impact and effectiveness.
1. Please list each project activity and tell us, if and when you achieved it?
Note: The project began for our team December 1, 2008. We received our first funding via Tides Center on December 10, 2008.
Activity 1: During a six-month period, Newscloud will launch two different social media publications on Facebook, each with a distinct strategic approach to promoting participation, community and awareness with young people using social networks over a six-month period Achieved launch within four months (December 2009 – March 2009) o The first publication will be a Climate Change site, which uses a “street team” incentive system to promote and encourage participation in the community. Members will receive points when they post stories, rate materials and participate in discussions. They can accumulate points over time and redeem them for small prizes. Launched in February 2009. o The second will use demographic and geographic targeting for a publication tailored for college students in the Minnesota region. Launched in March 2009. Activity 2: Study the results and impact of each particular strategy and publication. Achieved research goals and produced three research reports on each social media publication (i.e., one interim progress update, an executive summary and full report). These can be viewed at our Research page http://newscloud.com/research Activity 3: Publish qualitative and quantitative research from the study with the goal of documenting strategies that work best to engage youth in new media in social networking environments. Achieved publication and dissemination goals during the grant period (December 2008November 2009); research reports were published within self-publishing avenues; we published about the project in Harvard’s Nieman report, and in conference presentations and papers (see list in section 14 below). Peer-reviewed journal publication is a slower process. Typically, manuscripts take 2-4 months to prepare for submission and 8-12 months for actual journal publication. Therefore, we might expect to see published manuscripts from this project in fall 2010 at the earliest. Activity 4: If successful, a secondary phase of study will examine two to three additional publications.
Not Achieved: The Knight Foundation chose not to fund additional publications and research during H2 of the grant period. However, Mr. Reifman worked privately with a University of Washington
Communications school class to teach in the classroom and help them launch their own publication (based on the software funded by the grant), In:Site, on Arts & Culture topics. A University of Washington graduate student published a synopsis of this project available here http://blog.newscloud.com/2009/10/university-of-washington-analysis-of-newsclouds-facebookapplication-insite-and-related-curriculum-.html
2. Were there any major changes in the project activities and timetable? What caused them?
Technology Activities and Timetable There were no significant changes to the technology activities and timetable. Research Activities and Timetable We adhered fairly closely to our original timetable for research activities (within 2-3 months of the original plan). The following factors caused our adjustments to the timetable: (1) Hiring for the research team took much longer than expected. Although the first graduate assistant was hired within two weeks of the project’s start date (hired 12/15 with project start date of 12/1/09), the second graduate assistant was not hired until 2.5 months into the project because of the unique skill set we required (and had great difficulty finding) and because we had to re-open the search when a chosen candidate did not accept the offer. (2) We were challenged in our ability to contribute to both design and research efforts simultaneously. That is, we needed to contribute to the development of the social media publication’s research interface (i.e., thinking up front about all the data we wanted versus needed and in what format) while also having to think through and design the theoretical framework, methods, instruments, and communication forms and all other aspects typically required of a non-development-oriented research project. (3) Soliciting advice and approval from others took longer than expected. As with all good research, we followed human subjects procedures and sought advice from our advisors at key steps in the research process, but this also slowed our progress when feedback took longer than expected. (4) Working within the academic calendar meant delays in the implementation of some of our data collection instruments. Although we had initially planned to gather all data from both social media publications in March and April, we hadn’t appropriately budgeted for the lifecycle of the academic year, adjusting for times when students are especially hard to reach (e.g., spring break in March; end of April through May). We were able to collect some of our online data in March, April and May but data collection endured through the summer months with focus groups and interviews ending by late July. (5) Correspondingly, an extended period of data collection meant that data analysis and report write-up needed to extend beyond the July deadline (month 8 of the project) which we had originally targeted. Our research reports were issued on 9/3/09 for the Hot Dish publication and on 10/5/09 for The Daily publication, only 2 months beyond what we originally outlined.
3. Describe any set backs you encountered and how you addressed them?
Technology-related set backs Just prior to the launch of Hot Dish, our climate journalism partner Grist began insisting on a legal agreement and framework, which assigned liability for running the action team contest to Mr. Reifman and NewsCloud. Ultimately, NewsCloud kept ownership of the application, assumed liability and Grist became primarily an editorial and marketing partner. The original intent had been for Grist to own the application. The effectiveness of Grist’s e-mail and Website marketing was lackluster. While they told us they have e-mail list of 700,000 readers, we found the response rates from their efforts mediocre. Coordinating the timing of the Hot Dish promotion with them was difficult because Grist was very distracted by the upcoming redesign and re-launch of its own Website. The effect of this was that the sample size of the Hot Dish community, while good, did not fully meet our expectations. Completion of the research console that provided raw and computed data for the research team became bogged down in the spring. While some aspects of the console were outstanding (such as dynamic charts), the completed level of functionality for the console was not what we had hoped. This made some of the reporting tasks more difficult. In some specific cases, we had to provide data dumps manually to the research team. However, this also diverted development resources from doing as well on making the code base easily installable by open source partners. While we did the best we could to meet the critical requirements of the research team while wrapping up development in a timely and economical fashion, Mr. Reifman takes responsibility for not better managing the console development process. Changes to Facebook’s platform including the launch of its Open Stream API reduced the viral effects of both Hot Dish and The Daily applications. In the middle of the project, activities by application participants stopped publishing to their friends’ news feeds. You can read more about this on Mr. Reifman’s technology blog http://www.idealog.us/2009/06/facebook-stream-beta-challengeto-app-developers.html The complexity of the Facebook application permission system combined with the registration requirements for the use of human subjects in research by the University of Minnesota reduced the overall participation for Hot Dish and The Daily.
Research-related set backs (1) Smaller than expected sample size was a set back we encountered. We addressed this by adjusting the statistical tests we used. For instance, all of the analysis procedures used were appropriate for the sample size. However, in places where we wanted to examine the impact of demographic factors, we were limited in our ability to do so by small numbers within certain ethnic groups. This is a common issue among researchers. We also addressed this set back by relying more heavily on qualitative data to confirm/disconfirm and tease out trends seen in the quantitative analyses.
Factors that contributed to low sample size were the complexity of online ‘opt-into-study’ procedures; to be eligible for participation in our research study, users had to meet five criteria, including be 16-25 years of age; be a U.S. resident; have completed the registration form on the Facebook application; have authorized the Facebook application to access their profile; and have opted into the research study (i.e., university Institutional Review Board procedures require potential subjects to wade through a rather lengthy consent form online). Members of the application not in the study did not complete all of these steps. Factors contributing to the small sample size within The Daily application, in particular, were unforeseen leadership changes at The Minnesota Daily itself -- including the unexpected resignation of the Editor-in-Chief who originally championed the application, end-of-semester staff turnover, and a weeklong break at the term’s end. This turn of events lead to less than ideal staffing, promotion, and management of The Daily Facebook application. These factors, combined with a short, two-month window for data collection and at the end of the spring semester, surely contributed to the low sample size. A more longitudinal approach to data collection and analysis would allow further insights and confirmation/disconfirmation of the following themes. 4. Were there any surprises on the positive side? If so, please explain.
Quark Expeditions agreed to donate a trip for two to the Arctic as a grand prize for Hot Dish. This gave us a marquee prize at virtually no cost. It helped drive interest in Hot Dish and supplemented our grant prize budget. We also managed to recruit partners such as Equal Exchange and the Center for Biological Diversity to donate gift baskets, coupons and wildlife ringtones. Execution of the Hot Dish contest went well. The prize system was designed well with the leadership of Product Manager Zibby Wilder. In the end, there were few complaints and no legal challenges related to the contest. Dr. Greenhow did an excellent job building stakeholder support at the University of Minnesota to convince The Daily newspaper to work with us. A partnership with The Daily had always been a goal, but was not sure or promised in our grant application. Mr. Reifman winning Wired Magazine’s Vanish contest by creating a Facebook application, Vanish Team, to track writer Evan Ratliff. The Facebook application was based on the software funded by the grant. The resulting press attention from this has been outstanding, including appearance in Wired Magazine, Slashdot and Wired’s website linking to the NewsCloud blog (30,000 page views), a Skype interview by CBS News and an NPR Weekend Edition interview, among others. See http://wired.com/vanish Daily traffic to the NewsCloud blog has increased over time to regularly more than 100 – 300 page views per day. While some of this traffic arrives due to Wired’s Vanish, or Mr. Reifman’s writing on micropayments (SurfShare project), overall exposure to the grant-funded Facebook technology has increased. The NewsCloud Blog and NewsCloud on Twitter now have a moderate following.
Applying the grant-funded Facebook technology to hands-on classroom learning opportunities at the University of Washington was a great success. NewsCloud is also currently working with Hope College in Michigan and a Canadian nonprofit Genome Alberta. While national media exposure for the Hot Dish contest was less than expected, overall media exposure for the project in places that matter to the new media community (e.g. public radio, Poynter Online, Mashable, Nieman Labs) was fairly broad. See http://blog.newscloud.com/pressroom.html We also received some television coverage in North Carolina. A journalism student at Arizona State University was the grand prizewinner for Hot Dish. She won the trip to the Arctic but mechanical flight problems kept her from reaching Oslo in time to board the boat. This also prevented her from writing about the trip for Grist, which had been planned. The Charlotte Observer’s presentation of our Facebook application at Poynter’s Big Idea conference and their subsequent adoption, on their own, of the open source Facebook application. The Daily’s editor, Vadim Lavrusik, proved a regular advocate for our Facebook application technology and has continued to write and blog about our work on a regular basis. We completed the project nearly $10,000 under budget.
5. Please explain how you are meeting the overall goals stated in the anticipated outcomes. Overall goals We will launch and grow two social media publication communities: The first publication will be a Climate Change site which uses a “street team” incentive system to promote and encourage participation in the community. Members will receive points when they post stories, rate materials and participate in discussions. They can accumulate points over time and redeem them for small prizes. The second will use demographic and geographic targeting for a publication tailored for college students in the Minnesota region. As described in Section 1 above, we achieved the goals stated in the anticipated outcomes. The project is now complete aside from future conference presentations and journal publications. 6. How are you measuring your progress? Are those measurements working?
We were successful in measuring our progress all throughout the grant period. Our progress measurements included: (1) regular updates within the grant team, which were then summarized and distributed bi-weekly to Tides and the Knight Foundation; (2) checking these progress updates against originally proposed outcomes and deadlines; (3) conducting a research project with each social media publication developed to assess whether and how anticipated outcomes were met. Results of the research project were written up and disseminated via the following six reports: • Interim Report on Findings: Hot Dish • Executive Summary of Findings: Hot Dish • Full Report on Findings: Hot Dish • Interim Report on Findings: The Daily
Executive Summary of Findings: The Daily Full Report on Findings: The Daily
Research reports from the project are online at http://newscloud.com/research *The final reports were disseminated to the grant funders only, as this data is being written up for publication next year.
…Please attach copies of any evaluation reports, list results of any measurements, Web-traffic analysis, registered users. This information is all included within the reports we have previously issued for Hot Dish and The Daily (see attached).
7. If you were publicizing the single most important outcome of your work, what headline would you write for your news release?
The headline used for a recent press release was:
Could Facebook make America smarter? New U of M study finds social media engages young people in content better than traditional sites (released Nov. 16, 2009)
Moreover, we issued several press releases and attracted various media coverage along the course of this project. A list of project-related PR can be found here: http://blog.newscloud.com/pressroom.html The most important finding from an advocacy side was that Hot Dish was able to motivate young people to develop daily habits of news readership and to take action in the real world to stop climate change. I often refer people to Grist’s summary: “Facebook app translates online efforts into real-world environmental change” http://www.grist.org/article/2009-05-12facebook-efforts-real-change This kind of success is not typical for online advocacy campaigns. Michele McClellan also blogged this for the Knight Digital Media Center: “Hot Dish on Facebook engages young users with news” http://www.knightdigitalmediacenter.org/leadership_blog/comments/hot_dish_on_facebook_eng ages_young_users_with_news/
8. What did you do to market the project? Was it successful? What would you do differently next time?
Marketing Hot Dish
Through May 3, 2009, Hot Dish gave away more than $25,000 worth of eco-friendly prizes to eligible 16-25 year-old users who participated. Top point earners won eco-friendly prizes ranging from organic cotton T-shirts and gift certificates to an Amazon Kindle 2 and new “green” Apple MacBook computer. The grand prize was a trip for two to the Arctic, courtesy of leading polar exploration provider Quark Expeditions. To expand participation beyond the study
group, all participants received free downloadable endangered species ringtones from the Center for Biological Diversity and free shipping from fair trade coffee leader Equal Exchange
Working with the Knight Foundation, we generated as much free media as we could. We issued a national press release via PR Newswire. We advertised Hot Dish in print publications at selected four-year universities around the country. In the end, we found that advertising inside Facebook was the most effective to target our audience and drive traffic to Hot Dish. Early on, Mr. Reifman made the judgment to focus the prize budget on the 16-25 year old target age group for our research. While Product Manager Ms. Wilder supported making the prizes available to all readers of all ages (in the U.S.) In the end, it was probably a mistake to restrict prizes to the study group. The decision made it difficult to attract as large an audience to the application and to explain the eligibility rules of the contest. Marketing The Daily on Facebook
For The Daily competition, we spent approximately $2300 on prizes (water bottles, T-shirts and cinch bags with The Daily application logo) to be awarded to eligible 16- to 25-year-old users who participated. However, these prizes were not really put to use in competition until very late or after data collection had concluded. Staff at The Minnesota Daily generated the Grand Prize: two tickets to a Minnesota Twins baseball game.
We relied on The Daily to publicize the application in their print publication and their Website at MnDaily.com. We generated some free media coverage of The Daily Facebook application and purchased some Facebook advertising.
9. Demographics of staff?
The technology team consisted of three Caucasian males and two Caucasian females. The research team consisted of three Caucasian females.
10. Please provide your audited financial statements from your organization’s last fiscal year.
Not applicable. This grant was made to an individual, Jeff Reifman.
11. Please attach a statement showing the original budget for the Knight grant and the actual spending in each project category. Explain any significant changes from the budget.
See attached spreadsheet
12. Do you have a surplus from your Knight grant? If so, please tell us how much it is and explain why you have it. Please explain what you propose to do with the unspent funds, and indicate whether these are new or previously proposed activities.
Yes, approximately $10,000 savings will be applied to a subsequent grant made in November 2009 to continue development of the Facebook application technology and work with twelve news organizations to adapt it.
13. Who else funded this effort, and at what level? Was it necessary for you to make significant changes to the proposed project budget? If so, please explain and attach a copy of the revised budget.
There has been no additional funding of the effort. The primary change to the budget was to request additional funding ($10,400) for Dr. Greenhow. Because she was involved in the project from the proposal stage until the delivery of the final research we underestimated the budget for her time. Also, the data analysis and reporting stages of the project extended took much more of her time than we anticipated. Savings in other areas offset this. Mr. Reifman was hired by the University of Washington and recently Michigan’s Hope College to teach in the classroom (by phone) and set up Facebook applications to support their campus community and curriculum. A Canadian nonprofit Genome Alberta also hired Mr. Reifman to create a Facebook application for it.
14. Please describe your plans in detail to sustain the project long term.
Dissemination of project-related research
The project will be sustained and have long term impact via dissemination of our findings to appropriate professional groups and individuals. For instance, we have shared and plan to share insights from this project with other individuals in a number of knowledge-sharing venues: • • • • • • • • HASTAC conference in Chicago (April 2009), The Future of News and Civic Media Conference at MIT (June 2009) Social Media Research Conference in Minneapolis (September 2009) Institute for Advanced Studies, University of Minnesota (September 2009) Yale Information & Society Project (September 2009) Harvard-MIT-Yale Cyberscholars research forum (October 2009) Economic and Social Research Council Seminar Series, UK (January 2010) American Educational Research Association (April 2010)
Moreover, we’ve published an invited essay about this project in Fall 2009 issue of Harvard’s Neiman Report http://www.nieman.harvard.edu/reportsitem.aspx?id=101906 Research from this project is slated to appear in: Handbook of Literacy Research Methodologies. Duke, N. & Mallette, M. (Eds.). Guildford Press. (2011 publication to be used in 2011-2010 courses). Youth and Social Media: Trends in Education, Communication & Society (draft title only). Greenhow, C. (Ed.). MIT Press. (expected publication 2011)
Invited essay for the scholarly journal New Directions in Youth Development: Theory, Practice, and Research. (winter 2010 publication date) With additional manuscripts submitted to top peer-reviewed journals such as: Journal of Computermediated Communications, New Media & Society, First Monday, and Journal of Adolescent and Adult Literacy
15. Did you collaborate with other organizations, particularly Knight Foundation grantees, during the course of this project? How?
Yes, we launched a Facebook application for Knight grantee Chitown Daily News before it ceased operations. We also proposed an integration with grantee Populous at the Knight MIT Media Lab conference this year, which did not materialize. Michele McClellan of the Knight Digital Media Center also blogged about our Hot Dish findings, as mentioned above. We’ve kept in touch with her and she’s been a helpful advocate of our efforts.
16. Please describe your interaction with Knight Foundation staff. What was most useful and what changes would you suggest?
We have worked most closely with Gary Kebbel in developing the project. He was particularly helpful and responsive when we needed urgent input from the foundation. For example, when the Grist legal issues arose, he referred us to the Knight-funded Citizen Media Law Project. He also made time at various points in the project to schedule conference calls with Mr. Reifman and Dr. Greenhow to be briefed on our progress and our research findings. He routinely followed up on these calls by asking Knight-affiliated people to consider blogging about our work. He introduced us to Nieman Lab, which led to a journal article and increased the profile of Mr. Reifman and the NewsCloud blog with its news gathering efforts. I felt comfortable bringing both bad news (when necessary) and good to him. He showed a great deal of enthusiasm around our winning the Wired Vanish contest – and he followed up with Knight staff to highlight these efforts. When attention from this contest led to a hacker finding a vulnerability in our server, Gary was understanding and supportive. We greatly appreciate his efforts, encouragement and evangelism for our work. He has helped make us more successful than we would have been otherwise. We also worked closely with Jessica Goldfin. Jessica is one of the most positive and enthusiastic people we’ve encountered. Her enthusiasm and optimism is infectious. Because of the number of projects that the foundation must manage, Jessica’s accessibility makes a big difference to the process of the day-to-day challenges of implementing the goals of a foundation grant. Her attention to our project helped facilitate interfacing efficiently with other staff at the foundation. Our work would not have been possible without Gary and Jessica’s support – and even if we had received the grant without them (not likely), our work would not have been as good or as successful as it was with their help.
17. Was Knight Foundation able to facilitate contacts with experts in the field, professional peers and similar organizations?
This has been one of the most rewarding parts of being a grantee – the opportunities Knight has provided to interact both with other grantees, and to attend conferences (e.g. Knight MIT Media Lab Conference) where exposure to a wide range of topics and experts was possible. Most notably, Knight has helped connect us to: • • • Nieman Lab Citizen Media Law Project Knight Digital Media Center bloggers
In general, exposure through Knight Foundation press releases, blogs and twitter posts helped raise visibility for our work with other media. As a result of our work with Knight, Mr. Reifman’s individual efforts have been more widely covered such as his Micropayments essay (Slashdot, Nieman Lab) and SurfShare project.
18. What else would you like Knight Foundation trustees and staff to know about your experiences with this project?
Working with the Knight Foundation as an individual (as opposed to a nonprofit organization) poses a unique set of challenges. While nonprofit staff are on salary during grant approval cycles, individuals are unpaid. As a former Microsoft manager, I have personal savings to rely on during the cycles between grant application and approval. However, even for me, it’s a challenge to wait 3 to 6 months for a decision without income. During this time, I can’t seek other employment without risking my availability to work on the grant project I’ve proposed. I’d like to encourage the Knight Foundation to consider ideas that could make the process friendlier for individuals who propose projects. If it would be useful, I’d be glad to talk to foundation staff in more depth about this issue. We used a number of online tools, which helped facilitate the success of the project. It might be useful for the Knight Foundation to negotiate a subscription to some of these services that it could provide to all grantees free of charge. For example, we used: • Github for source code management and open source distribution • BasecampHQ for team communication (although this proved less popular over time) • eFax for incoming faxes and converting documents into PDFs for Tides Center et al. • LighthouseApp Ticket tracking, related application by the same company, TenderApp customer service is also available • Quickbooks (an online version is available, but we used the software package) • Google Docs (free)