Spring 2012

Synchronous	Asynchronous
Skype	Google Groups
Adobe Connect (UM Connect)	Ning

Environment	Technology
Make it Welcoming – Take some time to think through the way your online environment looks. Is it inviting? Is it user-friendly? You may want to film a brief introductory video to introduce the moderator(s), the purpose of the group, and the features of the online platform.Personalize it – Allow your participants to personalize their presence. Even in an anonymous group, you can have participants pick fun profile pictures, write a brief personal bio, answer a few introductory questions, upload a few personal photos, or even film a brief introductory video, if appropriate. There are several simple video tools available, such as: http://intervue.meSimple Layout and Design – Make it easy to find things on the site, build in some navigational redundancy, and avoid clutter. Make sure you only have on the site what is necessary for conducting the focus group.	Teach the technology – Participants will come with varying levels of technology expertise and anxiety, and it is important to get them more or less on the same page before starting the focus group.  Try creating a “how-to” guide or filming an introductory video that introduces the platform, and ask participants to look at it prior to joining the focus group.  Consider incorporating a “test run” or a “warm-up” activity at the beginning of the focus group to introduce and test out the features of your focus group platform.Stay behind the curve – Choose platforms and features that participants have the maximum amount of familiarity with.  Remember: focus groups are about getting rich information–not about demonstrating the latest technology.  If all goes well, the technology should be as transparent as possible.Keep the technology support “quiet” and omnipresent – Technology should remain as invisible as possible, in order to focus on participants’ voices and ideas. Build in multiple avenues for troubleshooting, and be explicit about how participants can request help if something breaks down.  For example, if a microphone or webcam isn’t working, encourage participants to ask for help via text chat; if participants are confused about how to engage in an asynchronous forum, consider holding “office hours” where moderators are available for live text chat support; if a participant’s computer breaks down altogether, have a phone number or e-mail address available to participants for “last ditch” tech support.
Participants	Moderator
Keep the group small – Somewhere around five participants is ideal.  This reduces the amount of reading for asynchronous text-based focus groups, and reduces the bandwidth and troubleshooting issues for synchronous voice/video focus groups.Know your audience – Make sure the platform you choose and the features you use will resonate with your participants.  There are some generational and even gendered patterns in the ways people prefer to engage with technology that may be helpful to consider when selecting a platform.  To get a sense of what you might reasonably expect from participants, you may want to look at the Pew Internet & American Life Project’s work on technology user types (see Pew Internet & American Life Project, 2009).Consider how you recruit – Recruiting for an online focus group can be very different than a face-to-face group. You have to consider motivations and ability. Do the people you are recruiting like communicating online? Or is communicating online the only way to participate in the group? When are good times to get together? What is their level of technology sophistication and will they have sufficient access to the technologies and bandwidth you require? Furnish the right incentives – Incentives help participants stay engaged throughout the online focus group. The incentive could be intangible (e.g. “You are helping the community.” or “This research project will help others in need.”) or tangible (e.g. a gift card from a popular store or a movie ticket).  If the incentive is intangible, be sure to describe the benefit. Don’t assume that it is obvious.	Establish expectations for engagement –  Social norms vary much more online than in a face-to-face environment.  Being explicit about how and how much you expect participants to engage will help avoid “culture clashes” and confusion.  In synchronous environments, you’ll want to spell out clear expectations of who should talk and when, or even try calling on people in turn. In asynchronous environments, you can communicate guidelines for how and how frequently participants should comment on each others’ contributions.  You can also encourage participants to use bullet points, bold key ideas, and give brief explanations to avoid rambling.Be socially present – As a moderator, it is important to appear present in the conversations–perhaps more so than in a face-to-face environment. Online environments can feel impersonal, so you may want to make particular effort to address participants by name. In synchronous environments, you may find you need to more actively moderate and “fill in the gaps” in conversation.  In asynchronous environments, you can create daily summaries and bullet points to highlight key ideas in the discussion and guide further discussion for participants who don’t have time to read all the posts.Have multiple moderators – Having two or more moderators is important.  You may want to divide roles into “talk-moderator” (to guide the discussion) and “tech-moderator” (to help with troubleshooting and tech questions).

GAPSA Proposal – Promoting Openness (Long Version)

The increasing need for a highly trained work force, together with the decreasing financial support for higher education by states governments across the country, encourages higher education institutions to search for new ways by which to break away from the iron triangle of higher education, or the difficult balance between cost, quality, and access to higher education (Immerwahr, Johnson, & Gasbarra, 2008; SHEEO, 2012). I believe that further “openness”, is one of the potential game changers to the future of higher education (Oblinger, 2012). EDUCAUSE, UNESCO, and COL, among other organizations have highlighted the potential of openness to greatly reduce the cost of education by encouraging the adoption of open access journals, open teaching, and open textbooks. Various institutions in the United States are currently using and testing the potential for open textbooks to help improve textbooks as well as reduce the cost of educational materials (Carpenter, 2010). The open access movement is of relevance to graduate students as libraries are increasingly unable to subscribe to many valuable journals (http://righttoresearch.org/). In recent years there has been a rapid rise in subscription costs of academic journals, increasing by 145% over the past six years, despite the decreased cost of production due to distribution and communication advances. Spending over $3.5 million a year in subscriptions, Harvard University libraries recently argued that “many large journal publishers have made the scholarly communication environment fiscally unsustainable and academically restrictive” (Harvard Library, 2012). They are not alone; thousands of educators have also joined to boycott Elsevier (Sample, 2012).

While the production of high quality products is and will likely continue to be expensive and it is important to pay content producers a competitive rate for the development of quality educational products, production models adopted by organizations such as Flat World Knowledge are able to do so while offering their textbooks for a fraction of the cost in its printed format. Spending an average of over $900 dollars on textbooks every year, textbooks are a major expense for college students (Wiley & Green, 2012; GAO, 2005). In contrast open textbooks are free to download digitally, and the faculty member gains the liberty to be legal able to modify the modules that are included within the e-book, and by adding, formatting or deleting, the specific content of any modules.

CEHD’s Open Textbook initiative promoted by CEHD’s Dean Jean Quam and David Ersnt, CEHD Director of Academic and Information Technology and Provost Karen Hansen, who state that “the University of Minnesota should be a leader in enabling faculty and students to benefit from open content and electronic textbook options” are hopeful about the potential for this new initiative to help improve higher education (Anderson, 2012). I support this statement and hope that more students, both within the University of Minnesota and outside of it, will increasingly benefit from open textbooks and other high quality open educational resources.

Open access to academic articles is another way in which the University of Minnesota could help to further reduce long term costs for students. With journal subscriptions costing millions of dollars to major universities, a lower cost to journal subscriptions would modify the financial requests of the University of Minnesota libraries, increasing their attention to improving other services. Open access journals does not detrimentally affect many researcher as research is regularly paid by grants or by the university as part of the researcher’s salary. As the recent reaction against Elsevier illustrates, there is support within parts of the academic community for a new equation that support greater access to high quality information. It is difficult for a young scholar to “stand in the shoulders of giants” without access to high quality resources and the most recent information. A recent study showed that 40% of researchers in the UK could not access the articles they needed on a daily or weekly basis (RIN, 2009).

Open teaching which includes the recent increase of large open online courses (also known as MOOCs) by MIT, Harvard, Stanford, Yale, and various other prestigious universities, has led to the increased sharing of high quality information and the increasingly the visibility of the university and its researchers (Masters, 2011; DeSantis, 2012). Other initiatives such as Khan’s Academy and educational resources in sites such as TED, YouTube EDU, iTunes U, or Sophia.org are examples of the possibilities for impact that an instructor can have not only statewide but across the world (Bonk, 2009).

The increased visibility provided by Open Teaching can benefit students and faculty members. Short segments of courses or speeches that are particularly captivating can help the career of a scholar while the diffusion of an innovation or a good idea. Recently the University of Minnesota had the opportunity to host a TEDx event, successful organized by a number of undergraduate students (http://tedxumn.com/). TED is a good example of the power of openness. While TED has been active since 1984, it lectures have only been available through the Internet since 2007. Since then, the best TED lectures have reached million viewers. Advocating for further openness, I believe it is important to support initiatives such as TEDxUMN to increase the impact of our best ideas and innovations. The university could also potentially host a similar initiative on a yearly basis to provide a platform for the diffusion of the best ideas and innovation by university scholars.

Unfortunately, openness is often criticized, as an agent of change. Yet while some may argue that the current system is working properly, students are increasingly leaving higher education indebted with lack of certainty about their financial future. Further openness can help to reduce the rising cost of education while increasing the sustainability of the system. Without changes it may be increasingly difficult for some students, particularly those of lower socio-economic standards to attend the University of Minnesota. As state funding decreases and other cost continue to rise, it is important to search for new and innovative ways by which to reduce the cost of schooling while maintaining a high level of quality and a level of access that is in accordance with the mission of the university as a land grant institution. “The land-grant university system is being built on behalf of the people, who have invested in these public universities their hopes, their support, and their confidence” President Abraham Lincoln upon signing the Morrill Act, July 2, 1862 (http://landgrant150.umn.edu/).

I believe that Open Textbooks can impact the overall cost of school for undergraduates, professional students and some graduate programs. I also believe that by promoting further openness through open access, open educational resources (in addition to open textbooks) and open teaching, the university community can contribute to the construction of a more affordable experience for students, the increased visibility of university projects, while also giving back to the state, the nation, and the world, moving forward in fulfilling a part of the mission of the University of Minnesota.

Works Cited

Anderson, K. (2012, April 23). U creates Open Academics textbook catalog to reduce student costs. Retrieved April 30, 2012, from University of Minnesota News Release: http://www1.umn.edu/news/news-releases/2012/UR_CONTENT_383497.html

Bonk, C. J. (2009). The World Is Open: How Web Technology Is Revolutionizing Education. New York: Jossey-Bass.

Carpenter, M. A. (2010). Flat World Knowledge: Creating a Global Revolution in College Textbooks! Irvington: Flat World Knowledge.

DeSantis, N. (2012, May 05). Harvard and MIT Put $60-Million Into New Platform for Free Online Courses. The Chronicle of Higher Education , pp. http://chronicle.com/blogs/wiredcampus/harvard-and-mit-put-60-million-into-new-platform-for-free-online-courses/36284.

GAO. (2005). College Textbooks – Enhance Offerings Appear to Drive Recent Price Increases. Washington DC: United States Government Accountability Office.

Harvard Library. (2012, April 17). Faculty Advisory Council Memorandum on Journal Pricing – Major Periodical Subscriptions Cannot Be Sustained. Retrieved April 29, 2012, from Harvard University: http://isites.harvard.edu/icb/icb.do?keyword=k77982&tabgroupid=icb.tabgroup143448

Immerwahr, J., Johnson, J., & Gasbarra, P. (2008). The Iron Triangle: College Presidents Talk about Costs, Access, and Quality. San Jose: The National Center for Public Policy and Higher Education and Public Agenda.

Masters, K. (2011). A Brief Guide To Understanding MOOCs. The Internet Journal of Medical Education , http://www.ispub.com/journal/the-internet-journal-of-medical-education/volume-1-number-2/a-brief-guide-to-understanding-moocs.html.

Oblinger, D. G. (2012). Game Changers: Education and Information Technologies. Louisville: EDUCAUSE.

RIN. (2009). Overcoming barriers: access to research information content. London: Research Information Network.

Sample, I. (2012, April 24). Harvard University says it can’t afford journal publishers’ prices. The Guardian , pp. http://www.guardian.co.uk/science/2012/apr/24/harvard-university-journal-publishers-prices.

SHEEO. (2012). State Higher Education Finance FY 2011. Boulder: State Higher Education Executive Officers (SHEEO).

Wiley, D., & Green, C. (2012). Why Openness in Education? In D. G. Oblinger, Game Changers – Education and Information Technologies (pp. 81-89). Louisville: EDUCAUSE.



 

For a More Robust Evaluation of 1 to 1 ICT for Education Adoption Projects 

The rapid change of information and communication technology (ICT) increases the challenge in determining how to best evaluate proficient use of these technological advances and their impact on learning. Through an overview of different initiatives, this paper illustrates the benefits of implementing a mixed-methods approach, and analyzing projects over a prolonged period of time. Looking at a program in a longer timeframe can enable us to be more aware of the impact a program has on an individual and a community. The use of mixed-methods allows us to analyze a program in various ways, studying variables that are measurable and generalizable, as well as elements that are specific to a particular situation. By incorporating these elements into evaluation studies we can potentially increase the quality and usability of the reports generated. To illustrate the benefits of mixed-methods and the continued analysis of a project, this paper discusses the 1 to 1 iPad project at the University of Minnesota.

Rapid Rate of Change – A Relevant Characteristic of ICT for Education Projects

It was only a few decades ago, in 1978, when top MIT computer scientists had reservations about the usability of the personal computer for tasks such as an address book or a personal calendars (Tippet & Turkle, 2011). Today, universities in the United States increasingly consider remodeling their computer labs, as almost all college students in the United States (89.1% – 2009 at UMN) bring their own laptops to the university (Walker & Jorn, 2009). Students bringing their laptops to college increased from 36% in 2003 to 83% in 2008 (Terris, 2009).

The rapid improvement of technology results in the rapid depreciation of gadgets, and contributes to the difficulty of evaluating them. The increased computational power and capabilities of technology has encouraged educational institutions and other industries to adopt them. Ownership of Information and Communication Technologies (ICTs) has decreased the costs of transferring data and increased workers’ potential productivity (Friedman, 2007). Influential ICTs such as the mobile phone, the television, the internet and the radio, have augmented the quantity of information available to individuals. The economic benefits from improvements in information and data transfers have led to increased investments. There has also been growing interest in digital literacy as a necessary skill in the 21st century (Flannigan, 2006; Jenkins et al., 2006). While not all of the changes brought by increased access to technology are positive, greater access to information and the rapid improvement of these technologies has a major impact in society (Carr, 2011; Kurzweil, 2000). Unlike some traditional fields such as mathematics, or history, where most basic concepts have remained unchanged, the impacts of new media and its prevalence in society has changed substantially over the past few decades and with it the difficulty in evaluating these projects. Mobile subscriptions alone increased from less than 3 billion in 2006 to 5.9 billion in 2011 (ITU, 2012)

This rapid change makes it difficult to determine the essential skills a learner must have in the workplace of tomorrow (Cobo & Moravec, 2011). With hundreds of thousands of computer applications and many types of hardware, some of high levels of complexity; it can take a person a significant amount of time to become adept in any complex program. A high level of specialization is often the norm, as using complex programs successfully requires a degree of mastery of statistical analysis or qualitative research methods. Similarly, programs such as Adobe Photoshop, or Python, among others, have a considerable learning curve (There are courses available for learning any of these programs). Being a specialist in a particular program can lead toward a very successful career, but simply mastering a single program can take  hundreds of hours of practice. While it may take 10,000 hours to become a successful reporter, a successful violinist, or a successful writer (Gladwell, 2008), ICTs encompass thousands of possibilities, each requiring differing amounts of time in which to become proficient (this includes unique musical instruments, and new ways of writing [via text or twitter]).

Understanding this rapid change is important in evaluating ICT adoption programs because it influences what we consider to be the effective use of these technologies by the general population. Texting, for example, is increasingly common and is considered by some experts to be a nascent dialect (Thurlow & Brown, 2003).  How important is it to know how to effectively send texts and use a mobile phone in the 21st century? It is hard to answer these questions as a technology may be displaced in a few years’ time. The rapid change of technology complicates how we measure digital literacy and through it the effectiveness of 1 to 1 adoption and usability programs. These complications are at times difficult to perceive because of generational differences between the evaluator and younger generations (Prensky, 2001).

Today young adults (18-24) send an average of 109.5 text messages a day or 3,200 text messages a month and many of them prefer communicating by text messages over emails. Email, a relatively recent invention, is to some already considered old fashioned and impractical (Smith, 2011). With this in mind, does an individual’s capacity to use emails effectively continue to be a 21st century digital literacy requirement? While the International Society for Technology in Education (ISTE) has developed ICT for education standards which can aid the evaluation of technology adoption programs (ISTE, 2008), these standards emphasize broad competencies and must be operationalized to reflect the distinctiveness of each 1 to 1 ICT program.

In this essay I propose to evaluate a 1 to 1 technology project over a long period of time to assess the impact of the program on the individual over time. One of the key advantages of 1 to 1 initiatives is that the participants are able to take home the devices. It is easier to become proficient using a device that one has access to at home, compare to one that is limited to use within the classroom setting. As argued by Seiter (2008), [t]here is an overestimation of access to computers in terms of economic class, and an underestimation of specific forms of cultural capital required to maintain the systems themselves and move beyond the casual, recreational uses of computers to those that might lead directly to well-paid employment” (Pg. 29). If Seiter (2008) is accurate, most of the economic benefits from ICTs come from their long term use.

ICT investment can be expensive and many  ICT projects could not be developed without the support of private industry and the government (Heshmati & Addison, 2003).  While ICT may not be as important as basic education, food, and health services, governments around the world have spent large amounts on ICT for education initiatives, hoping to imitate the success many advanced economies have obtained from their ICT industries and byproducts (MSC, 1996). “Investment in ICT infrastructure and skills helps to diversify economies from dependence on their natural-resource endowments and offsets some of the locational disadvantages of landlocked and geographically remote countries” (Heshmati & Addison, 2003, p. 5)

Adequately evaluating 1-to-1 technology adoption initiatives is increasingly important, as different education interventions may have different cost-effectiveness ratios, and cost-benefit ratios, with some interventions being much more effective than others (Yeh, 2011). Working with limited resources, governments must administer their funds in the best possible way to enable their citizens to meet their various needs, from food and shelter, to self-actualization. Just because one intervention is more cost effective, does not mean that the other intervention should be necessarily discarded. As Maslow (1943) suggested, many needs can, and should, be met simultaneously. An improvement in one area of life, such as shelter, does not occur in a vacuum, and is not exclusive from the individual’s desire to feel accepted by others, or to improve their problem-solving ability (Ibid.). Investing in ICT is important for states as they move towards becoming economically diverse, robust and more competitive, relying more on their human capital than their natural resources. To evaluate these projects more precisely; this paper encourages evaluators to use a mixed-method analysis with a long-term time perspective.

Evaluating Information and Communication Technology Projects

Evaluation can help to increase the effectiveness of programs and improve the distribution of the limited resources available to a society. The decisions made by an evaluator can impact the lives of many individuals. Evaluators can help improve a program as well as decide whether or not the program should be continued (Fitzpatrick et al., 2011). Discussing the methodology of evaluation, Scriven (1967) differentiated between formative (focus on development and improvement) and summative (focusing on whether the program is meeting its stated goals) evaluation.  By conducting an evaluation a decision-making body is able to make an informed decision about the future of the program. Yet, dealing with complex programs with large numbers of pieces it is difficult to frame an evaluation to obtain the most valuable information, particularly when there is a limited time to conduct it, and the brevity of a report can be one of its strengths (Krueger, 1986). Yet, different methods provide for valuable lenses through which to look at a problem, frames that the evaluator should consider before conducting their evaluation.

Possibly the most important elements to consider in a 1 to 1 ICT project are its cost, and its use by the learners. The most well-known 1 to 1 initiative is the One Laptop Per Child Program (OPLC) which has delivered hundreds of thousands of units (http://one.laptop.org/). Yet with over $100 cost per student it could cost $500 billion dollars to provide a computer to every person that currently lacks access to the internet worldwide ($5 billion people). This would not include continued maintenance and electricity cost or the cost to access the Internet. Is access to ICT really that important? According to a recent UNESCO (2012) publication, while 1 to 1 laptop projects are very costly, in Latin America “in the last three years, the 1:1 model has become increasingly widespread, and 1:1 programmes are now the primary focus of national policies for ICT in education in the region. Policy-makers are no longer discussing whether the 1:1 model is worthy of investment but rather how best to achieve it” (Lugo & Schurmann, 2012)

While a price tag of $100 appears as an expensive investment for developing countries, especially when some countries spend less than $100 per student a year within their educational budget, it is also important to consider that all programs have costs, even when they are not financial. Even 1 to 1 programs that are “free” (through donations) have a cost, including an e-waste disposal cost. Even when they are based on volunteer efforts, programs still have as a minimum a lost opportunity cost for instructors and learners. The cost of programs can be most effectively assessed by measuring their different ingredients. This allows programs to be quantified; for various elements to be weighted, and, as a result, for programs to be compared through a cost-effectiveness analysis (Levin, 2001). The financial benefit of the program can also be determined through a cost-benefit analysis. Through a qualitative study, “thick”, rich descriptive information can be obtained and thematically organized, helping key stakeholders to better understand elements that would otherwise go unnoticed (Geertz, 1973).

Programs can also be mapped through a logic model which can include inputs, activities, outputs, and outcomes (Alter & Murty, 1997; McLaughlin & Jordan, 1999). The order in which the elements of a program are implemented and the context may also influence the results of the program. There are also likely to be competing program alternatives, some of which may be more effective than the particular program being considered. Hoping to increase the transferability or generalizability of a study, an evaluation can also be theory driven (Weiss, 1997). These and other elements, can improve the quality and usability of data obtained by an evaluation. However, with limited time and resources, the methodology used to evaluate a program depends on both the strengths of the researcher, and what is considered of principal importance by key stakeholders.

Over time, every practicing evaluator is or is in the process of becoming a “connoisseur” (the art of appreciation), as well as a “critic” (the art of disclosure) (Eisner, 1994, p. 215). This knowledge allows him or her to more effectively recommend to key stakeholders  the best methods of evaluation to pursue in a particular scenario. However, the interests of secondary stakeholders are also important in many ICT adoption programs.

The Relevance of Mixed Methods and Triangulation

“The underlying rationale for mixed-methods inquiry is to understand more fully, to generate deeper and broader insights, to develop important knowledge claims that respect a wider range of interests and perspectives” (Greene & Caracelli, 1997, p. 7).

 

Mixed-methods can greatly benefit a study as they allow the researcher to ask questions that he or she may ignore otherwise, obtaining additional information. While “purists” oppose the use of mixed-methods due to potential epistemological and ontological contradictions, many evaluators take a more “pragmatic” approach (Greene et al., 1989). One of the concerns regarding the use mixed-methods is that they may compromise the methodological integrity of an experimental study. These are valid concerns, and it is important to consider carefully how methods are being utilized, to avoid unintended conflicts. Some of the theoretical concerns for researchers against using mixed-methods may not be as applicable to evaluators as evaluators do not have the same goals as researchers. While researchers are focused to a greater extent on theory and generalizability and transferability, many evaluators focus on utilization and the practical implication of their analysis to their key stakeholders and the future of the program (Patton, 2007). To the “pragmatist” evaluator, “philosophical assumptions are logically independent and therefore can be mixed and matched, in conjunction with choices about methods, to achieve the combination most appropriate for a given inquiry problem. Moreover, these paradigm differences do not really matter very much to the practice” (Greene, et al., 1989, p. 8).

Mixed-methods often refers to the use of methods from different paradigms, using both a qualitative method, such as unstructured interviews or participant observations, with a quantitative method, such as academic achievement scores, or another statistical value within the same study (Johnson & Onwuegbuzie, 2004). While it seems beneficial to analyze a problem in multiple ways, experts in both qualitative and quantitative methods express concerns about this approach. Johnson and Onwuegbuzie (2004) argued that some of these “purist” concerns stems from a “tendency among some researchers to treating epistemology and method as being synonymous,” which is not necessarily the case (Pg. 15). Johnson and Onwuegbuzie (2004) argue for a contingency theory approach to research which emphasizes that while no method is superior, there are instances when one is preferable to the other.

One of the biggest benefits of using mixed methods is that they allow for the triangulation of findings. According to Dezin (1978) triangulation is “the combination of methodologies in the study of the same phenomenon” (pg 291). Dezin describes four types of triangulation: data triangulation, investigator triangulation, theory triangulation, and methodological triangulation. He describes these as possible within-methods, or between-methods (Ibid.). The ways in which methods are mixed varies, sometimes all methods having the same amount of influence, while at other times one method holds preeminence. Triangulation is a common way in which to strengthen the generalizability and transferability of a study and the strength of its claims.  Other benefits of using mixed-methods include complementarity, where the results of one method are clarified by another, development, when one method informs the other, expansion, trying to increase the scope of one methodology, and initiation, which seeks the discovery of paradox by recasting results or questions from one method to another (Greene et al., 1989). Regardless of the initial results, it usually provides richer data. Comparisons between the data could lead to either “convergence, inconsistency, or contradiction” (Johnson et al., 2007, p. 115).

If there is a conflict or an inconsistency within the data, it increases the difficulty of establishing a causal relationship and the project may require further study and explanation. This explanation can be provided by a form of structural corroboration, further analysis, or by sharing both findings with the key stakeholder. He or she can then use both pieces of information to make his or her decisions (Eisner, 1994). While most evaluators feel a responsibility to provide recommendations to the stakeholders, these recommendations do not necessarily have to address the problem scientifically, rather a “connoisseur” may state that based on his experience, he or she believes this path may be the best path to follow. ICT adoption includes many invisible elements which increases the difficulty in evaluating them (Cobo & Moravec, 2011). Because of its complexity, it will be helpful for the evaluator to share his or her opinion as a “connoisseur”. Social programs are generally complex. By providing a focused report to the key stakeholders, that emphasizes the main findings of the mixed-methods evaluation, they will be more likely to make a good formative or summative decision. As will be illustrated, this was an objective pursued by to 1 to 1 iPad initiative at the University of Minnesota.

Encouraging the Long-Term Study of ICT Projects

The limited timeframe of a study can result in a restricted analysis. Iterative formative evaluations allow key stakeholders to constantly reevaluate ways in which to improve a program (Mirijamdotter et al., 2006). Iterative and continuous evaluations are very important for internet based companies.  Google, for example, is known to regularly test new algorithms and versions of their search engine simultaneously, to obtain helpful usability comparisons. They try hundreds of variations of their search engine a year in an attempt to improve their product. (Levy, 2010). Similarly, many ICT adoption projects include an iterative process in their analysis, yet in the discussion of their findings, evaluations regularly omit the potential long-term benefits of the programs, focusing instead on short-term costs and benefits.  While there are time constrains and financial limitations to evaluations of 1-to-1 laptop programs, these evaluations would benefit from more attention to measuring the long term benefits of the interventions, including cultural capital gain (Seiter, 2008).

Methodologies such as longitudinal studies, ethnographic research, and time-series are among those that can help illustrate the potential benefits of the long term analysis of an intervention. Some of these studies can be very expensive, but they allow for the observation of changes that would otherwise go unnoticed. Another example of the possibilities of looking at changes over time was recently made possible by the Google Books Project Ngram Viewer (http://books.google.com/ngrams). The NGram Viewer allows for word frequencies to be analyzed over a span of 200 years! This type of study, called Culturenomics, is one of the newest ways in which an analysis of a subject over time provides an additional insight to an issue (Michel, et al., 2010). While the NGram Viewer is not very useful for evaluators, other forms longer-term analysis can be of greater support.

Ethnography is a field of study in which time spent in the field is an important validity variable. Ethnographers focus primarily on the quality of the data, and validity can be increased if the researcher has lived in a community for a longer time and, in so doing, has obtained a greater understanding of the local culture. Some of the subtleties that are analyzed by ethnographers require time and involvement to be discovered. To some researchers, ethnography symbolizes a study that takes more than a year (Fuller, 2008). However, some projects could last perhaps a single long day, while other “projects are developed throughout the whole of a researcher’s life; an ethnography may become a long, episodic narrative” (Jeffrey & Troman, 2004). In quantitative analysis, a time series, as their name implies, also emphasizes the importance of collecting data over time. This set of statistical data can be collected at various intervals;  monthly for unemployment benefits data; or daily for the financial exchange rate; or even every 2 seconds for EGG brainwave activity. A commonly used and informative time series is population census data, which is collected by many countries in regular intervals to help their governments better understand broader demographical changes, migratory patterns, and the future outlook of various variables (Zhang & Song, 2003).

Longitudinal studies can also be very helpful in understanding how an intervention at an early stage of a person’s development influences them throughout the rest of their lives. Various longitudinal studies have been conducted within early education. Longitudinal studies include interventions in pre-natal care, youth reading programs, or the observation of children as they become older, among many other studies. One of the most famous longitudinal studies of education was the Student/Teacher Achievement Ratio (STAR) Tennessee Class Size Reduction study which began in 1985 and which continued until 1999 (Finn & Achilles, 1999; Hanushek, 1999). The study tracked students who were assigned at random to kindergartens with between 13 and 17 students, or larger classes of between 22 and 26.  Over 6000 students took part in the study in which they were kept in smaller classrooms for 4 years, and monitoring continued after the end of the intervention. The study found statistically significant changes to student achievement scores in three utilized measurements. The conclusions of this study strengthened claims regarding the positive impacts of class size reduction, encouraging the enactment of class reduction policies in California (1996) and other states. While later studies have contradicted the findings of the study, its use of an experimental design, its magnitude and its use of a longitudinal analysis strengthened its claims. There have been a number of important longitudinal studies in early childhood and other early interventions that have followed children’s development for decades (NCES, 2010).

Another popular, long-term, longitudinal study is the British Up Series which has followed a group of 14 children since age seven in 1964, and is still under production. Similar documentaries have been replicated in Australia (since 1975), Belgium (1980-1990), Canada (1991-1993), Czech Republic (1980s), Germany (1961-2006), Denmark (From 2000), Japan (from 1992), Netherlands (from 1982), South Africa (from 1982), Sweden (from 1973), USSR (from 1990), USA (from 1991). While these long-term studies can be expensive to conduct, they provide a different dimension to findings, a dimension that is sometimes not available in most 1 to 1 technology adoption evaluations.

The key benefit of including this dimension within an evaluation derives from the difficulty in knowing how the skills obtained from using new ICT devices will help an individual have the confidence and the background skills needed to develop future ICT competencies that may be beneficial to them in the job market. Will their familiarity with ICT at an early age bring about broader benefits later in their lives? A short-term outlook in an evaluation may, at times, provide a negatively skewed view of the impact of these projects, expecting more out of a pilot project than is realistic. In addition, it is common for program designers to overstate the potential outcomes of a project, expecting it to have a greater impact than it is likely possible. For example, as an evaluation of USAID basic educational projects (1990-2005) showed, most of its projects had less than a 4% increase in student achievement scores, despite the efforts of many specialists and the expenditure of millions of dollars. (Chapman & Quijada, 2007). One to one technology adoption projects can also be very expensive and, as such, can have a very negative cost-benefit analysis in the first years of the program. Evaluations should also take into account, future, longer-term benefits of the investment.

By evaluating a project while considering its impact over a longer time this article encourages the continued evaluation of a program over a number of years, on regular intervals, while providing recommendations, and reporting on the benefits and negative elements of the program as they are modified over time. This type of long term evaluation is best suited for an internal evaluator, or a combination or internal and external evaluators. When thinking of the cost of 1 to 1 programs over time, it is also important to keep in mind the rapid depreciation of technology. With the rapid depreciation of computer equipment, should 1 to 1 programs focus on purchasing the most up-to-date gadgets and tools? This is a question that is be best analyzed through the inclusion of a cost-effectiveness analysis which accounts for the depreciation of technologies.

One Laptop Per Child – An Evaluation of Peru’s Project

Possible the most controversial and also most commonly cited 1 to 1 initiative is the One Laptop Per Child (OLPC) initiative, which was started by Nicholas Negroponte, the founder of the MIT Media Lab (TED, 2008). According to Negroponte, by thinking in bytes instead of atoms, and by learning how to operate a computer, a child can learn that the world is increasingly available at the click of a button, and that they can construct and build anything that they can imagine by programing new and amazing environments (Negroponte, 1996). Following Papert’s Constructionism, Negroponte believes that programing teaches an individual how to learn, as they must go back, revisit their code and figure out why there is a mistake (Papert, 1980). As an ICT evangelist, Negroponte highlighted how simply by giving a child a computer his possibilities would be expanded (Negroponte, 1996). Since the beginning of OLPC in 2005, over 2.5 million laptops have been delivered (http://one.laptop.org/about/faq). However, despite the high level of investment, particularly in Latin America, project evaluations have not shown significant gains in achievement scores (Cristia et al., 2012).

A recent evaluation of OLPC in Peru expressed how, despite a high level of investment in these new machineries (902,000 laptops), and increasing the ratio of computers from 0.12 to 1.18, student performance in math and reading had not increased substantially. The project did find that students’ cognitive skills had improved over the time of the study. While analysts have since highlighted that the program had only limited effects on math and language achievement (0.003 standard deviations), little emphasis has been given to the potential impact of the improvement in cognitive skills, and perhaps more importantly, to what having improved their digital literacy skills will mean for these individuals in the future, as they are asked to learn other task specific digital and information literacy skills (Cristia et al., 2012).

It is also difficult to know from the available data whether a different investment would have been more cost-effective or result in a higher cost-benefit ratio in Peru. One of the unmet goals of OLPC was to produce a $100 laptop; however they currently cost around $200 (Ibid.). As a project which was not affiliated with Microsoft, Google or Apple, the OLPC laptops came with an operating system (OS) known as Sugar. While all operating systems share similarities, did the use of Linux Sugar limit or increase the possibilities for students? When testing student computer literacy skills, they found that the students quickly became more adept at using these devices. As explained earlier in this paper, evaluators also had difficulties in deciding which skills should be tested (Ibid., p. 15). Unfortunately, another unmet goal of the project was that Peru’s OLPC participants lacked internet connectivity. OLPC was partly designed so that students could benefit from increased connections either through the OLPC exclusive Mesh network or the Internet. The impacts of lacking access to the internet are hard to measure, however they may have affected the individuals’ development of information literacy skills. Peru’s evaluation of the OLPC project was very insightful. However, while it contained a qualitative element, the project had a quantitative focus, limiting reader’s understanding of how the initiative affected individuals. As a project which centers on the individual, learning more of the project’s impact on the person is increasingly relevant as ICT becomes more personalized. Apart from not discussing potential long-term gains, the evaluation also failed to mention the full cost of the devices. With the laptop only accounting for a tenth to a seventh of the total cost of the device, it is important to consider whether this is a cost-effective investment (Lugo & Schurmann, 2012). The evaluation would have benefited from a broader implementation of mixed methods, in particular on the qualitative-side, while also emphasizing these changes over a longer span of time. An element of time that is particularly important to first year initiatives is the teacher’s or instructor’s familiarity or learning curve, as they will slowly learn better ways in which to use the device and integrate them within the classroom.

A Case Study – University of Minnesota One iPad Per Student Initiative

The discussion surrounding the digital divide is traditionally centered around on access to the internet and a personal computer, yet the rapid change of technologies leads us to question whether the divide will be centered on these devices in the future (Warschauer, 2008; Zickuhr & Smith, 2012). What role will smart phones, reality augmented glasses, 3D printers or, farther into the future, nanotechnology implants signify in terms of the digital divide? (Kurzweil, 2000). A current technology that may further displace the purchase of paper books for K-12 and HE is e-reader technology, the most successful of which are the iPads (I, II, and III) and Amazon’s Kindle readers. A recent NDP report indicated that tablets may outsell laptop computers by 2016, expanding sales from 81.6 million units (2011) to 424.9 million units (2017) a year (Morphy, 2012). Will we then measure the digital divide in terms of who access and who doesn’t have access to an iPad?

Pilot projects in universities such as the University of Minnesota, the University of San Diego, Oberlin College and a few others have moved forward in answering this question. While the first successful tablet, the iPad, was released on April 2010, that same year, the University of Minnesota decided to purchase 447 units, to provide a tablet to every CEHD student in the upcoming undergraduate cohort. It was one of the first major initiatives of its type in the country. Because of its uniqueness, and being an early adoption project, its evaluation was based partly on the conclusions obtained from previous 1-to-1 projects such as the OLPC initiative and Maine’s 1-to-1 statewide adoption program. However, as a device that was substantially different from previous ICT devices, the operationalization of NETS standards, and an in-depth analysis of their potential use has not been acutely studied (ISTE, 2008). So far, only a few articles have been published regarding the use of the iPad in the classroom (EDUCAUSE, 2011). To better understand the possible educational implications of the adoption of this technological device, a CEHD research team conducted a mixed-methods evaluation (Wagoner et al., 2012). In addition, a commitment was made to continue evaluating the project for a consecutive number of years. The support of the dean was integral in the continuation of the program.

The first year, the project set a goal to increase the usability of the devices by both faculty and students, and to provide aid to faculty members so that they could familiarize themselves with the devices and consider the best ways to incorporate the devices within their classrooms. Soon after the distribution of iPads, evaluators also drafted a post-test and organized a series of interviews. The interviews asked faculty members how they learned to use their iPads, what were their plans for using them within the classroom, how the iPad had affected their teaching, and if the support received had been appropriate (From field notes).

A similar set of questions were asked to faculty members at the end of the school year, where they were asked what projects they had actually implemented, the opinions of students regarding ebooks, and their pedagogical concerns . Twenty two interviews were coded and themes were developed from the qualitative study, including concerns from faculty about time investment, how the iPad compares with other technologies, the impact of the iPad on faculty members’ pedagogy, the impact of the iPad on their classroom management, and details about faculty members’ technology learning processes. At the end of the year a series of faculty member focus groups were also conducted. Many of the details learned through the qualitative portion of the study would have been difficult to obtain otherwise. The common elements between the data from the focus groups and the interviews also allowed us to verify some observations. Below is an interesting quote from one of the participating faculty members:

“What I want, in terms of their behaviors, is for [the students] to be active explorers in the classroom, to bring the machines, and to actually utilize them for historical research … One of the things that we did as a first conversation is to describe the level of trust that is going to be involved … and they live up to those expectations. I’ve been really happy so far with what we’re learning. It conveys to them that they’re smart, capable discoverers that we’re co-creating knowledge—historical knowledge” (Wagoner, Hoover, & Ernst, 2012, p. 3)

 

While the quote above illustrates a very positive aspect, it is likely that this experience would not have been visible through an analysis of student achievement, illustrating the benefit of utilizing mixed-methods. Two student focus groups were also conducted yet unlike for faculty members where evaluators were able to interview the whole population, 447 students were more than the team could interview. To obtain a better analysis of the student responses, a survey was conducted which included a number of questions related to their use and experience with the iPad. 241 CEHD first year students responded to the survey (Wagoner et al., 2012). Having access to broader demographic data also allowed the evaluation team to compare student attitudes with socio-economic variables. Various strong correlations and significant relationships were found regarding the impacts of iPads on student learning. In particular, the evaluation found that students felt that the devices had been a positive experience in terms of their motivation. Students also expressed a high level of comfort using the devices and reported that the iPad helped them feel more engaged in some of their classes.

 

The study also showed that students who were part of Access to Success (ATS) or had been part of the TRIO program, usually students of color or from low socio-economic backgrounds, mentioned feeling more engaged and connected during classes. From the qualitative data the evaluators also learned that for some students the iPad had become a window into the internet, and a digital item for their whole household to use.

The success of the first year implementation, and the questions that evaluators were still unable to answer led to the continuation of the program for a second and third year. A similar number of iPads (now iPads 2) were purchased. Once again the rapid change of technology provided new possibilities for evaluators, as iPad 2s include cameras, permitting students to record HD video and have audio-visual communications with anyone with access to Facetime or Skype.  After analyzing the potential savings of the extensive use of iPads for e-reading by some students, CEHD also decided to support a pilot project for the testing and adoption of Open Textbooks, as well as the establishment of a workdesk where faculty members could obtain assistance and build iBooks and ePubs if interested.

The project is now planning its third year. Adapting to the result of the first year evaluation, many of the questions of the second year survey were modified to find additional valuable information. One of the limitations of the evaluation of the program so far has been a lack of a cost-effectiveness or a cost-benefit study. Yet, such a study should not only take into account the rapid depreciation of the devices, but also consider if students are learning skills that could potentially aid them when they join the workforce. While the costs have been high, over 300,000 dollars per year, it is difficult to assess the long term benefits for participants. The rapid devaluation of the devices is an important consideration, as it may be possible that in a couple of years these devices will cost only a fifth of their original cost and be even more feature rich and powerful, allowing students to obtain a similar skill set for a fraction of the cost. It is also possible that many of the skills obtained are not very different from those obtained from using other ICTs, reducing the importance of the investment.

Currently, a website is available where individuals interested in the results of the project can learn various innovative classroom projects that were developed and how they can be adapted to other classrooms, as well as suggested best practices. In a report, CEHD concluded that the iPad had been helpful addressing the concerns of the Digital Divide, increasing access to the tools needed for media production, access to tools that facilitate personal productivity, improve students’ possibilities for information access and consumption, helped reduce the cost of printing readings, and facilitated students’ learning outside of the classroom (Wagoner, et al., 2012). For year two, the program also hopes to further analyze the usability of the devices and recently developed a space for students to submit their creative productions with the iPads.

Despite the insights provided by the use of mixed-methods for this evaluation, the limited timeframe of the study makes it difficult to determine whether or not is a worthwhile investment. With the program costing over $400 dollars per student, excluding the cost of the administrative staff, is this the best investment for a university to make in terms of technology adoption? When will it be determined that the program is no longer worth its cost and it is no longer helping to find innovative ways of learning? One of the limitations of CEHD’s 1 to1 iPad program has been the limited emphasis on the possibilities for the device within informal learning. Some of these concerns will be better analyzed from the data collected from the second year survey recently administered to students. A new wave of interviews and focus groups is also planned for the evaluation of the 3rd year of the program.

With 500,000 applications there are almost endless possibilities as to how the devices can be integrated within the classroom. The production of more apps that match more closely with the goals of each individual is likely to increase. Because of these devices’ future relevance, and the high level of creativity and innovation within this industry, constant evaluation of these devices is important as it allows for the continued improvement of the project. The use of mixed-methods allowed the evaluation team to find many interesting details that the study would not have found otherwise. These details enriched the quality of the findings and provided faculty with valuable information for the improvement of the use of the iPad and for learning how their peers were using the devices.

Conclusion

 

It is difficult to understand the repercussions of an event while it is taking place. Only with hindsight do we notice how many unexpected turns have led society to where it is today. Evaluators do not have the luxury of looking only at the past, as they are focused on improving the tomorrow. With an emphasis not just on understanding but on helping projects and programs improve in quality, decisions are made guided by what may be the most likely outcomes. Yet, without realizing it, a project could be cancelled before it demonstrates its true strengths. Too often ICT one-to- one projects focus on student achievement gains after the first year of implementation. As a magic bullet, some stakeholders may expect that just by having the device individuals will become more competitive. Projects such as OLPC have helped to promote this viewpoint. Yet, while technologies have helped improve society, it may take years for them to demonstrate the benefits to the lives of individuals. Changing cultures or behaviors takes time, and as has been the case with a large number of development projects, impact is usually moderate. Nevertheless, some investments will be more cost-effective than others and an evaluation of ICT needs to carefully analyze the costs of the ingredients of the intervention. Depreciating these ingredients and the considering which are the best ways in which students can develop competitive ICT skills is a primary objective for ICT one to one adoption projects. This paper contends that using mixed-methods and a longer-than-usual time spectrum for ICT evaluations will be able to provide more useful information to its key stakeholders, resulting in better decision making.

………………Page Break………………

 

Works Cited

Alter, C., & Murty, S. (1997). Logic modeling: A tool for teaching practice evaluation. Journal of Social Work Education, 33.

Carr, N. (2011). The Shallows: What the Internet is Doing to Our Brains. New York City: W. W. Norton & Company.

Chapman, D., & Quijada, J. J. (2007). What does a billion dollars buy? An analysis of USAID assistance to basic education in the developing world, 1990-2005. Washington DC: USAID.

Cobo, C., & Moravec, J. (2011). Aprendizaje Invisible: Hacia Una Nueva Ecologia de la Educacion. Barcelona: Universitat de Barcelona.

Cristia, J., Cueto, S., Ibarraran, P., Santiago, A., & Severin, E. (2012). Technology and Child Development: Evidence from the One Laptop per Child Program. Washington DC: IDB.

Denzin, N. K. (1978). The Research Act, 2nd Ed. New York: McGraw-Hill.

EDUCAUSE. (2011, September 02). 7 Things You Should Know About iPad Apps for Learning. EDUCAUSE Learning Initiative (ELI), p. http://www.educause.edu/Resources/7ThingsYouShouldKnowAboutiPadA/223289.

Eisner, E. W. (1994). The forms and functions of educational connoisseurship and educational criticism. In E. W. Eisner, In The educational imagination: On the design and evaluation of school programs (pp. 212-249). New York: Macmillan.

Finn, J. D., & Achilles, C. M. (1999). Tennessee’s class size study: Findings, implications, misconceptions. Educational Evaluation and Policy Analysis, 97-109.

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2011). Program Evaluation: Alternative Approaches and Practical Guidelines. Upper Saddle River: Pearson Education.

Flannigan, B. R.-K. (2006). Connecting the Digital Dots: Literacy of the 21st Century. Educause Quarterly, pp. 8-10.

Friedman, T. L. (2007). The World is Flat 3.0: A Brief History of the Twenty-first Century. New York: Picador.

Fuller, H. G. (2008). What does the term ‘ethnography’ mean to you? Quirk’s Marketing Research Review, pp. 48-50.

Geertz, C. (1973). Thick Description: Toward an Interpretive Theory of Culture. In C. Geertz, In The Interpretation of Cultures: Selected Essays (pp. 3-30). New York: Basic Books.

Gladwell, M. (2008). Outliers: The Story of Success. New York: Little.

Greene, J. C., & Caracelli, V. J. (1997). Defining and describing the paradigm issue in mixed-method evaluation. New Directions for Evaluation, 5-17.

Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a Conceptual Framework for Mixed-Method Evaluation Designs. Educational Evaluation and Policy Analysis, 255-274.

Hanushek, E. A. (1999). Some findings from an independent investigation of the Tennessee STAR experiment and from other investigations of class size effects. Educational Evaluation and Policy Analysis, 143-163.

Heshmati, A., & Addison, T. (2003). The New Global Determinants of FDI: Flows to Developing Countries. Helsinki: World Institute for Development Economics Research.

ISTE. (2008). The National Educational Technology Standards. Washington D.C.: International Society for Technology in Education.

ITU. (2012). The World in 2011: ICT Facts and Figures. Geneva: International Telecommunication Union.

Jeffrey, B., & Troman, G. (2004). Time for Ethnography . British Educational Research Journal, 535-548.

Jenkins, H., Purushotma, R., Clinton, K., Weigel, M., & Robison, A. (2006). Confronting the Challenges of Participatory Culture: Media Education for the 21st Century An analysis of USAID assistance to basic education. Chicago: The MacArthur Foundation.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed Methods Research: A Research Paradigm Whose Time Has Come. Educational Researcher, 14-26.

Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a Definition of Mixed Methods Research. Journal of Mixed Methods Research , 112-130.

Krueger, R. A. (1986). Reporting Evaluation Results: 10 Common Myths. American Evaluation Association. Kansas City: American Evaluation Association.

Kurzweil, R. (2000). The Age of Spiritual Machines: When Computers Exceed Human Intelligence. London: Penguin.

Levin, H. M. (2001). Cost-Effectiveness Analysis. Thousand Oaks: SAGE.

Levy, S. (2010, February 22). Exclusive: How Google’s Algorithm Rules the Web. Wired, p. http://www.wired.com/magazine/2010/02/ff_google_algorithm/.

Lugo, M. T., & Schurmann, S. (2012). Turning Mobile Learning in Latin America. Paris: UNESCO.

Maslow, A. (1943). A Theory of Human Motivation. Psychological Review, 370-396.

McLaughlin, J., & & Jordan, G. (1999). Logic models: A tool for telling your program’s performance story. Evaluating and Program Planning, 65-72.

Michel, J.-B., Shen, Y. K., Aiden, A. P., Veres, A., Gray, M. K., Team, T. G., . . . Lieberma, E. (2010, December 16). Quantitative Analysis of Culture Using Millions of Digitized Books. Science, p. http://www.sciencemag.org/content/early/2010/12/15/science.1199644.

Mirijamdotter, A., Somerville, M. M., & Holst, M. (2006). An Interactive and Iterative Evaluation Approach for Creating. The Electronic Journal Information Systems Evaluation, 88-92.

Mohammed, N. (2007). Facing Difficulties in Learning Computer Applications. Mount Pleasant: Central Michigan University.

Morphy, E. (2012, May 05). Tidal Wave of Tablets on the Horizon. Retrieved from E-Commerce Times: http://www.ecommercetimes.com/rsstory/75039.html

MSC. (1996). Smart School Road Map 2005-2020. Kuala Lumpur: Multimedia Development Corporation .

NCES . (2010). Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 . Washington DC: U.S. Department of Education.

Negroponte, N. (1996). Being Digital. New York: Vintage.

Papert, S. (1980). Mindstorms: Children, Computers, and Powerful Ideas. Basic Books: New York.

Patton, M. Q. (2007). Utilization-Focused Evaluation. Thousand Oaks: SAGE Publications.

Prensky, M. (2001). Digital Natives Digital Immigrants. On the Horizon, http://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf.

Scriven, M. (1967). The methodology of evaluation. In R. W. Tyler, R. M. Gagne, & M. Scriven, Perspectives of curriculum evaluation (pp. 39-83). Chicago: Rand McMally.

Seiter, E. (2008). Practicing at Home: Computers, Pianos, and Cultural Capital. In T. McPherson, Digital Youth, Innovation, and the Unexpected (pp. 27-52). Cambridge: MIT Press.

Smith, A. (2011). Americans and Text Messaging. Washington D.C.: Pew Internet.

TED. (2008, December 05). Speakers Nicholas Negroponte: Tech visionary. Retrieved from TED: Ideas Worth Spreading: http://www.ted.com/speakers/nicholas_negroponte.html

Terris, B. (2009, December 6). Rebooted Computer Labs Offer Savings for Campuses and Ambiance for Students. The Chronicle of Higher Education, pp. http://chronicle.com/article/Computer-Labs-Get-Rebooted-as/49323/.

Thurlow, C., & Brown, A. (2003). Generation Txt? The sociolinguistics of young people’s text-messaging. Discourse Analysis Online, http://extra.shu.ac.uk/daol/articles/v1/n1/a3/thurlow2002003.html.

Tippet, K., & Turkle, S. (2011, August 25). On Being: Alive Enough? Retrieved from American Public Media: http://being.publicradio.org/programs/2011/ccp-turkle/transcript.shtml

Wagoner, T., Hoover, S., & Ernst, D. (2012). CEHD iPad Initiative. Minneapolis: CEHD.

Walker, J., & Jorn, L. (2009). 21st Century Students: Technology Survey. Minneapolis: University of Minnesota.

Warschauer, M. (2008). Whither the Digital Divide? In D. L. Kleinman, K. A. Cloud-Hansen, & a. J. C. Matta, Controversies in Science & Technology: From climate to chromosomes. (pp. 140-152). New Rochelle: Liebert.

Weiss, C. H. (1997). How can theory-based evaluation make greater headway? . Evaluation Review, 501-524.

Willoughby, T. (2008). A Short-Term Longitudinal Study of Internet and Computer Game Use by Adolescent Boys and Girls: Prevalence, Frequency of Use, and Psychosocial Predictors. Developmental Psychology, 195-204.

Yeh, S. S. (2011). The Cost-Effectiveness of 22 Approaches for Raising Student Achievement. Charlotte: Information Age Publishing.

Zhang, K. H., & Song, S. (2003). Rural–urban migration and urbanization in China: Evidence from time-series and cross-section analyses. China Economic Review , 386-400.

Zickuhr, K., & Smith, A. (2012). Digital differences. Washington DC: Pew Research Center’s Internet & American Life Project.



 

For a More Robust Evaluation of 1 to 1 ICT for Education Adoption Projects 

May 12, 2012

The rapid chance of information and communication technology (ICT) increases the challenge in determining how to best evaluate proficient use of these technological advances and their impact on learning. Through an overview of different initiatives, this paper illustrates the benefits of implementing a mixed-methods approach, and analyzing projects over a prolonged period of time. Looking at a program in a longer timeframe can help us to be more aware of the impact a program has on an individual and a community. The use of mixed-methods helps us to take into account different ways in which we can analyze a program, studying variables that are measurable and generalizable, as well as elements that are specific to a particular situation. By incorporating these elements into evaluation studies we can potentially increase the quality and usability of the reports generated. To illustrate the benefits of mixed-methods and the continued analysis of a project, this paper discusses the 1 to 1 iPad project at the University of Minnesota.

Rapid Rate of Change – A Relevant Characteristic of ICT for Education Projects

It was only a few decades ago, in 1978, when top MIT computer scientist had reservations about the usability of the personal computer and whether or not people would use for tasks such as an address book or a personal calendars (Tippet & Turkle, 2011). Since then, many technology adoption projects were promoted, but items that were originally only available for the few are much more common in the present. Today, universities in the United States increasingly consider remodeling their computer labs as almost all college students in the United States (89.1% – 2009 at UMN) bring their own laptops to the university (Walker & Jorn, 2009). Students bringing their laptops to college increased from 36% in 2003 to 83% in 2008 (Terris, 2009).

The rapid improvement of technology results in the rapid depreciation of gadgets, as well as the difficulty of evaluating them. The increase capacity of technology and their computational power has encouraged educational institutions and other industries to adopt them. The ownership of Information and Communication Technologies (ICTs) has decreased the costs of transferring data and increased worker’s potential productivity (Friedman, 2007). Other influential ICTs are the mobile phone, the television, the internet and the radio have augmented the quantity of information available to individuals. The economic benefits from improvements in information and data transfers have led to increased investments. There has also been an increased interest in the importance of information and digital literacy as a necessary skill in the 21st century (Flannigan, 2006; Jenkins, Purushotma, Clinton, Weigel, & Robison, 2006). While not all of the changes brought by increase access to technology are positive, the increased access to information and the rapid improvement of these technologies has a major impact in society (Carr, 2011; Kurzweil, 2000). Unlike some traditional fields such as mathematics, history where most basic concepts have remained unchanged, the impacts of new media and its prevalence in society has changed substantially in the past few decades and with it the difficulty in evaluating these projects. Mobile subscriptions alone increased from less than 3 billion in 2006 to 5.9 billion in 2011 (ITU, 2012)

This rapid change makes it difficult to determine the essential skills a learner must have in the work place of tomorrow (Cobo & Moravec, 2011). With hundreds of thousands of computer applications and many types of hardware, some of high levels of complexity; it can take a person a significant amount of time to become adept in any complex program. Many users of Nvivo, a qualitative research software, many not know how to use SPSS, quantitative research software, successfully. A high level of specialization is often the norm, as using specialized programs successful requires a degree of mastery over statistical analysis or qualitative research methods. Similarly, programs such as Adobe Photoshop, Bryce, Python, Android OS, Excel, Audacity, among others have a considerable learning curve (There are courses available for learning any of these programs). Being a specialist in a particular program can lead toward a very successful financial career, but simply by mastering a single program can take dozens or hundreds of hours of practice. While it may take 10,000 hours to become a successful reporter, a successful violinist, or a successful writer (Gladwell, 2008), ICTs contains within it thousands of possibilities each with their different proficiency levels (this includes unique musical instruments, and new ways of writing [via text or twitter]).

The relevance of rapid change when evaluating ICT adoption programs is important because it influences what we consider to be the effective use of these technologies by the general population. Texting for example is increasingly becoming more common place and it is consider by some experts to be a nascent dialect (Thurlow & Brown, 2003).  Therefore, how important is it to know how to effectively send text and use a mobile phone in the 21st century? It is hard to answer these questions as a technology may be displaced in a few years’ time. The rapid change of technology complicates how we measure digital literacy and through it the effectiveness of 1 to 1 adoption and usability programs. These complications are at times difficult to perceive because of generational differences between the evaluator and younger generations (Prensky, 2001).

Today young adults (18-24) send an average of 109.5 text messages a day or 3,200 text messages a month and many of them prefer communicating over text messages than emails. Email a moderately recent invention, is to some already considered old fashioned and impractical (Smith, 2011). With this in mind, does an individual’s capacity to use emails effectively continue to be a 21st century digital literacy requirement? While the International Society for Technology in Education (ISTE) has worked on developing ICT for education standards which can aid the evaluation of technology adoption programs (ISTE, 2008), these standards emphasize broad competencies and must be operationalized to the distinctiveness of each 1 to 1 ICT program.

If technology continues to improve at a very rapid rate, perhaps even an exponential rate, it brings forth questions regarding what are the best ways which to evaluate a 1 to 1 technology project (laptops, mobiles, e-readers, etc.). In this essay I propose the analysis over a long period of time to assess the impact of the program to the individual over time. As argued by Seiter (2008) increasing access to technology will likely help individuals become more proficient at using the devices, yet as with playing a piano, it takes many hours of practice to become a skilled pianist (Seiter, 2008).One of the key advantages of 1 to 1 initiatives is that the participants are able to take home the devices. It is easier to become proficient using a device that one has access to at home, to one that is limited to its use within the classroom setting. As argued by Seiter (2008) “There is an overestimation of access to computers in terms of economic class, and an underestimation of specific forms of cultural capital required to maintain the systems themselves and move beyond the casual, recreational uses of computers to those that might lead directly to well-paid employment” (Pg. 29). If Seiter (2008) is accurate, and most of the economic benefits from ICT use come from their long term use.

ICT investment can be very expensive and many of ICT projects could not be developed without the support of private industry and the government (Heshmati & Addison, 2003).  While ICT may not be as important as basic education, food, and health services, governments throughout the world have spent large quantities of funds in ICT for education initiatives hoping to imitate the success many advanced economies have obtained from their ICT industries and byproducts (MSC, 1996). “Investment in ICT infrastructure and skills helps to diversify economies from dependence on their natural-resource endowments and offsets some of the locational disadvantages of landlocked and geographically remote countries” (Heshmati & Addison, 2003, p. 5)

Adequately evaluating 1 to 1 technology adoption initiatives is increasingly important, as different education interventions can have different cost-effectiveness ratios, and cost-benefit ratios, with some interventions being much more effective than others (Yeh, 2011). Working with limited funds, governments must administer their funds in the best possible way to provide their citizens with the ability to meet their needs various needs, from food and shelter, to their self-actualization. Just because one intervention is more cost effective, does not mean that the other intervention should be necessarily discarded (Countries can implement multiple interventions if funds are available). As Abraham Maslow (1943) suggested, many needs can be and should be met simultaneously. While there is a clear hierarchy to human needs, an improvement in one area of life, such as shelter, does not occur in a vacuum, and is not exclusive from the individuals desire to feel accepted by others, or to improve their problem-solving ability (Maslow, 1943). Investing in ICT is important for states as they move towards becoming economically diverse, robust and more competitive, relying more in their human capital than their natural resources. To evaluate these projects more precisely; this paper encourages evaluators to consider conducting a mixed-method analysis and with a long-term time perspective.

Evaluating Information and Communication Technology Projects

Evaluation can help increase the effectiveness of programs and improve the effective distribution of the limited resources available to a society. The decisions made by an evaluator can impact the lives of many individuals. Evaluators can help improve a program as well as decide whether or not the program should be continued (Fitzpatrick, Sanders, & Worthen, 2011). Discussing the methodology of evaluation, Scriven (1967) differentiated between formative (focus on development and improvement [Cook tasking the soup]) and summative (focusing on whether the program is meeting its stated goals [Guest tasking the soup]) evaluation (Scriven, 1967). By conducting an evaluation a decision-making body is able to make an informed decision about the future of the program. Yet, dealing with complex programs with large numbers of pieces, and unique elements, it is difficult for an evaluator to frame an evaluation that can help them obtain the most valuable information about a program, particularly when there is a limited time to conduct it, and the brevity of a report can be one of its strengths (Krueger, 1986). Yet, different methods provide for different valuable lenses through which to look at a problem, frames that the evaluator should consider before conducting their evaluation.

Possibly the most important elements to consider in a 1 to 1 ICT project are its cost, and its use by the learners. The most known 1 to 1 idea is the One Laptop Per Child Program (OPLC) which has been most successful in Latin America delivering hundreds of thousands of units (http://one.laptop.org/). Yet with over $100 cost per student (closer to $200) it could cost $500 billion dollars to provide a computer to every person that currently lacks access to the internet worldwide ($5 billion people), and this would not include their continued maintenance and electricity cost or the cost to access the Internet. Is access to ICT really that important? According to a recent UNESCO (2012) publication while 1 to 1 laptop projects are very costly, in Latin America “in the last three years, the 1:1 model has become increasingly widespread, and 1:1programmes are now the primary focus of national policies for ICT in education in the region. Policy-makers are no longer discussing whether the 1:1 model is worthy of investment but rather how best to achieve it” (Lugo & Schurmann, 2012)

 

While a price tag of $100 appears as an expensive investment for developing countries, especially when some countries spend less than $100 per student a year within their educational budget, it is also important to consider that all programs have cost, even when they are not financial. Even 1 to 1 program that are “free” (through donations) have a cost, including an e-waste disposal cost. Even when they are based in volunteer efforts, programs still have as a minimum a lost opportunity cost for instructors and learners. The cost of programs can be most effectively asses by measuring their different ingredients. This allows programs to be quantified; for various elements to be weighted, and as a result for programs to be compared with each other through a cost-effectiveness analysis (Levin, 2001). The financial benefit of the program can also be determined through a cost-benefit analysis. Through a qualitative study, “thick”, rich descriptive information can be obtained and thematically organized helping key stakeholders to better understand elements that would otherwise go unnoticed (Geertz, 1973).

Programs can also be mapped through a logic model which can include inputs, activities, outputs, and outcomes (Alter & Murty, 1997; McLaughlin & & Jordan, 1999). The order in which the elements of a program are implemented and the context where a program is implemented may also influence the results of the program! There are also likely to be competing program alternatives some of which may be more effective than the particular program being considered. Hoping to increase the transferability or generalizability of a study, an evaluation can also be theory driven (Weiss, 1997). These and other elements, can improve the quality and usability of data obtained by an evaluation. However, with limited time and resources, the methodology used to evaluate a program depends on both the strengths of the researcher, and what is considered of principal importance by key stakeholders.

 

Overtime, every practicing evaluator is or is in the process of becoming a “connoisseur” (the art of appreciation), as well as a “critic” (the art of disclosure) (Eisner, 1994, p. 215). This knowledge allows him or her to more effectively propose to key stakeholders’ recommendations as to the best methods of evaluation to pursue in a particular scenario. However, the interests of secondary stakeholders are also important in many ICT adoption programs.

The Relevance of Mixed Methods and Triangulation

“The underlying rationale for mixed-methods inquiry is to understand more fully, to generate deeper and broader insights, to develop important knowledge claims that respect a wider range of interests and perspectives” (Greene & Caracelli, 1997, p. 7).

 

Mixed-methods can greatly benefit a study as they allow the researcher to ask questions that he or she may ignore otherwise, obtaining additional information. While “purists” oppose the use of mixed-methods due to potential epistemological and ontological contradictions, many evaluators take a more “pragmatic” approach to the use of mixed-method (Greene, Caracelli, & Graham, 1989). One of the concerns regarding the use mixed-methods is that they may compromise the methodological integrity of an experimental study. These are valid concerns, and it is important to consider carefully how methods are being utilized, to avoid unintended conflicts that jeopardize the integrity of the study. Some of the theoretical concerns for researchers against using mixed-methods may not be as applicable to evaluators as evaluators do not have the same goals as researchers. While researchers are focused to a greater extent on theory and generalizability and transferability, for many evaluators, their focus is on utilization and the practical implication of their analysis to their key stakeholders and the future of the program (Patton, 2007). To the “pragmatist” evaluator, “philosophical assumptions are logically independent and therefore can be mixed and matched, in conjunction with choices about methods, to achieve the combination most appropriate for a given inquiry problem. Moreover, these paradigm differences do not really matter very much to the practice” (Greene, Caracelli, & Graham, 1989, p. 8).

Mixed-methods, often refers to the use of methods of different paradigms, using both a qualitative method such as unstructured interviews or participant observations, with a quantitative method, such as academic achievement scores, or another statistical value within the same study (Johnson & Onwuegbuzie, 2004). While it seems beneficial to analyze a problem in multiple ways experts in both qualitative and quantitative methods express concerns against this approach. Johnson and Onwuegbuzie (2004) argued that part of “purist” concerns stems from “tendency among some researchers to treating epistemology and method as being synonymous” which is not necessarily the case (Pg. 15). To Johnson and Onwuegbuzie most researchers who use mixed-methods use them when they consider their use to be most appropriate. Johnson and Onwuegbuzie (2004) argue for a contingency theory of research approach which emphasizes that while no method is superior, there are instances when one is preferable to the other.

One of the biggest benefits of using mixed methods is that they allow for the triangulation of findings. According to Dezin (1978) Triangulation is “the combination of methodologies in the study of the same phenomenon” (pg 291). Dezin (1978) describes 4 types of triangulation: data triangulation, investigator triangulation, theory triangulation, and methodological triangulation. He described these as possible within-methods, or between-methods (Denzin, 1978). The ways in which methods are mixed varies, with both of them at times having the same amount of influence, while sometimes one method holds preeminence. Triangulation is a common way in which to strengthen the generalizability and transferability of a study and the strength of its claims.  Other benefits of using mixed-methods include a complementary, where the results of one method are clarified by another, development, when one method informs the other, expansion, trying to increase the scope of one methodology, and initiation, which seeks the discovery of paradox by recasting results or questions from one method to another (Greene, Caracelli, & Graham, 1989) Regardless of the initial results, it usually provides richer data. Comparisons between the data could lead to either “convergence, inconsistency, or contradiction” (Johnson, Onwuegbuzie, & Turner, 2007, p. 115).

If there is a conflict or an inconsistency within the data, it increases the difficulty in establishing a causal relationship and the study may require further study and explanation. This explanation can be provided by a form of structural corroboration, further analysis, or by sharing both findings with the key stakeholder, he or she can then use both pieces of information to make his or her decisions (Eisner, 1994). While most evaluators feel a responsibility to provide recommendations to the stakeholders, this recommendations do not necessarily have to address the contraction scientifically, rather a “connoisseur” may state that based on his experience, he or she believes which path may be the best path to follow. ICT adoption includes many invisible elements which increases the difficulty in evaluating them (Cobo & Moravec, 2011). Because of its complexity, it will be helpful for the evaluator to share his or her opinion as a “connoisseur”. Social programs are generally complex. By providing a focused report to the key stakeholders, that emphasizes the main findings of the mixed-methods evaluation, they will be more likely to make a good formative or summative decision. As will be illustrated, this was an objective pursued by to 1 to 1 iPad initiative at the University of Minnesota.

Encouraging the Long-Term Study of ICT Projects

 

The limited timeframe of a study can result in a restricted analysis. Iterative formative evaluations allow key stakeholders to constantly reevaluate ways in which to improvement a programs (Mirijamdotter, Somerville, & Holst, 2006). Iterative and continues evaluations are very important for internet based companies.  Google for example is known to regularly test new algorithms and versions of their search engine simultaneously with consumers, to obtain helpful usability comparisons. They try hundreds of variations of their search engine a year in an attempt to improve their product without customers noticing minor modifications and changes (Levy, 2010). Many other ICT firms regularly test new features. Similarly, many ICT adoption projects include an iterative process in their analysis, yet in the discussion of their findings, the evaluations regularly omits the potential long term benefits of the programs, focusing instead on short term costs and benefits.  While there time constrains and financial limitations to evaluations of 1 to 1 laptop programs, these evaluations would benefits from a stronger effort in measuring the long term benefits of the interventions, including cultural capital gain (Seiter, 2008).

Methodologies such as longitudinal studies, ethnographic research, and time-series are among the methodologies that can help illustrate the potential benefits from the long term analysis of an intervention. Some of these studies can be very expensive, but they allow for the observation of changes that would otherwise go unnoticed. Another recent example of the possibilities of looking at changes overtime was recently made possibly by the Google Books Project Ngram Viewer (http://books.google.com/ngrams). The NGram Viewer allows for words frequencies to be analyzed for over a span of 200 years! This type of study called Culturenomics is one of the newest ways in which an analysis of a subject over time provides an additional insight to an issue (Michel, et al., 2010). While the NGram Viewer is not very useful for evaluators, other forms longer-term analysis can be of greater support.

Ethnography is a field of study in which time spend on the field is an important validity variable. Ethnographers focus primarily on the quality of the data, which validity can be increased by the researcher if him or her has lived in a community for a longer time-frame and has obtain through this extended visit a greater understanding of the local culture. Some of the subtleties that are analyzed by ethnographers require time and involvement to be discovered. To some researchers, ethnography symbolizes a study that takes more than a year (Fuller, 2008). While some projects could last perhaps a single long day, other “projects are developed throughout the whole of a researcher’s life; an ethnography may become a long, episodic narrative” (Jeffrey & Troman, 2004). In quantitative analysis, a time series, as their name imply, also emphasizes the importance of collecting data over time. This set of statistical data can be collected at various intervals such as monthly for unemployment benefits data, or daily for the financial exchange rate, or monitoring an individual’s pulse over an exercise period, or even every 2 seconds for EGG brain wave activity. A commonly used and informative time series is population census data which is collected by many countries in regular intervals to help their governments better understand broader demographical changes, migratory patterns, and the future outlook of various variables (Zhang & Song, 2003).

Longitudinal studies can also be very helpful in understanding how an intervention at an early stage of a person’s development influences them throughout the rest of their lives. Various longitudinal studies have been conducted within early education to identify the changes these interventions may have in the lives of these individuals. Longitudinal studies include interventions pre-natal care, youth reading programs, or the observation of children as they become older, among many other studies. One of the most famous longitudinal studies of education was the Student/Teacher Achievement Ratio (STAR) Tennessee Class Size Reduction study which began in 1985 and which continued until 1999 (Finn & Achilles, 1999; Hanushek, 1999). The study tracked students who were assigned at random to kindergarten having between 13 and 17 students, or larger classes having between 22 and 26.  Over 6000 students took part in the study were they were kept in smaller classrooms for 4 years, and were continued to be monitored after the end of the intervention. The study found statistical significant changes to student achievement scores in three utilized measurements. The conclusions of this study strengthened claims regarding the positive impacts of class size reduction which encouraged the enactment of class reduction policies in California (1996) and other states. While other studies have contradicted the findings of the study, its use of an experimental design, its magnitude and its use of a longitudinal analysis strengthened its claims. There have been a number of important longitudinal studies in early childhood and other early interventions that have followed children development for decades (NCES , 2010). It is also used frequently within health sciences.

Another popular, long-term, longitudinal study is the British Up Series which has followed a group of 14 children since age seven in 1964, and is still under production. Similar documentaries have been replicated in Australia (since 1975), Belgium (1980-1990), Canada (1991-1993), Czech Republic (1980s), Germany (1961-2006), Denmark (From 2000), Japan (from 1992), Netherlands (from 1982), South Africa (from 1982), Sweden (from 1973), USSR (from 1990), USA (from 1991). While these long term studies can be expensive to conduct, they provide a different dimension to findings, a dimension that is sometimes not available in most 1 to 1 technology adoption evaluations.

 

The key benefit of including this dimension within an evaluation is due to the difficulty in knowing how the skills obtained from using new ICT devices will help an individual have the confidence and the background skills needed to develop future ICT skills competencies that may be beneficial to them in the job market. Will their familiarity with ICT at an early age, bring about broader benefits later in their lives? A short term outlook to an evaluation may at times provide a negatively skewed view of the impact of these projects, expecting more out of a pilot project than should be expected. In addition, it is common for program designers to overstate the potential outcomes of a project, expecting it to have a greater impact than it is likely possible. For example, as an evaluation of USAID basic educational projects (1990-2005) showed, most of its projects had less than a 4% in student achievement scores, despite the efforts of many specialists and the expenditure of millions of dollars. (Chapman & Quijada, 2007). One to one technology adoption projects can also be very expensive and as such can have a very negative cost-benefit analysis in the first years of the program. It is very important to take into account the rapid depreciation rate of CIT, but evaluations should also take into account, future, longer-term benefits of the investment.

Having access to personal computers is essential for most of the workforce in the 21st century. As argued by Seiter (2008), having a computer at home is almost a necessity for development competent skills in the subject. “The likelihood of gaining strong digital literacy skills on this type of machine [a computer lab] is much slimmer than on a home computer. In other words, learning to use computers at school is like the music education class in which you have forty minutes to hold an instrument in your hands once a week, along with thirty other kids” (Pg. 37).

 

Many of the computer programs that students may eventually learn to use will require them to invest dozens, hundreds, and perhaps thousands of hours mastering. In addition individuals, who are less familiar with computer, tend to be less confident about becoming proficient in using new programs (Mohammed, 2007). While a television and a radio, or a “feature” mobile phone may have a short learning curve, the same cannot be said of personal computers, the internet or smart phones. Each of which is complex to different extents. Digital literacy programs such as RIA can teach a digital immigrant a basic set of skills in 72 hours, but many more hours are needed for complex use of a personal computer or an internet capable device (http://www.ria.org.mx). Just learning how to type rapidly on a QWERTY keyboard will take many hours of practice.

By evaluating a project while considering its impact over a longer frame of time this article encourages the continued evaluation of a program over a number of years, on regular intervals, while providing recommendations, and reporting on the benefits and negative elements of the program as they are modified over time. This type of long term evaluation is best suited for an internal evaluator, or a combination or internal and external evaluators. When thinking of the cost of 1 to 1 programs over time, it is also important to keep in mind the rapid depreciation of technology. With the rapid depreciation of computer equipment, should 1 to 1 programs focus on purchasing the most up to date gadgets and tools? This is a question that is be best analyzed through the inclusion of a cost-effectiveness analysis which accounts for the depreciation of technologies.

A Case Study – University of Minnesota One iPad Per Student Initiative

As previously discussed, the evaluation of technology adoptions programs has tended to focus on a short-term analysis, without sufficiently addressing or discussing the importance of analyzing the implications of adoptions over a longer time spectrum. As advanced economies are increasingly fueled by the ownership of patents and new inventions, so to have other countries attempted to further develop these sectors (Heshmati & Addison, 2003). The information transferred through ICT can help countries develop into more diverse and sustainable economies. It is through ingenuity, creativity, innovation, or “Mindware”, that groups and individuals come together to form new industries and adapt to different types of crises (Cobo & Moravec, 2011). Via technology adoption programs, individuals can increasingly access the information that will help them develop valuable skills. By evaluating with a long-term focus, and incorporating both qualitative and quantitative elements to the evaluation, an evaluation will be better able to address the questions of key stakeholders. This paper illustrates the limitations and strengths of a recent evaluation of a one to one iPad initiative in the University of Minnesota.

One Laptop Per Child – An Evaluation of Peru’s Project

Possible the most controversial and also most commonly cited 1 to 1 initiative is the One Laptop Per Child (OLPC) initiative, which was started by Nicholas Negroponte, the founder of the MIT Media Lab (TED, 2008). According to Negroponte, by thinking in bytes instead of atoms, and by learning how to operate a computer, a child can learn that the world is increasingly available at the click of a button, and that they can construct and build anything that they can imagine by programing new and amazing environment (Negroponte, 1996). Following Paper’s Constructionism, Negroponte believes that programing teaches an individual not how to learn, as they must go back, revisit their code and figure out why there is a mistake (Papert, 1980). As an ICT evangelist, Negroponte highlighted how simply by giving a child a computer his possibilities would be expanded (Negroponte, 1996). Since the beginning of OLPC in 2005, over 2.5 million laptops have been delivered (http://one.laptop.org/about/faq). However, despite the high level of investment, particularly in Latin America, project evaluations have not shown significant gains in achievement scores (Cristia, Cueto, Ibarraran, Santiago, & Severin, 2012).

A recent evaluation of OLPC in Peru expressed how despite a high level of investment in these new machineries (902,000 laptops), and increasing the ratio of computers from 0.12 to 1.18, student performance in math and reading had not increased substantially. The project did find that students’ cognitive skills had improved over the time of the study (measured by Raven’s Progressive Matrices, a verbal fluency test and a Coding test). While analysts have since highlighted that the program had only limited effects on math and language achievement (0.003 standard deviations), little emphasis has been given to the potential impact of the improvement in cognitive skills, and perhaps more importantly what having improved their digital literacy skills will mean for this individuals in the future, as they are asked to learn other task specific digital and information literacy skills (Cristia, Cueto, Ibarraran, Santiago, & Severin, 2012). As mentioned by Seiter (2008) developing high level ICT may take many years to fully demonstrate themselves as marketable skills in the lives of students.

It is also difficult to know from the available data whether a different investment would have been more cost-effective or result in a higher cost-benefit ratio in Peru. One of the unmet goals of OLPC was to produce a $100 laptop; however they currently cost around $200 (Cristia, Cueto, Ibarraran, Santiago, & Severin, 2012). As a project which was not affiliated with Microsoft, Google or Apple, the OLPC laptops came with an operating system (OS) known as Sugar. While all operating systems share similarities, did the use of Linux Sugar limit or increase the possibilities for students. When testing student computer literacy skills, they found that the students quickly became more adept at using these devices. As explain earlier in this paper, they also had difficulties in deciding which skills should be tested (Cristia, Cueto, Ibarraran, Santiago, & Severin, 2012, p. 15). Unfortunately, another unmet goal of the project was that Peru’s OLPC participants lacked of internet connectivity. OLPC was partly designed so that students could benefit from increase connection either through OLPC exclusive Mesh network or the Internet. The impact of lacking access to the internet are hard to measure, however they may have affected the individuals’ development of their information literacy skills. In conclusion, Peru’s evaluation of the OLPC project was very insightful, while it contained a qualitative element; the project had a quantitative focus, limiting reader’s understanding of how the initiative affected individuals. As a project which centers on the individual, learning more of the project’s impact on the person is increasingly of relevance as ICT becomes more personalized. Apart from now discussing potential long-term gains, the evaluation also failed to mention the full cost of the devices. With the laptop only accounting for a tenth to a seventh of the total cost of the device, it is important to consider whether this is a cost-effective investment (Lugo & Schurmann, 2012). The evaluation would have benefited from a broader implementation of mixed methods in particularly in the qualitative-side, while also emphasizing these changes over a longer span of time. An element of time that is particularly important to first year initiatives is the teachers or instructor familiarity or learning curve, as they will slowly learn better ways in which to use the device and integrate them within the classroom.

University of Minnesota iPad Initiative

The discussion surrounding the digital divide is traditionally centered around on access to the internet and a personal computer, yet the rapid change of technologies leads us to question whether the divide will be centered on these devices in the future (Warschauer, 2008; Zickuhr & Smith, 2012). What role will smart phones, reality augmented glasses, 3D printers, or farther into the future nanotechnology implants signify in terms of the digital divide? (Kurzweil, 2000). A current technology that may further displace the purchase of paper books for K-12 and HE is e-reader technology, the most successful of which are the iPads (I, II, and III) and Amazon’s Kindle readers. A recent NDP report indicated that tablets may outsell laptop computers by 2016, expanding sales from 81.6 million units (2011) to 424.9 million units (2017) a year (Morphy, 2012). Will we then measure the digital divide in terms of who access and who doesn’t have access to an iPad?

Pilot projects in universities such as the University of Minnesota, the University of San Diego, Oberlin College and a few others have move forward into answering this question. While the first successful tablet, the iPad was released on April 2010, that same year, the University of Minnesota decided to purchase 447 units to provide a tablet to every CEHD student in the upcoming undergraduate cohort. It was one of the first major initiatives of its type in the country. Because of its uniqueness, and being an early adoption project, its evaluation was based partly on the conclusions obtained from previous 1 to 1 projects such as the OLPC initiative and Maine’s 1 to 1 statewide adoption program. However, as a device that was substantially different from previous ICT devices, the operationalization of NETS standards, and an in-depth analysis of their potential use has not been acutely studied (ISTE, 2008). So far, only a few articles have been published regarding the use of the iPad in the classroom (EDUCAUSE, 2011). To better understand the possible educational implications of the adoption of this technological device, a CEHD research team decided to conduct a mixed-methods evaluation (Wagoner, Hoover, & Ernst, 2012). In addition, an initial commitment was made to continue evaluating the project for a consecutive number of years. The support of the dean was integral in the continuation of the program.

The first year, the project consider as a goal to increase the usability of the devices by both faculty and students, and to provide aid to faculty members so that they could familiarize themselves with the devices and consider the best ways in which they could incorporate the devices within their classrooms. Faculty members were then encouraged to incorporate the devices as they best saw within their syllabus. Various graduate assistants as support staff.  Soon after the distribution of iPads, evaluators also drafted a post-test and organized a series of interviews. The interviews asked faculty members a number of questions, including how they learned to use their iPads, what were their plans for using them within the classroom, how the iPad had affected their teaching, and if the support received had been appropriate (From field notes).

A similar set of questions were asked to faculty members at the end of the school year, where they were asked what projects they had actually implemented, the opinions of students regarding ebooks, pedagogical concerns among others. Twenty two interviews were coded and themes were developed from the qualitative study including concerns from faculty about time investment, how the iPad compares with other technologies, the impact of the iPad to faculty members’ pedagogy, the impact of the iPad to their classroom management, and details about faculty members’ technology learning process. At the end of the year a series of faculty member focus groups were also conducted. Many of the details learned through the qualitative portion of the study would have been difficult to obtain otherwise. The common elements between the data from the focus groups and the interviews also allowed us to verify some observations. Below is an interesting quote from one of the participating faculty members:

“What I want, in terms of their behaviors, is for [the students] to be active explorers in the classroom, to bring the machines, and to actually utilize them for historical research … One of the things that we did as a first conversation is to describe the level of trust that is going to be involved … and they live up to those expectations. I’ve been really happy so far with what we’re learning. It conveys to them that they’re smart, capable discoverers that we’re co-creating knowledge—historical knowledge” (Wagoner, Hoover, & Ernst, 2012, p. 3)

While the quote above illustrates a very positive emotion, it is likely that this experience will not have been visible through an analysis of student achievement, illustrating the benefit of utilizing mixed-method. Two student focus groups were also conducted where they shared some of their favorite apps and how they had used the iPad through the semester, yet unlike faculty members where evaluators were able to interview the whole population, 447 students were more than the team could interview.

To obtain a better analysis of the student response, a survey was conducted which included a number of questions related to their use and experience with the iPad. The survey was responded by 241 CEHD first year students (Wagoner, Hoover, & Ernst, 2012). Having access to broader demographic data also allowed the evaluation team to compare student attitudes with socio-economic variables. Various strong correlations and significant relationships were found regarding the impacts of iPads to student learning. In particular the evaluation found that students felt that the devices had been a positive experience in terms of their motivation they also expressed having a high level of comfort using the devices and the iPad helped them feel more engage in some of their classes.

The study also showed that students which were part of Access to Success (ATS) or had been part of the TRIO program, usually students of color or from low socio-economic backgrounds mentioned feeling more engaged and connecting during classes. From the qualitative data the evaluators also learned that to some students the iPad had become a window into the internet, and a digital item for their whole household to use.

 

The success of the first year implementation, and the questions that evaluators were still unable to answer led to the continued of the program for a second and third year. A similar number of iPads (now iPads 2) were purchased the second year of the program. Once again the rapid change of technology provided new possibilities for evaluators, as iPad 2 include cameras which permitting students to record HD video and have audio-visual communications with anyone with access to Facetime or Skype, and other programs. After analyzing the potential savings the extensive use of iPads for e-reading by some students, CEHD also decided to support a pilot project for the testing and adoption of Open Textbooks, as well as the establishment of a work desk where faculty members could obtain assistance and build iBooks and ePubs if interested.

The project is now planning its third year. Adapting to the result of the first year evaluation, many of the questions of the second year survey were modified to find additional valuable information. One of the limitations of the evaluation of the program so far has been a lack of a cost-effectiveness or a cost-benefit study. Yet, such a study should not only take into account the rapid depreciation of the devices, but also consider if students are learning through the use of the devices skills that could potentially aid them when they join the workforce. While the cost have been high with over 300,000 dollars per year, it is difficult to assess the long term benefits for participants (students and faculty members). The rapid devaluation of the devices is an important consideration, as it may be possible that in a couple of years these devices will cost only a fifth of their original cost and be even more feature rich and powerful, allowing students to obtain a similar skill set for a fraction of the cost. It is also possible that many of the skills obtained are not very different from those obtained from using other ICTs, reducing the importance of the investment.

 

Currently, a website is available were individuals interested in the results of the project can learn various innovative classroom projects that were developed and how they can be adapted to other classrooms, as well as suggested best practices. Some of the innovative uses of the iPads by students include the creation of digital stories, accessing unique applications including interactive stories, data visualization, among others, as well as rapidly accessing websites, and developing an e-book library. In a report, CEHD concluded that the iPad had been helpful addressing the concerns of the Digital Divide, increasing access to the tools needed for media production, increased access to tools that facilitates personal productivity, improve students’ possibilities for information access and consumption, helped reduce the cost of printing readings, and facilitated students’ learning outside of the classroom (Wagoner, Hoover, & Ernst, 2012). For year two, the program also hopes to further analyze the usability of the devices and recently developed a space for students to submit their creative productions with the iPads.

Despite the insights provided by the use of mixed-methods for this evaluation, the limited timeframe of the study makes it difficult to determine whether or not is a worthwhile investment. With the program costing over $400 dollars per student, apart from the cost of the administrative staff, is this the best investment for a university to make in terms of technology adoption? When will it be determined that the program is no longer worth its cost and it is no longer helping to find innovative ways of learning? One of the limitations of CEHD’s 1 to1 iPad program has been the limited emphasis on the possibilities for the device within informal learning. Some of these concerns will be better analyzed from the data collected from the second year survey that was recently administered to students. A new wave of interviews and focus groups is also planned for the evaluation of the 3rd year of the program.

With 500,000 applications there are almost endless possibilities as to how the devices can be integrated within the classroom. The production of more apps that match more closely with the goals of each individual is likely to increase. Because of these devices future relevance, and the high level of creativity and innovation within this industry, constant evaluation of these devices is important as it allows for the continued improvement of the project. The use of mixed-methods allowed the evaluation team to find many interesting details that the study would not have found otherwise. These details enriched the quality of the findings and provided faculty with valuable information for the improvement of the use of the iPad and for learning how their peers were using the devices.

Conclusion

 

ICT 1 to 1 adoption projects are difficult to evaluate and the short-term focus of some evaluations results in a limited view of their potential impact. One of the difficulties in evaluating these programs results comes as a consequence for rapid technological change.

“You Decide” (Loans and Donation Assignment)  – 4/10/-2012

 

“Since October 2004, 169,000 scholarships have been distributed to girls and boys who are economically disadvantaged, disabled, orphaned, and/or affected or infected by HIV/AIDS.” – http://agsp.worlded.org/

With a growing world population and limited national resources, as well as conflict and many other crises happening around the world, and, it is not an unexpected consequence than only some of those in need will benefit from foreign aid programs. However, choosing sometimes between who eats, or who goes to school is a very difficult problem. We have here a number of potential aid beneficiaries. By clicking on their image you will be able to visit their profile. Taking into account what you consider most important, which three individuals will you fund? There is no right answer, just difficult choices.

[Text Box]

Criteria Use to Make a Decision

 

Assignment Details

In the table above there are 9 pictures of individuals who could benefit from foreign educational aid. Three of these individuals would benefit from a higher educational loan, three from assistance with secondary education, and three more with assistance with attending primary education. Unfortunately due to limited resources only some students can benefit.

The images above, the student profiles, are linked to pages on the internet that contain additional information about the subjects. These pages may no longer be available but similar students can be fund through institutions such as Kiva or Compassion International

Websites Used: www.Kiva.org and www.compasion.com.

The goal of the assignment is to discuss the biographical information of the 9 possible funding candidates. They all have different backgrounds and educational goals. After looking at their personal data, we are to then decide which individual will and will not be funded. Having access to the biographical data personalizes the simulation.

The idea would be to give every person three post-it notes (of three colors – ranking 1 to 3) and then after talking or discussing the idea in small groups (of 4 to 5 members) they could indicate who each group member would like to fund by placing the stickers by their names. Nine larger images will be placed in the whiteboard were people would be able to tag their post it notes.

There will be an additional area where people can also decide that they rather not fund a person and they rather give the money to a general fund. Personalizing foreign aid is a recent trend by aid organizations that are trying to further humanize through the use of technology a situation that is often reduced to statistics.

We hope you enjoy this assignment. Feel free to modify it as needed.

The URLs used for the assignment may become broke over time. Below we copied the description that was available on the website when the assignment was created.

………………Page Break………………

A loan of $850 helped Yasmeen to pay for her current semester fees.

 

Yasmeen is a 21-year-old single lady. She is the oldest of five sisters and a brother. After her father retired from the military, he started working for a company to be able to support his family and make sure that all of his daughters and his son complete their studies and education.

Yasmeen is the first to enter university. She is now in her third year, majoring in Financing and Banking and hoping to work in the banking field after graduation. She also wants to set a good example for her sisters, so she is trying her best to graduate early and find a job. She plans to later register to get an MBA. Now she has applied for this loan to pay for her current semester to be able to achieve her goals.

Additional information about this loan

About National Microfinance Bank

Jordan is a small country located in the heart of the Middle East’s Levant region. Regional instability has caused Jordan to be excluded from global investments for decades. Additionally, a lack of natural resources—including water and energy—places tremendous strain on the Jordanian economy, over 80% of which consists of small businesses, largely in the service industry. Despite their importance in the economy, many of these businesses are excluded from the banking sector because of their size. Microfinance institutions like National Microfinance Bank offer them an opportunity to enhance and expand their businesses. NMB also offers its services to other underprivileged peoples in Jordan, providing them with loans to pursue educational goals and improve their homes and livelihoods.

About the Country

Country: Jordan

Average Annual Income: $3,740

Currency: Jordan Dinars (JOD)

Exchange Rate: 0.7100 JOD = 1 USD

Field Partner:

National Microfinance Bank

Field Partner Due Diligence Type: Full Due

Diligence Field Partner Risk Rating: 3.5 out of 5

Time on Kiva: 12 months

Kiva Entrepreneurs: 732 Total Loans: $857,625

Interest & Fees are Charged Yes Portfolio Yield: 36.56%

 

 Profitability (Return on Assets): 9.19%

Average Loan Size (% of Per Capita Income): 15.04%

Delinquency Rate: 0.03%

Loans at Risk Rate: 0.58%

Default Rate: 8.03%

Currency Exchange Loss Rate: 0.00%

………………Page Break………………

 

A loan of $575 helped Sewar pay for her semester fees.

 

Sewar is a 21-year-old single woman. She has four brothers and sisters.

Sewar is in her last semester majoring in finance and banking. She has applied for this loan to be able to pay for her semester fees and graduate.

Additional information about this loan

About National Microfinance Bank

Jordan is a small country located in the heart of the Middle East’s Levant region. Regional instability has caused Jordan to be excluded from global investments for decades. Additionally, a lack of natural resources—including water and energy—places tremendous strain on the Jordanian economy, over 80% of which consists of small businesses, largely in the service industry. Despite their importance in the economy, many of these businesses are excluded from the banking sector because of their size. Microfinance institutions like National Microfinance Bank offer them an opportunity to enhance and expand their businesses. NMB also offers its services to other underprivileged peoples in Jordan, providing them with loans to pursue educational goals and improve their homes and livelihoods.

About the Country

Country: Jordan

Average Annual Income: $3,740

Currency: Jordan Dinars (JOD)

Exchange Rate: 0.7100 JOD = 1 USD

Field Partner

National Microfinance Bank

Field Partner Due Diligence Type: Full Due

Diligence Field Partner Risk Rating: 3.5 out of 5 Stars

Time on Kiva: 12 months

Kiva Entrepreneurs: 732

Total Loans: $857,625

Interest & Fees are Charged Yes Portfolio Yield: 36.56%

 

Profitability (Return on Assets): 9.19%

Average Loan Size (% of Per Capita Income): 15.04%

Delinquency Rate: 0.03%

Loans at Risk Rate: 0.58%

Default Rate: 8.03%

Currency Exchange Loss Rate: 0.00%

………………Page Break………………

 

A loan of $575 helped Samuel to purchase a laptop to help him with his projects, assignments and revision for exams.

 

Samuel is a student applying for a loan to purchase a laptop to assist in his studies at Strathmore University and this is his story:

I am Samuel from Nairobi, Kenya. I am from a family of six: that is my parents, my two brothers and a sister. I am a second year student at Strathmore University pursuing a Bachelor in Business and Information Technology degree. I chose to come to Strathmore because of its professionalism and expertise in teaching.

I am applying for a laptop loan because the course is getting to be more demanding. There are projects and assignments to be done and I have to read for exams. Currently, I only access my coursework when am at the University. Therefore, having a laptop will be more convenient, because I can do my coursework at home and; hence, manage my time better.

Additional information about this loan

Important information

Strathmore University is a non-traditional Field Partner, in that they are an educational institution, not a microfinance institution. Lenders to these loans should be aware of the following:

1 – You might not begin to receive repayments on this loan for many years, and this loan may have terms as long as 11 years. The full-tuition loans administered by Strathmore University can have loan terms of up to 11 years, which includes a grace period of up to 5 years. The grace period includes the time during which the student is enrolled at the university, and the first year after graduation. During the grace period, the borrower is not required to make any repayments, and you would not receive any repayments. Please be sure the loan terms and repayment schedule of this loan suit you.

2 – Administering loans is a new activity for Strathmore University, which can increase the risk on this loan. While most Kiva Field Partners are microfinance institutions and therefore experienced at administering loans, Strathmore University’s core business is providing education, and they are inexperienced at administering loans. This partner has passed Kiva’s due diligence requirements for a non-traditional Field Partner, however lenders to this loan should be aware that this loan carries some additional risk as Strathmore University becomes more experienced with administering loans to students.

For more information on Strathmore University, and the Kiva loans they are administering, please visit the Field Partner page.

About the Country

Country: Kenya

Average Annual Income: $1,445

Currency: Kenya Shillings (KES)

Exchange Rate: 83.1000 KES = 1 USD

Field Partner

Strathmore University

Field Partner Due Diligence Type: Full Due Diligence

Field Partner Risk Rating: Non-Rated

Time on Kiva: 2 months Kiva

Entrepreneurs: 25 Total Loans: $43,075

Interest & Fees are Charged Yes Portfolio Yield: N/A

 

Profitability (Return on Assets): 0%

Average Loan Size (% of Per Capita Income): N/A

Delinquency Rate: 0.00%

Loans at Risk Rate: 0.00%

Default Rate: 0.00%

Currency Exchange Loss Rate: 0.00%

………………Page Break………………

 

 

Vijay Singh

 

Age: 16

Birthday: June 30, 1995

Gender: Male

Location: India

Center: MTU Vidya Kendra Student Center

Child ID: IN2250026

 

Support Needed

 

$38.00 per Month

 

Overview

 

Vijay lives with his father and his mother. He is responsible for gardening, running errands and cleaning. His father is sometimes employed as a seller in the market and his mother maintains the home. There are 2 children in the family.

 

As part of Compassion’s ministry, Vijay participates in church activities, Bible class and Vacation Bible School. He is also in high school where his performance is average. Art, listening to music and hide-and-seek are his favorite activities.

 

Because of your sponsorship, Vijay will have new opportunities to learn and grow physically, mentally, and spiritually. Thank you for your concern and prayers.

 

Community

 

Vijay lives on the plains of New Ashok Nagar, home to approximately 16,200 residents. Typical houses are constructed of cement and have corrugated iron roofs. The primary ethnic groups are Bojpuri, Bengali and Bihari and the most commonly spoken languages are Hindi and Punjabi..

 

The regional diet consists of beans, bananas, chicken, bread, potatoes and fish. Common health problems in this area include malnutrition, skin diseases, typhoid fever, cholera and jaundice. Half the adults are unemployed but some work as day laborers or rickshaw pullers and earn the equivalent of $60 per month.

 

Your sponsorship allows the staff of MTU Vidya Kendra Student Center to provide Vijay with Bible teaching, nutritious food, health checkups, sports, field trips, special celebrations, income-generating programs and counseling. The center staff will also provide adult literacy programs, medical checkups and meetings for the parents or guardians of Vijay

 

Country

 

India, the world’s largest democracy, has a tropical climate in the south and a more temperate climate in the north. The cool season is from November to March; the dry, hot season is from March to June; and the weather is hot and rainy the rest of the year.

 

In southern India, languages spoken are Tamil, Telugu, Malayalam and Kannada. In this region Christianity dates back to the first century when St. Thomas the apostle landed in Kerala and established a church there. He was eventually martyred in Chennai, where Compassion’s southern office is located.

 

India is home to one of the world’s oldest urban civilizations, dating to about 3500 B.C. This civilization declined around 1500 B.C., and soon after came the Aryan invasion and the gradual establishment of Aryan rule. In 1192, Arabs invaded, bringing with them the Islamic faith. British influence came with trading companies in the 1600s. British rule over India was established in 1757 and lasted until 1947, when a peaceful independence movement, made famous by Mohandas Gandhi, achieved its goal. The country now has a federal republic, and Pratibha Patil was elected the first female president of India in 2007.

 

India’s diverse economy encompasses traditional village farming, modern agriculture, handicrafts, a wide range of modern industries, and a multitude of services. The country has capitalized on its large numbers of well-educated people skilled in the English language to become a major exporter of software services and workers.

………………Page Break………………

Ivelise Bigil Altido

 

Age: 12

Birthday: February 21, 2000

Gender: Female

Location: Dominican Republic

Center: Centro de Desarrollo Infantil Educando Para El Reino

Child ID: DR1630221

 

Support Needed

 

$38.00 per Month

 

Overview

 

Ivelise lives with her father. She is responsible for caring for children, making beds and cleaning. Her father is sometimes employed as a farmer. There are 3 children in the family.

 

For fun, Ivelise enjoys playing with dolls, running and playing group games. She attends church activities, Bible class and Vacation Bible School regularly and is in primary school where her performance is average.

 

Your love and support will help Ivelise to receive the assistance she needs to develop her potential. Please pray for her.

 

Community

 

Ivelise lives on the plains of La Luisa Blanca, home to approximately 3,000 residents. Typical houses are constructed of cement floors, wood walls and zinc roofs. The regional diet consists of chicken, bread and rice.

 

Common health problems in this area include parasites, malnourishment and skin diseases. Most adults in La Luisa Blanca work on plantations and earn the equivalent of $121 per month. This community needs employment opportunities, basic services and recreation facilities.

 

Your sponsorship allows the staff of Centro de Desarrollo Infantil Educando Para El Reino to provide Ivelise with Bible teaching, medical and dental care, field trips, summer camps, music classes and academic support. The center staff will also provide literacy classes and vocational training for the parents or guardians or Ivelise.

 

Country

 

The Dominican Republic occupies the eastern two-thirds of the island of Hispaniola, which it shares with Haiti. It has a tropical climate, but moist, year-round trade winds keep temperatures between 72 and 83 degrees.

 

Nearly three out of four Dominicans have both black and Caucasian ancestry. Spanish is the official language, and 95 percent of Dominicans are Catholic. Nearly 90 percent of Dominicans live in rural areas where unemployment is high and malnutrition is widespread. A family’s diet consists mainly of rice, beans and chicken. Though agriculture was long the economic mainstay, in recent years growth in tourism and free-trade zones has made the service sector the country’s largest employer. Compassion works with children in nearly every region of the country.

 

The Taíno people were the country’s original inhabitants. In 1492, they welcomed Christopher Columbus in his first voyage to the island, but subsequent colonizers were brutal, reducing the Taíno population from about 1 million to about 500 in 50 years. Hispaniola became the center of Spanish rule in the West Indies. The indigenous people were wiped out and slaves were brought from Africa to populate the island. The descendants of those slaves form most of the population today. For three centuries, Spain governed Hispaniola, followed by France. In 1804, the western part of the island won independence as the Republic of Haiti. In 1844, the eastern two-thirds of the island revolted and formed the Dominican Republic. In 2004, Leonel Antonio Fernández Reyna was elected to his second term as president.

………………Page Break………………

 

Willinton Ramon Garcia Cedeño

 

Age: 15

Birthday: September 4, 1996

Gender: Male

Location: Ecuador

Center: Estrella de Jacob Student Center

Child ID: EC3732461

 

Support Needed

 

$38.00 per Month

 

Overview

 

Willi lives with his father and his mother. His father is sometimes employed as a seller in the market and his mother maintains the home. Willi works at home running errands. There are 2 children in the family.

 

For fun, Willi enjoys soccer, swimming and singing. He attends church activities regularly and is in high school where his performance is average.

 

Please remember Willi in your prayers. Your love and support will help him to receive the assistance he needs to grow and develop.

 

Community

 

Willi lives in the hillside community of 15 de Abril, home to approximately 7,200 residents. Typical houses are constructed of bamboo and have corrugated tin roofs. The most commonly spoken language is Spanish.

 

The regional diet consists of beans, bananas, bread, chicken, fish, beef, plantains, potatoes and rice. Common health problems in this area include dengue fever, typhoid, cholera, intestinal infections, asthma, malaria, chicken pox, mumps and rotavirus. Most adults are unemployed but some work merchants, fisherman, or day laborers and earn the equivalent of $240 per month. This community needs scholastic materials and employment opportunities.

 

Your sponsorship allows the staff of Estrella de Jacob Student Center to provide Willi with Bible classes, health education, sports, field trips, academic support, educational classes and homework help. The center staff will also provide parents’ school for the parents or guardians of Willi.

 

Country

 

Straddling the equator, Ecuador has two Andes mountain ranges that split it into three zones: the western coastal lowlands, the central Andean highlands and the eastern jungles of the Amazon basin. The lowlands and islands are hot and humid and the highlands are temperate.

 

The Ecuadorian population is about 25 percent Amerindian and 65 percent mestizo (Amerindian and Caucasian). The remainder is of Spanish or African descent. Most people live in urban settings. Spanish is the official language but many Indians speak Quechua, the language of the Incas, and practice traditional religions. Ninety-five percent of Ecuadorians are Catholic. Compassion works throughout central and western Ecuador.

 

Spanish conquistador Francisco Pizarro invaded Ecuador, home of the Inca Empire, in 1532 and controlled it within two years. In 1822, Ecuador gained freedom as part of a federation known as Gran Colombia. In 1830, it gained independence as Ecuador.

 

In recent decades, Ecuador’s economy has relied heavily on oil export revenue, so fluctuations in world market prices have a significant economic impact. A drop in world oil prices combined with natural disasters in the late 1990s to drive Ecuador’s economy into poverty. In 2000, Congress enacted reforms and adopted the U.S. dollar as legal tender, which helped stabilize the economy. In recent years, however, economic reforms have been reversed, making Ecuador again vulnerable to oil price swings and financial crises. And though Ecuador marked 25 years of civilian governance in 2004, it has been troubled by political instability, including the ouster of the last three democratically elected presidents. Rafael Correa is the current president.

 

………………Page Break………………

 

 

Karla Beatriz Guerra Guerra

 

Age: 6

Birthday: July 3, 2005

Gender: Female

Location: El Salvador

Center: Unidos por Jesucristo Student Center

Child ID: ES9000169

 

Support Needed

 

$38.00 per Month

 

Overview

 

In her home, Karla helps by helping in the kitchen and running errands. She lives with her father and her mother. Her father is sometimes employed as a farmer and her mother maintains the home.

 

Karla is not presently attending school. Singing, playing with dolls and hide-and-seek are her favorite activities. She also attends church activities and Bible class regularly.

 

Please remember Karla in your prayers. Your love and support will help her to receive the assistance she needs to grow and develop.

 

Community

 

Karla lives on the hills of Canton Guachipilin, home to approximately 5,000 residents. Typical houses are constructed of cement floors, adobe walls and tin roofs. The primary ethnic group is Mestizo and the most commonly spoken language is Spanish.

 

The regional diet consists of maize, beans, rice and potatoes. Common health problems in this area include dengue, diarrhea and fevers. Most adults in Canton Guachipilin are unemployed but some work as day laborers and earn the equivalent of $75 per month. This community needs employment opportunities and educational programs.

 

Your sponsorship allows the staff of Unidos por Jesucristo Student Center to provide Karla with Bible teaching, medical and dental checkups, tutoring, hikes and summer school. Thirty percent of children in this project do not attend school as it is not required before the age of seven in El Salvador. The center staff will also provide meetings and devotionals for the parents or guardians of Karla

 

Country

 

 

Most Salvadorans live in the country’s subtropical central region of valleys and plateaus. A hot coastal plain is in the south and the north is mountainous. Nearly all the land is farmed; coffee is grown in the mountains.

 

Ninety percent of El Salvador’s population is of mixed Indian and Spanish descent; about 60 percent live in urban areas. Although a few Indians have retained many of their old customs and traditions, most people have adopted the Spanish language and culture. Fifty-seven percent of Salvadorans are Catholic but there is extensive activity by Protestant groups and over twenty-one percent are Protestants. Compassion works in the central, southern and western regions of the country.

 

El Salvador is the smallest and most densely populated country in Central America. Spain claimed the country in 1525. Almost 300 years later, El Salvador declared its independence and, after a brief union with Mexico, joined four other Central American countries to form a federation. When this union ended in 1838, El Salvador became a republic. The country experienced a period of increasing unrest during the 1970s that culminated in a civil war in the 1980s. In 1992, a peace agreement was reached.

 

………………Page Break………………

 

Jordy Manuel Sifontes Barrientos

 

Age: 8

Birthday: January 26, 2004

Gender: Male

Location: El Salvador

Center: Monte Gilboa Student Center

Child ID: ES7390323

 

Support Needed

 

$38.00 per Month

 

Overview

 

In his home, Jordy helps by running errands. He lives with his father and his mother. His father is sometimes employed as a laborer and his mother maintains the home. There are 2 children in the family.

 

Soccer and bicycling are Jordy’s favorite activities. In primary school his performance is average and he also regularly attends church activities and Bible class.

 

Please remember Jordy in your prayers. Your love and support will help him to receive the assistance he needs to grow and develop.

 

Community

 

Jordy lives in the hills of Candelaria de La Frontera, home to approximately 800 residents. Typical houses are constructed of cement floors, adobe walls and tile roofs. The regional diet consists of beans, maize and bread.

 

Common health problems in this area include malnutrition, respiratory diseases and diarrhea. Most adults in Candelaria de La Frontera are unemployed but some work as subsistence farmers and earn the equivalent of $40 per month. This community needs educational materials, employment opportunities and alcohol abuse prevention programs.

 

Your sponsorship allows the staff of Monte Gilboa Student Center to provide Jordy with Bible teaching, medical checkups, nutritious food, health and hygiene education, recreational activities, academic support and vocational training. In addition, non-schooled children receive developmental activities. The center staff will also provide meetings, parents’ school and social activities for the parents or guardians of Jordy.

 

Country

 

Most Salvadorans live in the country’s subtropical central region of valleys and plateaus. A hot coastal plain is in the south and the north is mountainous. Nearly all the land is farmed; coffee is grown in the mountains.

 

Ninety percent of El Salvador’s population is of mixed Indian and Spanish descent; about 60 percent live in urban areas. Although a few Indians have retained many of their old customs and traditions, most people have adopted the Spanish language and culture. Fifty-seven percent of Salvadorans are Catholic but there is extensive activity by Protestant groups and over twenty-one percent are Protestants. Compassion works in the central, southern and western regions of the country.

 

El Salvador is the smallest and most densely populated country in Central America. Spain claimed the country in 1525. Almost 300 years later, El Salvador declared its independence and, after a brief union with Mexico, joined four other Central American countries to form a federation. When this union ended in 1838, El Salvador became a republic. The country experienced a period of increasing unrest during the 1970s that culminated in a civil war in the 1980s. In 1992, a peace agreement was reached

 

………………Page Break………………

 

Nathalie Akossiwa Koudaya

 

Age: 8

Birthday: July 20, 2003

Gender: Female

Location: Togo

Center: Centre de Dévélopement des Enfants Sinai Tabligbo

Child ID: TG8080136

 

Support Needed

 

$38.00 per Month

 

Overview

 

In her home, Nathalie helps by carrying water, running errands and cleaning. She lives with her father and her mother. Her father is sometimes employed as a farmer and her mother is sometimes employed as a farmer. There are 4 children in the family.

 

Playing group games is Nathalie’s favorite activity. In primary school her performance is below average and she also regularly attends church activities and Bible class.

 

Because of your sponsorship, Nathalie will have new opportunities to learn and grow physically, mentally, and spiritually. Thank you for your concern and prayers.

 

Community

 

Nathalie lives on the plains of Tabligbo Akpadjavi kondji, home to approximately 6,300 residents. Typical houses are constructed of cement floors, brick walls and corrugated iron roofs. The primary ethnic group and spoken language is Ouatchi.

 

The regional diet consists of maize, beans, bananas, chicken, fish, bread, cassava, rice and potatoes. Common health problems in this area include malaria and worms. Half the adults in Tabligbo Akpadjavi kondji work as street vendors, subsistent farmers or market traders and earn the equivalent of $22 per month. This community needs a cultural center and proper sanitation.

 

Your sponsorship allows the staff of Centre de Dévélopement des Enfants Sinai Tabligbo to provide Nathalie with Bible teaching, retreats, hygiene and nutrition lessons, field trips, community work, tutoring and educational materials. The center staff will also provide training for the parents or guardians of Nathalie.

 

Country

 

Togo is a small country located in Western Africa. Despite its size, it includes several distinct geographic regions: gently rolling savanna in north; hills in the central part of the country; southern plateau; and low coastal plain with many lagoons and marshes. The climate is as diverse as the geography: tropical in the south to semi-arid in the north.

 

The people of Togo are made up of more than 37 different African tribes, the most prevalent being Ewe, Mina, and Kabre. French is the official language, along with four regional African languages. More than half of the population practices a religion made up of indigenous beliefs, while the remaining are split between Christianity and Islam. Togo’s economy is based mostly on commercial and subsistence agriculture, with cotton being the most important cash crop. Cocoa and coffee are exported as well. Togo is also the world’s fourth-largest producer of phosphate. Economic instability has lead to 32% of the population in Togo living below the poverty line.

 

French Togoland became Togo in 1960. Beginning in 1967, Togo was ruled by a military general whose Rally of the Togolese People (RPT) party has maintained power almost continually since that time. The RPT still maintains a majority of seats in today’s legislature. Upon the president’s death in 2005, the military installed his son and then engineered his formal election two months later, beginning Togo’s transition to a democracy and leading to its first legitimate elections in October 2007. While Togo has a long history of political unrest and has been accused numerous times of human rights violations, recently it has achieved some political stability and recognition in the international community.

 

Qualitative Research and Subjectivity:

Second Discussion of Qualitative Approaches (April 10)

Reading for the Week:

Eisner, E. W. (1994). The forms and functions of educational connoisseurship and educational criticism. In The educational imagination: On the design and evaluation of school programs (3rd ed.) (pp. 212-249). New York: Macmillan.

Eisner, E. W. (1991). Taking a second look: Educational connoisseurship revisited.

In M. W. McLaughlin & D. C. Phillips (Eds.), Ninetieth yearbook of the National Society for the Study of Education, Part II. Evaluation and education: At quarter century (pp. 169-187). Chicago, IL: The University of Chicago Press.

 Hemwall, M. K. (1991). Ethnography as evaluation: Hearing-impaired students in the mainstream. In D. M. Fetterman (Ed.), Ethnography in educational evaluation (pp. 133-152). Beverly Hills, CA: Sage Publications.

Pitman, M. A., & Dobbert, M. L. (1986). The use of explicit anthropological theory in educational evaluation: A case study. In D. M. Fetterman & M. A. Pitman (Eds.), Educational evaluation: Ethnography in theory, practice, and politics (pp. 78-100). Beverly Hills, CA: Sage Publications.

 

 

Through the use of the scientific methods, modern natural science has produced thousands of findings that have reshaped the path of modern man, yet man itself remains an enigma. We are yet to fully understand some of the questions posed by the complexity of the human mind and the endless variability of individuals’ perceptions and understandings of phenomena. Since no two people are alike and can be fully comprehended it is misleading and reductionist to reduce us to symbol or a number. If to Eisner the beauty of a simple rock can be studied for a lifetime, can anyone truly understand a human being? A qualitative study of education provides a glimpse into the complexity of the person.

 

General conclusions are sometimes hard to draw from a qualitative study because they focus on large quantities of information which are particular to a situation, a school, a program, or a person. As more and more information is produced by human being, there is an eagerness, a desire to organize data through algorithms and equations, to find trends and patterns. But when we do so, when we reduce reality, what do we lose in the process?

 

When reading the papers for this week, these are some of the questions that intrigued me. I hope you also find them interesting. Looking at accepted ethnographic methods, is there a best way in which to conduct educational ethnographies? What is the role of theories in conducting a high quality educational evaluation? Can we generalize from an individual’s auto-ethnography and, from it, gain a better understanding of other people? What is the role of variability? Who is a connoisseur and how important is it to be able to tell a “good” story and explain knowledge in both discursive and non-discursive ways? Finally, given the subjectivity of a qualitative study, can we trust this information, and to what extent is it valid? These are some of the questions that are of importance to a qualitative researcher. This week’s readings discussed some of these concerns, as well as the value of qualitative research to the field of educational evaluations.

 

The most significant articles discussed this week were Eisner’s (1994) The Forms And Functions Of Educational Connoisseurship And Educational Criticism, where he discusses the concepts of connoisseurship and critics, and Eisner’s (1991) Taking a second look: Educational connoisseurship revisited. Eisner, who both studied art and social sciences, is well versed in the subject of aesthetics and art and, from his work at Stanford, he shares with the reader the importance of understanding a subject beyond its description. He emphasizes the importance of having a good perception of an object so that we can better understand the qualities of a project that would otherwise be lost through variable-level analysis. Eisner (1994) explained how it often takes many years for a person to become a connoisseur, a term referring to an individual with a high level of expertise and an accurate perception of a subject.

 

From his work as an artist, he believes that connoisseurship is a skill that comes to some more naturally than to others, but that through time and the analysis of subjects to minute detail, individuals can obtain a better appreciation of a subject (be it schools, music, poetry, swimming, etc.). By doing so they are more capable of evaluating a subject and later using that knowledge (transferring) to better understand similar projects. He explains the process by dividing a critic’s task into four components: description, interpretation, evaluation, and thematic.

 

Due to the difficulty of developing a good perception Eisner argues that time is not the only element that should be considered in becoming a connoisseur. Most individuals working within a field develop a better understanding of the field over time. To some extent teachers, students, administrators, and parents all have a good understanding of a schools, but they likely do not have the fine-tuned perceptions skills of a  connoisseur and the storytelling skills of a critic. As stated by Dewey (1934), recognition is perception aborted (Pg. 324). To Eisner (1994) “connoisseurship is the art of appreciation”, while “criticism is the art of disclosure” (Pg. 215).

 

Implicitly, Eisner (1994, 1991) argues that to some extent being a critic is akin to being a good storyteller. A good storyteller understand the elements of a good story, and he or she will help you to feel those unspeakable emotions, understand that which is poetic and beyond the scientific. In this sense each critic has a unique voice and it is therefore very important for a critic to use his or her powers of persuasion in honesty and truthfulness, according to his or her perception. “Criticism is the art of disclosing the qualities of events or objects that connoisseurship perceives” (Pg. 219).

 

In illustrating how critics address validity questions, Eisner (1994) mentions how critics are like lawyers putting the pieces of a case together in an attempt to convince their jurors. The attorneys, through a process of placing all of the pieces in order, also known as “structural corroboration”, are able to convince the court of their version of events. However, because of the co-constructed nature of knowledge, and the current psychological understanding that we recreate events when we remember events (changing elements of our stories), Eisner (1994) also acknowledges that there may be multiple versions of events and, depending on how divided different possible “structural corroborations” are, this method may not be sufficient to ascertain the validity of the critique. When thinking of connoisseurs he mentions how critics mainly agree on most elements (something somewhat true if we look at Rotten Tomatoes). Yet, if we consider Supreme Court justices as connoisseurs then there are clear examples where experts in a field (and it would be hard to argue that the Justices are not connoisseurs and critics of their field), after deliberating and deconstructing cases may disagree and vote 5 to 4. With this example, can we then argue that structural corroboration may not be enough to increase validity?

 

The other element Eisner (1994) discusses for increasing validity is referential adequacy. This refers to the process of “testing the criticism against the phenomena it seeks to describe, interpret and evaluate” (Pg 241). This process is sometimes done by going back to the site of study or the data set and testing the findings with a portion of the data that is part of the larger data set but a part that was not previously analyzed. If the framework or the analysis previously developed also applies, then the critique in question would increase in validity.

 

Another topic discussed by Eisner (1994) is the generalizability of critiques. While some qualitative researchers emphasize transferability over generalizability, Eisner (1991, 1994) argues that some form of generalizability is possible. Unlike a mathematical understanding based on laws, theorems and axioms, according to Eisner (1994) qualitative studies can be generalized as they allow the critic to apply the skills learned from becoming a connoisseur to other related subjects as one is able to appreciate them with a better or improved perception. Eisner (1994) discusses how generalization is also possible by the acquisition of “new forms of anticipation” (Pg. 242). By understanding the complexities of a particular situation, we can then better understand other situations. According to Eisner’s definition, generalizability of critiques appears to be internal (within the critic) rather than external (within his work), as it is the critic alone who can critique according to his own subjectivity.

 

Another critic may have a similar opinion about a subject but it will be somewhat different subjectively and in its critique. While Eisner argues that critics generally agree about broad conclusions, would they agree with all of the words in a descriptive paragraph that discusses Paul Brach pictures? This is unlikely, as it is improbable that two individuals, two connoisseurs, will criticize a subject in the same manner. While each analysis may have its own strengths, complications when generalizing the data raise questions about the claims of Pittman and Dobbert (1986) regarding the design of a “true” ethnographic educational evaluation.

 

Their use of the word “true”, is, in my opinion, somewhat controversial even taking into account the use of quotations, as it implies that there is possibly a best or an ideal form of applying anthropological theory in educational evaluation. It seems as if in this way we get closer to fully deciphering the true figure of the shadow emanating from the cave. It can be argued that Pitman and Dobbert (1986) have a more objective view of reality than other ethnographers. However, despite this concern, their inclusion and valuation of theory as an essential element in educational evaluation raises some important questions. In including theory as a primary element and illustrating its benefits effectively in a teacher training program in a day care center, the authors strengthened their claims. Yet, they also illustrated some major limitations.

 

The main critique to their design is discussed by Pitman and Dobbert (1986) as they state that “a traditional (i.e. Mallnowskian) approach was not possible” (Pg. 79) and explain that they were only able to study the school during a month long study. Yet because of the rigor of their evaluation, and each evaluator having spent 120 hours per person for around a month (April 24 to May 22) conducting the study, the authors have a strong claim that the study did include the level of quality required of an ethnographic evaluation.

 

The authors emphasize the importance of the explicit use of theory in conducting education evaluations. They illustrate their case by citing the importance of the literature review in noticing patterns and comparing and contrasting the study findings to various theories including symbolic interactionism, cognitive anthropological approaches, rites of passage theory, small group theory, gearing notion of old hands and new hands in transaction, role theory, among others. The value of these theories in increasing the validity of the study was illustrated in various examples including how “relying on Homans’s and Herzberg’s theories, [they] recommended that the center take an additional step to create positive student perceptions by distributing a handout that would provide solid information about its policies and philosophies.” (Pg. 94) The article contains various instances in which theory informed the findings, increasing their external validity.

 

Having a team of 6 members also increased the rigor of their evaluation. The steps taken through planning included testing a procedural outline, identifying and refining a manageable team evaluation process, and lobbying for the increased use of qualitative methodology. (They do not share their value premises for these and other decisions in their study). By conducting secondary, and sometimes triple interviews, as well as dividing into subgroups to analyze the data and conduct a pattern analysis, the study increases the validity of their findings, and potential impact (transferability and generalizability) of their conclusions. The study also had a high level of utility as the recommendations of the study were mostly implemented soon afterwards, likely improving the program’s outlook for future years.

 

The findings of the study have been reinforced by other studies, with limited time available for instructors being one of the most common findings. Despite the best attempts of the lead teachers to serve as mentors and guides to the practicum students, they had limited time to communicate effectively with the students and address their concerns. One of the biggest concerns expressed by practicum students was the lack of awareness of how they were being evaluated and whether or not they were being seen as successful in completing their tasks. The teachers, on the other hand, argued that while not all students were willing to listen to their advice, some of them were very attentive to comments and criticisms. However, despite some of the differences of opinion, the students seemed to agree that “the lead teachers were very responsible and related well with children and their needs” (pg 88). While the authors argued that all students agreed with this statement, the individual’s political position is not discussed as it in Hemwall’s (1991) study.

 

Pitman and Dobbert’s (1986) article was very helpful in illustrating how a team of researchers can conduct an in-depth study within a short time frame. Yet is it possible for most evaluations to assemble a team and develop an extensive system for member checking and triangulation? In addition, in the end of the paper, the authors argue that “in this chapter we have sketched the distinguishing characteristics of “true” ethnography versus other kinds of fieldwork” (Pg. 97). Despite discussing their methodology as a “true” ethnography, they do not include many of the essential elements of a feminist ethnography or a critical ethnography. This ethnography is one out of many possible types of ethnographic research, being “true” only to the extent that it is perceived as “true” by the author and its reader.

Hemwall (1991) was the fourth reading for the week and it discussed the mainstreaming or integration of a group of hearing impaired students, a program which is formatively evaluated by the use of classroom observations, short structured interviews, and distributing questionnaires. Similar to Pitman and Dobbert (1986), this study also took only a few days, which leads the researcher to question the richness of the study. Hemwall (1991) provides extensive examples of how, despite the interest in increasing the hearing impaired student’s mainstreaming, many of them had yet to be fully accepted by their peers, break their dependency on each other and on the interpreters, and to develop a relationship that was similar to that of other teacher-instructor relationships. In her discussion of events, Hemwall saw the attachment of interpreters to students as a negative change which broke protocol. In addition, students who were not hearing impaired felt their hearing impaired classmates were being treated in a beneficial way by the instructor. “Aw, those guys just get away with murder!” (Pg. 141).

 

Hemwall (1991) cites examples of when the teacher spoke directly to the interpreter instead of trying to learn from the student in question. The student expressed frustration and decided not to ask a question to the instructor. An element in this paper that was missing to me was a discussion of the author’s opinions about mainstreaming, or her value premises. Hemwall (1991) also argued for the breaking up of the group of hearing impaired students, so that they could better relate to their non-hearing impaired classmates. The researcher feared that “the students in these groups had become so dependent on each other that the groups isolated themselves, thus preventing further integration.” (Pg 145). Yet her argument does not discuss how social integration of all students in high school can be difficult for many students, not only students that are hearing impaired. Her finding also seems to illustrate that many students felt marginalized by their peers.

 

The researcher concluded by providing a list of recommendations including: skill tutoring, establishment of better support systems, which would be ideally “a deaf adult who was previously mainstreamed” (pg 154), more informal academic and social experiences, separating the groups of hearing-impaired student. She highlighted the need for intensive in-servicing and training available for everyone, and lastly special training for interpreters, which despite its importance had been difficult to implement. To finalize, the article highlights the strengths of ethnographic research as well as its weaknesses. The three main weakness of the study were (1) obtaining access to the classrooms to observe; (2) the careful formulation of the results of the ethnographic evaluation to avoid any unintended consequences; and (3) a discussion as to how to adequately protect the informants.

 

To conclude, this article provides a close and detailed look about the experiences of mainstreaming students in this school while also raising questions about generalizability and transferability, as well as the subjectivity of the critic. One of my concerns when reading this study is its lack of a discussion of what is considered “normal”? Or is there such thing as a “normal” child? Is “normal” an objective term, or rather another social construct that is culturally dependent? The article is effective in explaining, as mentioned by the PE teacher, that hearing impaired students are as diverse a group as other students; “handicapped kids run the gamut like hearing kids… some brats, some good kids, some smart, some slow”.  This is important to highlight, but also that “normal” itself may be a misconception. Students have different learning modalities and living experiences, many of which are unique. While debating what is “normal” is beyond the extent of this paper, it is important to discuss it as a social construct and subjective concept.

 

 

Questions for Class Discussion:

 

1 – Would you consider Supreme Court justices to be connoisseurs and critics? Why or why not?

2 – Can a qualitative study be reproduced?

3 – Are aggregators of critics’ opinions better than a critic’s opinion?

4 – Is there an objective truth? Are there infinite ways of seeing things?

5 – Do you consider yourself to be normal? Why or why not?

6 – In Outliers, Gladwell argues that it takes 10,000 hours for someone to master a subject (with regular critical feedback), how is this related to Eisner’s concept of the connoisseur?

7 – What comes to mind when you think of an artistic teacher?

8 – Is a test fair (is testing fair)? What is fairer; a critic’s opinion, or a test?

9 – Is there a “true” ethnography? What are the components of a “true” ethnography?

10 – What is “Contemporary Time”? (Pittman and Dobbert)

11 – Three Cups of Tea’s author was recently involved in a controversy. His story, however, had been very captivating and felt “real”, leading to millions of dollars in donations. With this in mind, what is the responsibility, if any, of the storyteller to explain his experiences honestly?

12 – Are the 10 weeks mentioned in Pitman Dobbert sufficient time for an ethnographic study?

13 – Do using different qualitative methodologies lead to the same or different results? Would different experts yield different results with the same methodologies?