Media Viewing Habits Survey Report (12/8/11)

Elizabeth Hedin, Idris Kamara, Heather Keelon, Sarah Komperud,

Alfonso Sintjago, Maren Stoddard, Sarah Walker.

Introduction of Issues and Survey Development

In the non-stop world of entertainment, the ways in which we access and engage with entertainment is quickly evolving.  Our group developed a survey about Media Viewing Habits and a person’s change in habits and preferred media sources within the last five years.

Our group took a multi-faceted approach to developing this survey.  First, to frame our survey questions, we defined our target as U of M undergraduate and graduate students.  Then, as a group we discussed overall themes of questions and areas of interest within media.  It was decided that our survey should focus on the kinds of media preferred, the vehicle used to consume the media, how these two factors have changed in the past five years, and how living situations affect media viewing.  Next, each group member individually contributed five questions to the preliminary survey.  Existing surveys about similar subjects were used as references for ways to write questions in a manner that will yield the most valuable information.  Next, with all of the group members’ questions amalgamated, final questions were written, sometimes as a combination of several individual questions.  We gave great consideration to the order of the questions, trying to build upon the previous question in a logical progression.  The final survey, titled Media Viewing Habits, was administered to the class on November 17, 2011.

Analysis and Results

Closed-choice options were tabulated by SPSS (Statistical Package for the Social Sciences) to find frequencies and cross-tabulations, and open-choice options were coded to find prevalent themes.  We focused on differentiating the data by student status, grouping replies by undergraduate or graduate (including both Master’s and Ph.D. students) status, though a variety of other cross-tabulations are possible.

Although the sample size was not big enough to yield much statistical significance, we did discover some interesting relationships in our three main areas of questioning: media-viewing devices and methods, internet-based versus non-internet-based media, and the effect of living situation on media viewing.  Attached are data for the questions we found most interesting.

The first category of questions focused on media-viewing devices and methods. When asking respondents about media viewing devices, almost all stated they had access to television and a personal computer. Fewer respondents had access to smart phones, other mobile devices and gaming systems. Furthermore, when comparing which services respondents had personal access to today and what they had access to five years ago, the majority access to many of the services remained the same. However, there was an increase in access for services such as streaming video subscription services, digital video recorders, and smart phones. For example, only four respondents replied they had smart phone access five years ago when sixteen respondents currently have access.

When asked whether they preferred internet-based or non-internet-based media, most respondents replied that they had no preference, and of those who indicated a preference, it was split evenly between internet-based on non-internet-based.  Replies to the open-choice option indicate some of the reasons why respondents did or did not have a preference: those who preferred internet-based media tended to cite convenience, greater access, fewer commercials, and greater value for the money, whereas those who preferred non-internet-based media cited greater relaxation and problems with internet connections.  Those who indicated no preference responded that convenience was the biggest factor in determining how to view media, so they used a combination of internet and non-internet based media.  Responses to questions regarding the number of hours spent watching internet-based and non-internet-based media elucidate how these preferences are enacted in reality; see appendix for graphs of these data.

Additionally, when asked in an open-ended question how media viewing habits have changed since the introduction of watching television, movies, or other media online, the majority of respondents replied they watch more online then they have in the past. Some responses indicated this increase is due to convenience, lower costs, and ease of access on multiple viewing devices. The respondents who stated their habits only increased slightly also contributed this increase to the availability of television online. Very few responses indicated a decrease in viewing habits.

The final category of questions dealt with one’s living situation and the effect it has on media viewing habits.  The majority of respondents do not live alone, with undergraduates tending to live with non-family roommates and graduate students tending to live with either family or non-family roommates.  Few respondents lived in a home with children.  An open-choice option asked respondents to describe how their living situation affects their media viewing habits; responses were equally split between those who watch more media when living with other people and those who say their living situation has no effect on their media viewing.  Interestingly, no one reported watching less media overall as a result of living with people, although one respondent indicated that the type of media she watched was affected..  Also of interest is the language used when describing this effect; of the respondents who answered that they watch more media as a result of living with others, half of them used language that seems to blame their roommate(s) for this, stating that they “get sucked into what [roommate] was watching,” or “get hooked into watching media I otherwise wouldn’t.”

Limitations

While reviewing our survey after it had been administered we realized that we hadn’t collected gender demographics from the respondents.  That information could have potentially shown nuances that are hidden in the data due to the unbalanced sample population of our class the night our survey was administered; the majority of the sample being female.  Another limitation of our survey relating to the sample was the relatively small sample size, one group of 30 respondents, from which to draw any meaningful conclusions about media viewing habits today as compared to five years ago.

Limitations due to the survey questions themselves include a miscommunication of what information was sought after in question eight.  The phrasing of the question on the administered survey asks respondents to provide information about two separate questions within one answer.  Therefore it is not possible for us to use the data gathered from that question.

Implications and Recommendations

Our survey and others like it, i.e. studies conducted by Nielsen, report on a target population’s purchasing and media habits whose uses can reach far and wide; from research and development to marketing in areas from the entertainment industry to consumable products.

With the information gathered, it is inappropriate to make any recommendations on media viewing habits due to both the small and unbalanced sample, and infringing on people’s rights in choosing which activities to participate.  However, to improve the survey tool we recommend refining and piloting the survey questions before administering it to a larger and more balanced sample population.

Appendix A: Media Viewing Habits Survey

Demographics

1.     Which of the following best describes your student status?

□          Undergraduate

□          Graduate – Masters

□          Graduate – Doctoral

□          Non-degree seeking

2.     Which of the following best describes your age group?

□          18-22 years

□          23-27 years

□          28-32 years

□          33-40 years

□          41 year and above

Media-viewing methods and devices

We are going to ask you some questions about your media-viewing habits.  For all of these questions, we are interested only in your media-viewing habits during your personal time, i.e., in your home or on your personal mobile device.  Do not include media services or devices you have access to only at work or school.

3.     Which of the following media services do you have personal access to, and which would you like to have?  Score each media service with one of the following options:

A.    I get (or have regular personal access to) this service

B.     I don’t currently get this service, but I would like to.

C.     I don’t currently get this service and I do not want to.

D.    I don’t know.

_____              Broadcast television

_____              Cable television

_____              Internet

_____              Streaming video subscription service (ex: Netflix)

_____              Digital video recorder (DVR)

_____              Smart phone

4.     Which of the following media-capable devices do you have personal access to, and which would you like to have?  Rank each device with one of the following options:

A.    I own (or have regular personal access to) this device

B.     I don’t currently own this device, but I would like to.

C.     I don’t current own this device and I do not want to.

D.    I don’t know.

_____              Television

_____              Personal computer

_____              Smart phone

_____              Other mobile device (iPod, iPad or other tablet, or other)

_____              Gaming system (X-Box, PS3, or other)

5.     Which of the following media services and devices did you have personal access to 5 years ago (in 2006)?  Check all that apply.

□          Broadcast television

□          Cable television

□          Personal computer

□          Internet

□          Streaming video subscription service (Netflix or other)

□          Digital video recorder (DVR)

□          Smart phone

□          Other mobile device (iPod, iPad or other tablet)

□          Gaming system

6.     How has the quantity (i.e., number of hours per week) of your media-viewing habit changed in the past 5 years (since 2006)?

□          I watch more media now than I did 5 years ago.

□          I watch less media now that I did 5 years ago.

□          I watch the same amount of media now as I did 5 years ago.

7.     How much do you currently pay per month for internet and/or cable television at home?  Do not include mobile phone costs.

□          None

□          $0.01 – $20.00 per month

□          $20.01 – $40.00 per month

□          $40.01 – $60.00 per month

□          $60.01 – $80.00 month

□          More than $80.01

8.     How much do you currently pay per month for media services other than internet and/or cable at home?  This might include Netflix, Hulu, DVR How much are you willing to pay in total, per month, for entertainment media?

□          None

□          $0.01 – $10.00 per month

□          $10.01 – $20.00 per month

□          $20.01 – $30.00 per month

□          $30.01 – $40.00 month

□          More than $40.01

Internet vs. non-internet

9.     How many hours per week do you spend watching any internet-based media (i.e., streaming movies or television, Hulu, YouTube, or other)?

□          None

□          1-3 hours

□          4-6 hours

□          7-9 hours

□          10 or more hours

10.  How many hours per week do you spend watching any non-internet-based media (i.e., broadcast television, cable television, rented DVDs, other non-streamed television or movies)?

□          None

□          1-3 hours

□          4-6 hours

□          7-9 hours

□          10 or more hours

11.  Which statement best describes your preference for media viewing?

□          I prefer internet-based media over non-internet-based media.

□          I prefer non-internet-based media over internet-based media.

□          I have no preference between internet-based and non-internet-based media.

□          I do not watch any media.

12.  In question #10, you identified your preference for media viewing.  Why do you have this preference?

13.  Since the introduction of watching television, movies, or other media online, have your media viewing habits changed?  How so?

Living situation

14.  Which of the following best describes your living situation?

□          I live alone.

□          I live with one or more non-family members.

□          I live with one or more family members.

□          I live with a combination of family and non-family members.

15.  Do you live with children under the age of 18?

□          Yes

□          No

□          I live alone

16.  How does your living arrangement as described in questions #13 and #14 affect your media viewing habits (i.e., the amount of media, the type of media, the device(s) you use to watch media, or other)?

Open-ended Question Responses:

Question 12

Question 13:

Question 16

College of Education and Human Development Graduate Students’

Opinions about Open Access Journals

December 7, 2011

Most traditional academic journals, also known as toll-access journals, are accessible to the libraries that can afford to pay an annual subscription fee to a series of companies that sell journal access packages. Journal articles and subscriptions to individual journals can also be paid by either institutions or students, yet despite the possible contribution of many articles to broader society, they are not openly accessible, and this is particularly a problem for individuals who would like to contribute despite having limited financial resources or access to a subscribed institution. While information and communication technologies (ICT) has increased the reach of information, journal subscription fees have risen sharply in recent years (200% over inflation), resulting in many universities, including top tier research institutions, reducing their number of subscriptions (http://righttoresearch.org/).

Unfortunately, as subscriptions are cancelled by universities, the number of journal articles that researchers at that particular university are able to access decreases. Therefore, publishing in expensive toll-access journals can lead to a number of consequences which are both detrimental to the scholar who publishes an article and to those that are hoping to learn from the article or build on its findings. It affects the writer in terms of the visibility and impact of his or her work, while it affects the reader in the possible duplication of efforts. It can also result in the inability of the researcher to produce the most comprehensive or beneficial literature review, ignoring points of view that would have complemented the writer’s research. Fortunately, more information is becoming openly available, albeit much of it of limited quality.

To reduce these problems increasingly journals have decided to publish their articles openly by becoming an Open Access Journal (OAJ) or by allowing authors to retain some author’s rights and for the authors to be able to place a copy of the article that is accessible to the public through either their personal blog or their institution’s library website. By publishing openly, individuals across society could quickly access the scholar’s work and the benefit from understanding its implications, yet why is it that OAJ account for only ten percent of the total number of journals, according to recent Study of Open Access Publishing (SOAP) statistics (http://project-soap.eu/)? Part of the reason for the limited number of OAJs is because of the difficulties that a journal can face when trying to change its business model. Having a high quality peer review system and distributing articles to readers is very expensive. Editing is a time consuming and expensive process and should be adequately remunerated. However, because of ICTs, the cost of distributing articles to readers has been significantly reduced to the extent that they are usually only a minor expense. The rapid decrease of cost per calculation has reduced the price of hosting a site with the bandwidth needed for a journal to even less than $20 a month, depending on the journal. Being aware of the decreasing costs of running a journal, it is hard to justify the cost inflation of journal articles. In recent years, some scientist have promoted OAJ as an alternate journal business model, and have begun to campaign for increase openness within academia.

In particular the natural sciences have promoted an open access policy. The Arxiv (http://arxiv.org/) for example has over 719,000 academic pieces that are publically available and are published only months after the findings are obtained, instead of the longer periods of time of one or two year of administrative and editing processes that are needed to publish in more traditional journals. Particularly in fields such as information technologies, including the College of Education and Human Development’s (CEHD) program in educational technology, if an article takes two years to become available, that article is likely already out of date, and as such it is no longer of the highest quality or usability (For example the iPad version 1 is less than 2 years old). Unfortunately, while the natural sciences have promoted openness to increase the rate of innovation and their productivity, the social sciences and the humanities have attached more strongly to the traditional toll-access journal system. These focus groups tried to answer the why to that statement. Why would fields such as education, which hopes to improve the lives of others through their work and are not as concerned as the natural sciences in protecting patents, continue to primarily publish in traditional toll-access journals? With Open Access Week 2011 approaching, the UMN libraries decided to host a series of events (http://z.umn.edu/openaccess2011) for which I was asked to discuss my involvement on Reconsidering Development, an open access student run journal based at the University of Minnesota where I work as an assistant web-editor (http://journal.ipid-umn.org).  With this in mind, to supplement my presentation, I organized a series of four focus groups to discuss in greater detail CEHD students’ opinions about Open Access Journals

Recruiting of Participants

An invitation was sent to students two weeks before the focus group with a flier (http://z.umn.edu/oa2011poster) which directed them to the Right to Research Coalition website (http://righttoresearch.org/) as well as a Doodle poll where CEHD graduate students could register to attend to one of the four focus group sessions. The three in-person focus groups were held Thursday, October 20, Friday October 21, and October 24. An internet focus group was offered so that participants could join from October 21 to the 24 and answer the same questions in an asynchronous way ( http://oafocusgroup2011.freeforums.org). Another online focus group which answered a very similar set of question had also been held online from October 11 to October 14. Having held a total of 5 focus groups with CEHD graduate students discussing their opinions of Open Access (OA), this study feels that various saturation points have been reached and a subsequent study would benefit more in asking CEHD faculty members instead of students about their opinions regarding OAJ.

Participants did not seem to have any problems using the Doodle poll (z.umn.edu/openpoll2011). A couple of participants commented that they were attracted by the statements on the flier and having visited the included website before deciding to attend. A concern when selecting participants was that, despite regularly using open resources on the internet, participants did not generally consider themselves knowledgeable enough in the subject to openly share their opinion. Based on this concern, it is perhaps not surprising that various participants were members of the Learning Technology (LT) program within the Curriculum and Instruction (CI) department in CEHD. Purposely selecting participants, and sending every single participant a personalized email likely contributed to the high attendance rate. While there were only limited times when participants could attend a focus group and an email with a flier was sent to less than 100 students, 14 students signed up to attend the face to face (F2F) focus groups, while 10 students signed up to attend the online version of the focus group. The online focus group was held through freeforums.com. This site allows me to create user accounts and provide users with anonymity. This was mentioned to them before participating. In contrast to the online focus group, anonymity is not possible within the face to face version of the focus group. Out of the 14 scheduled face to face participants, one did not attend, resulting in 13 participants or two focus groups of 4 individuals and one focus group of 5 individuals. The online focus group had only two responses. Eight of the 10 students did not respond to the questions. However, based on the tone of their initial email response it is possible that some of these participants did not intend to participate online from the beginning. A number of them had mentioned that they couldn’t attend in person but would try to participate online. This could be interpreted as a way of expressing that they will likely be unable to participate. One of the two who participated mentioned that the questions scared him at first because being able to see all of them he felt it would take more than the time he could afford to share to answer them. However, after completing them, similar to the face to face participants, he expressed having enjoyed the focus group. Each face to face focus group lasted for two hours.

Below I summarized their responses into themes that were brought up repeatedly throughout the focus groups. In addition, for a visual overview of the different themes that were discussed in the focus group you can see or download a PowerPoint from the following link: http://www.slideshare.net/fastfonz/presentation-open-accessweek2011. It includes embedded audio clips of participants for most of the questions.

Defining Open: How do you define open, in terms of information? What qualities would you include in what it is to be open?

Students had similar opinions about the meaning of open. To them it meant that materials would get reused. A journal would be available from a computer and a mobile phone. They felt that it an article was open to the public, then it would reach a broader audience. Open access journals were also seen as helping individuals who are not academics to publish, allowing also for individuals to become a knowledge dispenser, as well as a knowledge receiver. One of the participants expressed how Open Access Journals helped increase transparency in the knowledge development process; “I can access it anywhere”. While to some participants it seemed that openness was linked mainly to greater accessibility, to others it included some of the other elements discussed within the open content literature and the benefit of being able to redistribute, remix, reuse, and revise information. Most of the participants expressed not being familiar with topics such as open education resources, open content, open data, and open courseware, but that they had found them interesting and were supportive of them after understanding them. In the online forum they were asked to visit links that explained the concepts after answering the question, then they were asked a subsequent question based on that experience.

Extent of the Influence: To what extent is the ability to access information freely on the internet influencing the type of information you consume? (not at all, some (little), a lot, a great deal)?

When asked to what extent having access influenced their research and education, most of them expressed an unwillingness to pay to read the news. While many used the Chronicle of Higher Education and the New York Times, they used the free portions of the site or accessed them through the university. Despite her interest in the paper, a student expressed her unwillingness to purchase the Wall Street Journal. By being open content was more accessible and influential. In terms of their academic work, a couple of students mentioned that they would reorganize their project if they could not access a particular source or simply dismiss it and change it for a different one. For one student, if a site required subscription, then they decided that it was not worth examining. Students use what is available, and mentioned that without a UMN ID they would not be able to have the same access.

How is Access Important: To what extent is the lack of access to information a problem? (not at all, some (little), a lot, a great deal)?

This question was somewhat related to the one above. In general lack of access led to either students borrowing access from someone or for them disregarding the source and attempt to find a similar article. They expressed how at the University of Minnesota they have more access than at smaller schools. There also seems to be a growing lack of patience with internet articles, and a student mentioned becoming frustrated after losing twenty minutes trying to get through all the security and sign up requirements when accessing an article off-site. The need to sign up to access even if the article was openly available was problematic and a source of frustration for some students. However, they felt that not having access to one article was not necessarily a major problem since; they could read more articles and find other studies. There seems to be a problem with sorting out sources and finding the best pieces of information.

Impact of Open Access Outside of Academia

While it was not its own question, many students expressed a concern about the impact that open access could have outside of higher education institutions. One of the participants mentioned how her father was able to learn more about her mother’s medical condition thanks to Open Access articles. A student from Africa mentioned his frustration of only being able to share a few articles with students. With internet being a privilege where he worked as a teacher abroad he would print a couple of openly accessible articles and share them with his students. Various of the students attending worried about not being able to access articles once they graduated and how important and helpful it could be to access these materials as an educator, a researcher, or an employee of an NGO. Organizations with a smaller budget will be more handicapped as into what they can afford to purchase. A student from Asia mentioned how there is a black market for articles in his home country and how it was more important for him, and more beneficial for society, if people were able to pursue their interests and satisfy their curiosity instead of limiting themselves to whatever the local institution can purchase. As mentioned before, there was also a concern regarding what could be accessed at a smaller university and how it was probably impacting students’ quality of education.

Generational Differences: To what extent do you think there is a generational difference in the support for openness, and open access journals?

While students seem to think that professors were not against the idea of open access, they argued that they seemed to be less aware of how it works or less optimistic about open access journals’ potential. However, students did not think of it so much as a generational issue, but perhaps also a disciplinary issue. Students then mentioned some of their problems with open access, including the difficulty of finding quality articles. A student mentioned how her professor warned her against using open articles. Another student wondered if younger students who grew up with the internet would be supportive of the toll-access model. In general, they felt that professors within CEHD were not very supportive of open access journals but that they encouraged students to publish in traditional journals. Trying to improve their marketability and obtain a tenure track job, many doctoral students feel that therefore it may be best to primarily focus on toll-access journals because of their prestige and because of the recommendations of their professors. One student, however, argued that it was important for students and professors to know that they can reserve most of their copyright and that many publishers are allowing faculty members who want to to publish while retaining many of their author rights.

Sustainability of OAJ: To what extent do you consider Open Access Journals to be sustainable? What do you think needs to happen for them to be sustainable?

One of the most common critiques of OAJ was their quality. To many students this is linked to them being free and a sense that they would accept articles that would not be accepted elsewhere. For OAJ to be sustainable it is therefore important to develop a funding model that allows them to pay for the staff needed in order to retain the quality of the journal. While in many journals the editing staff is composed largely by some of its readers who work pro-bono, understanding the importance of editing the work of their colleagues, journals have other expense such as calling for articles and organizing who need to edits which article. In some cases reviewers are paid. For the quality to be maintained and for OAJ to be sustainable, one student argued that perhaps OAJ could imitate internet games that function under the 1% or 2% models and obtain valuable funds and human resources in this way. Other students wonder if OAJs could sustain themselves by asking individuals for donations. Overall, they felt that money needs to come from somewhere and that asking individuals to pay to publish does not seem fair, “they should pay us to publish”.

Public vs. Private: Is there a difference between privately sponsored research and publically sponsored research in terms of access by the public?

Whereas some companies sponsor their own research, participants felt that if the public is paying for research then it should be available to them. A few students argued, that in their opinion, the medical field had done a great job in becoming as open as possible. NIH grants require that findings are released publically. A couple of participants were aware of this change and felt that education research, and research in other fields should be open to the public.  A student joked about why restricting access to education research seemed ridiculous and to go against the accepted aims of the field; “We have to keep our secrets about teaching and learning, nobody can know.” One participant turned the question around and asked, “In terms of access, should anything be private?” while another participant asked “is there anything that shouldn’t be public?” In addition, another participant mentioned that even sensitive data should be public because criminals will likely obtain access regardless of official controls on access. One participant mentioned that if private research is not open then companies could simply discard trials that were not positive and publish only the trials that were beneficial for them.

To what extent do you feel openness increases the quality of scholarship? (strongly agree, agree, neutral, disagree, strongly disagree)

Concerns about quality were likely the most negative attitudes students had regarding Open Access Journals. “The internet is filled with junk” was a common opinion and filtering this junk was a common concern. A student from Eastern Europe mentioned how she had difficulties accessing articles back home. In one instance, she found an archive with articles, but as she read through it, she noticed that it had not been accurately translated. This concern with accuracy and reliability was very common and led to one student repeating the advice given to her by her undergraduate instructor not to trust open access journals. However, a few students pointed out that they had used open access articles for their recent presentations and how some open journals have the same level of quality as toll-access journal articles. On a related theme another student argued that “if everybody was doing it, then there would be quality control. You want your stuff to look good anyway.” Another student also pointed out that while some online publications may publish anything they obtain, other open access journals have a very traditional peer review system. Because of the ease with which a site can be hosted, a journal with low quality can exist despite having a low quality product, yet other open journals can and do have the same level of quality as traditional toll-access journals. However, participants felt that many OAJ do not have the same level of credibility as traditional toll-access journals and, when most or all of the major journals in a field are subscription based, it is more likely that the faculty members, particularly older faculty members, will continue to publish in and advocate the use of these journals.

Moral Obligation: To what extent do you feel opening or increasing access to information is a moral obligation for educators, for the CEHD?

This question originated organically from the first focus group and was asked to the rest of the focus groups afterwards. One participant repeated various times during the focus group that access to education and knowledge should be a basic human right. In her opinion, without open access we are contributing to social reproduction as only the institutions where individuals with wealthier backgrounds attend will be able to access the most recent and relevant information. In her opinion piracy sometimes is warranted because we all have a responsibility to serve the common good. Another participant argued that if people felt that “they needed to follow unethical avenues; we need to restructure the way information is accessed.” During the focus group, information piracy was discussed as a social ill rather than an individual ill. Participants concluded that they needed “to make it possible for anyone who is seeking information to do so ethically… reworking the way we think…” Because of the moral benefits of openness and its relationship to the institutional goals of the University of Minnesota and the College of Education and Human Development one participant felt that the university itself should lead this transformation by encouraging faculty members and students to use and publish in Open Access Journals; “Academic institutions, humans, have a moral obligation to connect with each other and tell their stories”.

Innovation: To what extent do you think open access encourages or discourages innovation?

For most students open access would increase innovation because researchers could more easily develop accurate literature reviews and build upon the work of others rather than reinventing the wheel. One of the participants, explained it in the following way: “You see, we are doing something funny, we are trying to discover really good things, but then we put them in a box and lock them up. There are very wonderful things that haven’t been read…. I tried to access this article, and couldn’t access it. I thought this guy did a very good job, but what does that mean, I just forget it. I can’t pay the 39 dollars.” In this sense he and other students argued that open access contributed to innovation. Other students argued that it would increase competition and, as a consequence, journals would have to find ways to be innovative in order to increase their audience. Another student mentioned that capitalism and innovation thrive on competition so there would not only be innovation but that would also allow them to generate revenue and become sustainable. In conclusion, “if authors aren’t innovative (and if publishers aren’t innovative), readers will gravitate to other authors/publishers” Related to innovation, another student felt that open acess would result in a greater number of voices that would be heard internationally. Currently the World Bank and other major international organizations publish their voice, their viewpoint openly, but other voices are more silent. Openness would increase the visibility of other voices and, as a consequence, more countries may enact innovative policies and be themselves more innovative.

Future: Projecting yourself two, five and ten years into the future what role do you see open access playing for academia and society?

When discussing the future of OAJ, a few students pointed to how the natural sciences are at the forefront of the Open Access Movement, while the humanities, social sciences, and fields such as education are behind the curve. They argued that this is common with these fields as they are less technology oriented. However, in the future, they seemed to think that open access would likely be more commonplace. One of the students mentioned the rate of change would depend on key players in the various disciplines and their choices. Regarding quality concerns, students generally agreed that over time the quality of open access journals would likely increase. They questioned each other about how this could be done. “How were the other journals doing it… I just don’t know but it’s doable.” To various students it seemed as if the rate of change could snowball if more and more prestigious journals changed their financial structure or if authors increasingly became worried about being less cited if they only published in toll-access journals. “If everyone is invested in it, there would be a quality control. You want your stuff to look good anyway.” According to various participants, open access will probably develop layers or levels and through this system, some layers would use terms that are accessible to all while other layers would be more complex in terminology.

Choices: Which one of these three journals would you choose as the site through which to submit your article and what are some of the reasons you would chose this journal (please indicate what would be the most important aspect in making this decision)?

Despite the frequently expressed support for open access, CEHD graduate students were mostly concerned with publishing in a high impact journal or a “quality” journal at least during the first years of their careers. After obtaining tenure, they felt that they could more easily publish in open access journals. This was a very common statement, even among those that felt that open access should be a right. One of the few students that preferred an alternate route argued that her alternate route was not just in terms of publishing but because of the “academic game” and that she felt that she could accomplish more with her career elsewhere. To her, education should be about serving people, but writing for journals rather than the broader public was, she felt, a poor time investment. While she had been interested in academia before, the linking of success to publishing in prestigious journal steered her away from the profession.

Most other students, however, planned on becoming academics or publishing in their future careers. It is therefore important to create a strong professional image. One student mentioned that she would likely look at what the tradition is in a field. In a field with well-established traditions, even if it saddened her, she would likely publish in traditional toll access journals. Students expressed their interest in supporting open access journals, but at the same time were conscious of how important it is strengthen their CV. They stated that perhaps, over time, as the general trend changes, they will be able to publish more in open access journals, thereby supporting the initiative while not hurting their careers. Should open access journals become more accepted and gain prestige, students felt confident about the future of open access publishing and their ability to participate in it; “A hierarchy of journal prestige always exists.  Researchers are always trying to get their articles accepted by the most prestigious journal they can.”

Sophia.org 

Project Part 3: Exploring and Analyzing Data (40 points)

December 12, 2011

How did you come up with the idea? (2 points)

Computer enhanced learning has increased over the years. Currently online college courses account for close to 30% of student enrollment and has experience an average annual growth of over 10% per year since 2002 (http://sloanconsortium.org). While traditional higher education institutions have grown in average by 2% per year, including a growth of 1.2% from the fall of 2008 to the fall of 2009, in contrast online education grew by 21.1% from the fall of 2008 to the fall of 2009 alone.  Not only is formal online education growing, but information online education has also grown at a rapid pace. YouTube was created in 2005 by three former PayPal employees and it rapidly transformed into the second largest search engine experiencing a rate of growth of 65,000 new videos every 24 hours. While many of these videos are only seen by a handful of people, others are seen by hundreds of thousands and millions of users. Independent educators have seen decided to create educational videos to share freely online for anyone to use and benefit from their teaching methods and experience. One of the best known examples of these independent teaching initiatives has been Khan’s Academy by Salman Khan. Over the period of a few years Salman has created over two thousand videos covering hundreds of subjects and having thousands of viewers per video. Khan’s Academy has seen partnered with the Gates Foundation and was one of the inspirations for Don Smithmier, the CEO of Sophia for leaving his position as VP of Capella University to fund Sophia.org as an online educational packet sharing site were any individuals could develop an education packet covering an accepted educational topic, which then could be used by anyone over the internet and rated for quality by users and other instructors. Sophia.org potential is to decentralize and personalize learning by creating a site were hundreds of inspired individuals can openly share their knowledge and create educational packets.

Sophia.org, located in Minneapolis, Minnesota, provides students all around the state and the world with the possibility of improving both their education and teaching skills by using and creating educational packets. A site such as Sophia, where developers can create educational packets that include videos, PowerPoints, essays, among other resources is now possible partly because of these recent technological changes and the implications of the continuously decreasing costs of processing power and computer size per dollar, the possibilities to which information and communication (ICT) technologies. As the price per calculation further decreases, and programmers build on the experience of other programmers, particularly when they are building on an openly available Open Source Software project, it becomes difficult to predict how these initiatives will impact education. There are many projects such as Sophia that are increasingly being used by learners and educators. So far, it is too early to tell which of these projects or companies will become the dominant players in this rapidly expanding market. Even perhaps more difficult that knowing which of these projects will become the next Dot Com success or Dot Com bust, is to measure what impact these initiatives are having in the educational achievement of its users. Because of the lack of understanding of their impact, it is important to further study and analyze these projects. This paper attempts to take on part of that challenge by helping to finding out more additional information about Sophia.org. It is important to better understand what is the appeal this site and learn a bit more about how it is being used by teachers and learners. While many questions would be best answered through a qualitative analysis, and through the use of other methods such as surveys, interviews, focus groups, among others forms of data collection, this paper looks at the quantitative data about the site and its resources is openly available on the internet and asks how this data can also help in improve our understanding of Sophia. Sophia was chosen because of my interest in the general area of study as well as due to the geographical proximity of Sophia. I had the opportunity of visiting Sophia.org earlier this semester to learn more about their project.

• What are your hypotheses (e.g., how you think the two samples will compare)?  Please write these hypotheses out in words AND using appropriate symbols.  You should include both a null and alternative hypothesis.      (2 points)

Using primarily data that is available to anyone that visits the site, this paper, compares 60 educational packets from two categories (Humanities and Sciences) that can be found by users visiting Sophia.org. These categories were created to guide both users and creators to where they can better locate the packet they are searching for. Sophia.org employs a taxonomy organizational system, which among many others includes 10 different major categories, two of which are Humanities and Sciences. While the project originally planned to compare Sophia.org with a different organization such as UDemy.com, Wikieducator.org or AcademicEarth.com, for the purposes of this assignment, the data gathered and compared is all from Sophia.org and was obtained by selecting two of the main categories at random and then comparing the popularity or the number of visits in a month Science packets had in comparison to humanities packets. The null hypothesis for this study is that both packets have in average the same number of visitors per month or H₀: µ = µ₀  (Null Hypothesis) which is to say that difference in the number of visitors between both samples is not statistically significant. In contrast, the alternate hypothesis argues that in average the number of visitors per month to Science packets is greater than the number of visits to Humanities packets or that vs. H_a: µ₁ > µ₀ (One-sided – Alternate Hypothesis). Since the packets are exclusive from each other, the samples are independent. To find the relationship I therefore used SPSS to calculate the p-value from a one-sided independent t-test.

• What is the reason for your hypotheses (e.g., why do you think the samples will differ in the way you predict)?  (2 points)

After randomly deciding on two categories to compare, I access the list of resources that were classified under each of the two categories. After a basic overview of the data, it appeared that the Science packets were reviewed more often, and visited more than the humanities packets. While there is no way of know whether the Science packets began to be developed first and therefore had a higher total number of viewers per packet or if one only a few packets that were very popular and had a large number of viewers were skewing the results, it was likely that in addition to the Science packets having in average a greater number of viewer per packet (statistic that has been collected by the site for each packet as soon as it is posted) that they also had a significantly higher number of viewers or users per month. While this project could not make comparison using the historical data and the average growth rate of the resources of each category it could collect information from a sample of resources of both categories and conduct and statistical test to find if there is a statistically significant difference in the degree of use of these two packet categories during the month the statistics were collected. After calculating this difference, if the alternate hypothesis is supported by the data comparison, it could therefore be argued that there is a greater demand or need for educational packets in the Sciences than there is in the Humanities? While after collecting the data, it appeared that this difference was there, but an statistical t-test was needed to determine the extent of this relation.

• How did you gather your data?  (2 points)

The data was gathered over a period of a month. While the project originally planned on collecting data from two different websites or projects and then comparing the usage of a similar type of educational packet from two different places (Use of history packets in UDemy.com vs. Sophia.org vs. AcademicEarth.com), after attempting to collect comparable data for multiple sites for a number of weeks, in the end, the data shared to the public by different programs was too different for an statistical comparison to be possible. Whereas UDemy.com focused on the number of registered users for a course, and some of the courses were only accessible for a price, Sophia.org does not require the user to register to use the resource and it focuses instead on the number of people who visit the webpage of the educational packet. In the case of Sophia.org a visitor may not intend to use the packet but may be instead reading to find out if it’s useful, whereas because of the need to register to use certain resources within UDemy.com, the user is more likely to use it. Because of these differences, as well as similar differences of other programs, this study decided to focus only on Sophia.

Once that decision was made, obtain the data required accessing the site and selecting the two categories then adding to a spreadsheet information about the top 30 packets that appeared when clicking on the category. It is possible that the website has an algorithm that determines which resources are displayed first. I was unable to therefore randomly select the samples. However, since the same criteria was used for selecting the Sciences and the Humanities packets the samples are therefore comparable. They also represent only a sample of what is available for each category. Once the 30 resources on each category were selected, I entered the resource and recorded a series of variables including: type of packet, packet name, packet URL, number of packet views, number of packet shares, number of packet followers, packet rating, number of packet ratings, copyright license used for the resource, Sophia user score of the packet creator, number of follower to the creator of the packet, and the number of people the creator of the packet is following.

Within these, various variables were collected in two occasions, collecting a value for September and a value for October. To improve the quality and reliability of the data the data for a whole month with be collected over a number of hours in a single day. The variables that were collected twice for the purposes of a future comparison were: number of packet views (Sept, Oct), number of packet shares (Sept, Oct), packet rating (Sept, Oct), and the average number of ratings (Sept, Oct). As mentioned before, once the packets were selected, gathering the data was single as the data is openly available.

Graphs and Descriptive Statistics (10 points)

Statistics for Science Packets:

OVERALL SHAPE: The three curves are unimodal, yet they do not resemble normal distributions. The three curves are skewed to the right. However, it seems that the science packet curves have a broader range than the humanities curve. The shape can be visualized in greater detail in the graphs below. The shape can be considered to be closer to a reverse j-curve than a normal distribution. This is probably because of the low entry cost of producing an educational packet. There is a large quantity of packages but there many of them have a small number of viewers.

Shape: The curve is unimodal. It could overtime look more as a normal distribution curve, yet because based on current trends it will likely increasingly resemble a reverse j-curve. If packets that were not visited were deleted over time, then the distribution would likely be different. Is skewed to the right.

Center / Location: It has a mean value of 615 which is to say that according to the statistics collected in September, out of the top 30 packets, the average number of views per packet was 615. The median or the value which is in between half of the value (half greater, half smaller) was 435.50 which indicates that there are some outliers to the right that are contributing to the skewness of the curve.

Spread / Variability: The curve has a range of 2178 and a standard deviation of 474.937. While the range includes over a 2000 viewers value difference, most values are located closer to the mean or a value of 615.

Shape: This curve is also unimodal. The curve is also skewed to the right, yet over the past month there seems to have been an increase in the range and an increase in the mean. Interestingly, there is also an increase in the standard deviation.

Center / Location: Unlike I originally predicted, this curve seems to more closely resemble a standard deviation and may be an indication of future trends, yet the mean (803.33) and the median (607) are very far from the maximum value.

Spread / Variability: It has a greater range than the previous month. The range is now of over 2300 views. The standard deviation also increased to 561.387 from 474.93. Along with this change, the variable has as expectedly increased.

Shape: When comparing the differences or the growth from September to October in views of science related videos, the curve continues to be skewed but the values are much smaller. It looks close to a standard curve and less like a reverse j-curve, a change that was noticed when analyzing the October views curve after analyzing the September views curve. The curve is once again skewed to the right. The curve is unimodal.

Center / Location: The center of the curve can be determined by the mean or median. As with the other two graphs the mode was not a relevant value since in two of the graphs no values were repeated.  The mean is of 187.93 while the median is of 142.5.

Spread / Variability: Unlike the other two graphs, the range of this graph is substantially smaller. Since it only represents the viewers over a month this is to be expected. The range is barely over 500 viewers. There is a much smaller variance than the other two graphs for Science packets, and the standard deviation is also much smaller, having a value of 134.56

Statistics for Humanities Packets:

Shape: While there are also 30 packets for the humanities graph, the graph is substantially more skewed (to the right). One outlier is having a major impact on the spread of the graph. This greatly affects the shape as the curve also resembles a reverse j-curve. The curve is unimodal. It does not resemble a normal distribution.

Center / Location: For this graph the median is a better indicator of the center than the mean. The use of the median as the most accurate center is due to the degree to which this curve is skewed. This curve has a mean of 383.5 and a median value of 104.

Spread / Variability: The variance of this curve is greater than the variance of any of the Science packet curves. This curve has a range of 3733 with a maximum value of 3737 and a large standard deviation of 726 despite half of the values being below 104.

Shape: This graph is the most skewed of all graphs. It is also skewed to the right and it also resembles a reverse j-curve more than a normal distribution curve. The curve is unimodal. There is again a very clear outlier which may or may not be indicative of a trend. It is possible that very high quality or valued Humanities packets obtain a much higher visibility than the other Humanities packets. The collection of future packets in the future as the population increase make help answer this question.

Center / Location: As with the previous graph, the most adequate unit of center for this graph is the media (127 views). While with a normal curve the mean may be a more adequate statistical variable, in this case having such a distant outlier influencing the curve, the median is more indicative of the center than the mean (481 views).

Spread / Variability: The variability of this curve is the most extensive having a range of 4616 views. The variance is also the most extensive as well as the standard deviation. While half of the values are below 127, the standard deviation is of 895.90 views.

Shape: This curve is also much skewed to the right and also resembles a j-curve although to a lesser extent than the two other curves for Humanities packets. Since this curve represents the difference, it has a smaller center and range of values. As with the rest of the curve, this curve is unimodal. It does not resemble a normal distribution.

Center / Location: For this curve the median is again more indicative of the center than the mean.  The mean value is of 97.8 views, while the median is of 32.5 views. As with the other curves, the mode is not practically significant since there are few or no values repeated for all six curves.

Spread / Variability: Being a curve based on the difference or the change in views from September to October of Humanities packets its range, standard deviation, and variance are small than the other two curve for the Humanities packets. It has a range of 894 views and a standard deviation of 172.32 views.

Verifying Necessary Data Conditions (4 points)

The data analyzed in this assignment is grouped in two independent pairs. When conducting a t-test it is important to have a large sample size. The larger than sample sizes the more indicate they are of the population distribution and the smaller the sample error. Some samples of populations resemble a normal distribution while others do not. In the cases discussed above, most of them resembled a reverse j-curve, as the number of cases decreases quickly when plotted by the number of viewers. Another problem with the data collected is that there are strong outliers particularly when looking at the Humanities data curves. Despite having had a sample size of 30 (usually consider large) because of the high level of skewness, having more cases would have been beneficial. Having mentioned some of the problems with using this data, I will now test for significant relationships between the two variables, Humanities to Science packets, yet it is important to keep in mind that the data is skewed and while part of the selection was randomized, since items were selected from a website with its own classification rules, those rules may have negatively influenced the selection process. However, many other variables were controlled for: the data has been collected from the same site, the site became live and started counting views for both categories simultaneously, and the selection rules of one group were exactly the same as the selection rules of the other group.

Conducting a hypothesis test (10 points)

A one-sided independent t-test was conducted to test for significant relationships between the variables.

Based on the results of the hypothesis test, do you reject or fail to reject Ho?  Why?  Do you feel your results are statistically significant?  Are they practically significant?  What is the p-value?  Interpret the p-value in your own words. You discuss the results of your hypothesis test.  What was the p-value?  Interpret the p-value in your own words.  Based on the results of the hypothesis test, do you reject or fail to reject Ho?  Why? Are the results statistically significant?  Are they practically significant?  (4 points)

Because of some of the differences in the variance of the data, I did not assume that they are from the same population. As such, the p-values were calculated with “equal variances not assumed”. The p-values obtained are .149, .102, .028. Since this was a one-sided t-test, those values can be divided by 2, but this also divides the alpha-level from .05 to .025 for significance. According to these p-values we can reject the null hypothesis when we compare the difference in the total number of viewers from Sept to Oct as the p-value is of .014 which is to say that in 1.4% of the time the conditions needed for the null hypothesis to be true are there. The p-value or probability values indicate how likely we are of obtaining a test statistics as extreme as the one obtained in this test. The other p-values obtained .075 and .051 are very close to the alpha-level yet, the null hypothesis cannot be rejected as it could result in a type 1 error.

The null hypothesis is rejected based on the results of the third t-test, yet the null hypothesis cannot be rejected based on the other two results that compared the total views from October and September for Humanities and Science packets. Because of the extent to which the values were skewed more cases would been beneficial to finding out if the first two test could also be significant. They were both very close to another commonly used alpha value of .10. However, using an alpha-value of .05, only the difference between the total number of views from September to October between Humanities and Science packets was significant. This is partly visible in the means of both types of packets. Humanities packets had a mean of 97.8 while the Science packets had a mean of 187.9. However similar ranges of difference were also visible in the mean for the total number of viewers. Yet, we can only safely conclude that having a p-value of .028, it appears that they number of views of Science packets is increasing statistically significantly faster than the total number of views of Humanities packets. Therefore, because of how unlike it is for this values to happen without, we can reject the null hypothesis –  H₀: µ = µ₀  (Null Hypothesis) and accept the alternate hypothesis – H_a: µ₁ > µ₀ (One-sided – Alternate Hypothesis) which as a one sided test has a p-value of .014. More cases are needed to have be certain that the other differences are or are not significant. To avoid a type 1 error, I cannot reject the null hypothesis with the other two tests, but I may be committing a type 2 error and would benefit from increasing the number of cases.

These results are also practically significant. It is clear that some types of packages are visited more often than others and this difference is linked to their category. While this difference may also be linked to their rating, it seems that more individuals visit Sophia to view and possibly use Science packets. This may be related to the relationship between computer science and mathematics and more directly to the relationship of the Open Source Software Movement to the Open Education Resource Movement. While it is not possible to determine this relationship from the data, understanding this difference could lead some business to market Humanities materials more aggressively to try to differentiate themselves, or have the opposite reaction and stop or reduce the production of Humanities materials so that the quality of the resources that are more likely to be used increases, the quality of Sophia increases, and marginal, low-quality products are discarded.

• Based on the results of your hypothesis test, what kind of error could you have made?  Please explain, and indicate just how you might control for this kind of error.  (3 points)

As previously explained because of the skewness of the data, more cases would have helped in increasing the validity of the t-test results, yet despite not assuming equal variances, the results of the third test are clearly significant. Since I selected an alpha level of .05, the other two test with results of .149 and .101 were not significant. While the .149 value would not have been significant if the alpha level was of .1, the .101 result could have been seen as significant or could have been determined either way. Yet, it may have resulted in inappropriately rejecting the null hypothesis and committing a type 1 error. It is also possible that not enough cases were included and had more cases been included both of those values would have been significant. If this is the case, then type 2 errors may have taken place, yet without further testing, this conclusion cannot be reached. The only conclusion that can be reached from the data using a .05 alpha level is that the third test was statistically significant and therefore it is appropriate to reject the third null hypothesis.

• A confidence interval is automatically generated when you conduct a t-test.  Please indicate what this interval is and how it should be interpreted.  (2 points)

The confidence interval is used to estimate the reliability of the data or the relationship between the sample estimates and the population estimate. For this test, I obtained the confidence intervals mentioned below. The greater the confidence level (usually it is either: 90%, 95%, or 99%) the greater the range of possible values. Confidence regions can help indicated if there are likely sampling errors as well as help indicate if one estimate for a quantity is unreliable, and as such if there are also others that may be unreliable.

Conclusion and Summary (8 points)

Did you discover anything that surprised you when you analyzed the data?  Do you think the results would have been different if you had bigger sample sizes?  If you had to do the project again, how would you do it differently?

You summarize your project and include some mention of how you came up with your idea, how you collected your data, and what you found when you explored and analyzed your data.  (3 points)

After collecting the data and analyzing the samples, we can conclude that there is a relationship between the types of packets that are viewed by users most often. As previously mentioned, this may provide a market advantage for those that produce high quality Humanities packets, or more likely lead for business to consider expanding their Science packets since there seems to be a greater use and demand for these packets. This is not to say Humanities packets should generally be discarded, but only that currently they are less viewed than Science packets. This is not a surprise as Khan’s Academy, one of the inspirations for Sophia.org primarily focuses on Science packets. In addition, the university that began the Open CourseWare Movement by opening their courses to the public is also a Science packets oriented institution. Sophia.org is experiencing the same trend and it may be due to Science packets not being as contextual as Humanities packets. Humanities packets may be more political and site specific. In addition, Humanities instructors tend to not be as technological oriented and innovation oriented as Science instructors. Sophia.org itself may be capitalizing of this trend as they recently began offering a paid service of over 20,000 math only videos to supplement classroom and home school instruction. The findings of this study were helpful in illustrating this trend.

• You discuss any shortcomings of the methods you used to gather data.  (3 points)

Unfortunately, being only in its first public year, despite the rapid growth of the site, there are only a limited number of packets within the Sophia website. Because of the small number of packets, the packets are more indicate or more closely resemble the population, but they were selected via the ranking of the website instead of a randomizing process. While the categories were randomly selected, the site classification also influence the data selection process.

• You talk about how you would do the project differently if you were to do it over again (2 points)

If this project was conducted again, additional data would be gathered. In addition, I would communicate with Sophia.org so that they would allow me to access their site analytics. While I collected the data that was openly available in a per month basis, this data is collected every day by the Sophia.org servers. Having access to their servers and site analytics would increase the depth and quality of the study. Another path to follow is to possible compare a different site or more categories. However, as I experienced initially when constructing this study, the differences in how sites are organized makes it difficult to compare between sites. Waiting until more data is available would help to more comprehensively develop a study of Sophia.org.

Project Part 3: Exploring and Analyzing Data (40 points)

12/2/11

How did you come up with the idea? (2 points)

Computer enhanced learning has increased over the years. Currently online college courses account for close to 30% of student enrollment and has experience an average annual growth of over 10% per year since 2002 (http://sloanconsortium.org). While traditional higher education institutions have grown in average by 2% per year, including a growth of 1.2% from the fall of 2008 to the fall of 2009, in contrast online education grew by 21.1% from the fall of 2008 to the fall of 2009 alone.  Not only is formal online education growing, but information online education has also grown at a rapid pace. YouTube was created in 2005 by three former PayPal employees and it rapidly transformed into the second largest search engine experiencing a rate of growth of 65,000 new videos every 24 hours. While many of these videos are only seen by a handful of people, others are seen by hundreds of thousands and millions of users. Independent educators have seen decided to create educational videos to share freely online for anyone to use and benefit from their teaching methods and experience. One of the best known examples of these independent teaching initiatives has been Khan’s Academy by Salman Khan. Over the period of a few years Salman has created over two thousand videos covering hundreds of subjects and having thousands of viewers per video. Khan’s Academy has seen partnered with the Gates Foundation and was one of the inspirations for Don Smithmier, the CEO of Sophia for leaving his position as VP of Capella University to fund Sophia.org as an online educational packet sharing site were any individuals could develop an education packet covering an accepted educational topic, which then could be used by anyone over the internet and rated for quality by users and other instructors. Sophia.org potential is to decentralize and personalize learning by creating a site were hundreds of inspired individuals can openly share their knowledge and create educational packets.

Sophia.org, located in Minneapolis, Minnesota, provides students all around the state and the world with the possibility of improving both their education and teaching skills by using and creating educational packets. A site such as Sophia, where developers can create educational packets that include videos, PowerPoints, essays, among other resources is now possible partly because of these recent technological changes and the implications of the continuously decreasing costs of processing power and computer size per dollar, the possibilities to which information and communication (ICT) technologies. As the price per calculation further decreases, and programmers build on the experience of other programmers, particularly when they are building on an openly available Open Source Software project, it becomes difficult to predict how these initiatives will impact education. There are many projects such as Sophia that are increasingly being used by learners and educators. So far, it is too early to tell which of these projects or companies will become the dominant players in this rapidly expanding market. Even perhaps more difficult that knowing which of these projects will become the next Dot Com success or Dot Com bust, is to measure what impact these initiatives are having in the educational achievement of its users. Because of the lack of understanding of their impact, it is important to further study and analyze these projects. This paper attempts to take on part of that challenge by helping to finding out more additional information about Sophia.org. It is important to better understand what is the appeal this site and learn a bit more about how it is being used by teachers and learners. While many questions would be best answered through a qualitative analysis, and through the use of other methods such as surveys, interviews, focus groups, among others forms of data collection, this paper looks at the quantitative data about the site and its resources is openly available on the internet and asks how this data can also help in improve our understanding of Sophia. Sophia was chosen because of my interest in the general area of study as well as due to the geographical proximity of Sophia. I had the opportunity of visiting Sophia.org earlier this semester to learn more about their project.

• What are your hypotheses (e.g., how you think the two samples will compare)?  Please write these hypotheses out in words AND using appropriate symbols.  You should include both a null and alternative hypothesis.      (2 points)

Using primarily data that is available to anyone that visits the site, this paper, compares 60 educational packets from two categories (Humanities and Sciences) that can be found by users visiting Sophia.org. These categories were created to guide both users and creators to where they can better locate the packet they are searching for. Sophia.org employs a taxonomy organizational system, which among many others includes 10 different major categories, two of which are Humanities and Sciences. While the project originally planned to compare Sophia.org with a different organization such as UDemy.com, Wikieducator.org or AcademicEarth.com, for the purposes of this assignment, the data gathered and compared is all from Sophia.org and was obtained by selecting two of the main categories at random and then comparing the popularity or the number of visits in a month Science packets had in comparison to humanities packets. The null hypothesis for this study is that both packets have in average the same number of visitors per month or H₀: µ = µ₀  (Null Hypothesis) which is to say that difference in the number of visitors between both samples is not statistically significant. In contrast, the alternate hypothesis argues that in average the number of visitors per month to Science packets is greater than the number of visits to Humanities packets or that vs. H_a: µ₁ > µ₀ (One-sided – Alternate Hypothesis). Since the packets are exclusive from each other, the samples are independent. To find the relationship I therefore used SPSS to calculate the p-value from a one-sided independent t-test.

• What is the reason for your hypotheses (e.g., why do you think the samples will differ in the way you predict)?  (2 points)

After randomly deciding on two categories to compare, I access the list of resources that were classified under each of the two categories. After a basic overview of the data, it appeared that the Science packets were reviewed more often, and visited more than the humanities packets. While there is no way of know whether the Science packets began to be developed first and therefore had a higher total number of viewers per packet or if one only a few packets that were very popular and had a large number of viewers were skewing the results, it was likely that in addition to the Science packets having in average a greater number of viewer per packet (statistic that has been collected by the site for each packet as soon as it is posted) that they also had a significantly higher number of viewers or users per month. While this project could not make comparison using the historical data and the average growth rate of the resources of each category it could collect information from a sample of resources of both categories and conduct and statistical test to find if there is a statistically significant difference in the degree of use of these two packet categories during the month the statistics were collected. After calculating this difference, if the alternate hypothesis is supported by the data comparison, it could therefore be argued that there is a greater demand or need for educational packets in the Sciences than there is in the Humanities? While after collecting the data, it appeared that this difference was there, but an statistical t-test was needed to determine the extent of this relation.

• How did you gather your data?  (2 points)

The data was gathered over a period of a month. While the project originally planned on collecting data from two different websites or projects and then comparing the usage of a similar type of educational packet from two different places (Use of history packets in UDemy.com vs. Sophia.org vs. AcademicEarth.com), after attempting to collect comparable data for multiple sites for a number of weeks, in the end, the data shared to the public by different programs was too different for an statistical comparison to be possible. Whereas UDemy.com focused on the number of registered users for a course, and some of the courses were only accessible for a price, Sophia.org does not require the user to register to use the resource and it focuses instead on the number of people who visit the webpage of the educational packet. In the case of Sophia.org a visitor may not intend to use the packet but may be instead reading to find out if it’s useful, whereas because of the need to register to use certain resources within UDemy.com, the user is more likely to use it. Because of these differences, as well as similar differences of other programs, this study decided to focus only on Sophia.

Once that decision was made, obtain the data required accessing the site and selecting the two categories then adding to a spreadsheet information about the top 30 packets that appeared when clicking on the category. It is possible that the website has an algorithm that determines which resources are displayed first. I was unable to therefore randomly select the samples. However, since the same criteria was used for selecting the Sciences and the Humanities packets the samples are therefore comparable. They also represent only a sample of what is available for each category. Once the 30 resources on each category were selected, I entered the resource and recorded a series of variables including: type of packet, packet name, packet URL, number of packet views, number of packet shares, number of packet followers, packet rating, number of packet ratings, copyright license used for the resource, Sophia user score of the packet creator, number of follower to the creator of the packet, and the number of people the creator of the packet is following.

Within these, various variables were collected in two occasions, collecting a value for September and a value for October. To improve the quality and reliability of the data the data for a whole month with be collected over a number of hours in a single day. The variables that were collected twice for the purposes of a future comparison were: number of packet views (Sept, Oct), number of packet shares (Sept, Oct), packet rating (Sept, Oct), and the average number of ratings (Sept, Oct). As mentioned before, once the packets were selected, gathering the data was single as the data is openly available.

Graphs and Descriptive Statistics (10 points)

Statistics for Science Packets:

OVERALL SHAPE: The three curves are unimodal, yet they do not resemble normal distributions. The three curves are skewed to the right. However, it seems that the science packet curves have a broader range than the humanities curve. The shape can be visualized in greater detail in the graphs below. The shape can be considered to be closer to a reverse j-curve than a normal distribution. This is probably because of the low entry cost of producing an educational packet. There is a large quantity of packages but there many of them have a small number of viewers.

Shape: The curve is unimodal. It could overtime look more as a normal distribution curve, yet because based on current trends it will likely increasingly resemble a reverse j-curve. If packets that were not visited were deleted over time, then the distribution would likely be different. Is skewed to the right.

Center / Location: It has a mean value of 615 which is to say that according to the statistics collected in September, out of the top 30 packets, the average number of views per packet was 615. The median or the value which is in between half of the value (half greater, half smaller) was 435.50 which indicates that there are some outliers to the right that are contributing to the skewness of the curve.

Spread / Variability: The curve has a range of 2178 and a standard deviation of 474.937. While the range includes over a 2000 viewers value difference, most values are located closer to the mean or a value of 615.

Shape: This curve is also unimodal. The curve is also skewed to the right, yet over the past month there seems to have been an increase in the range and an increase in the mean. Interestingly, there is also an increase in the standard deviation.

Center / Location: Unlike I originally predicted, this curve seems to more closely resemble a standard deviation and may be an indication of future trends, yet the mean (803.33) and the median (607) are very far from the maximum value.

Spread / Variability: It has a greater range than the previous month. The range is now of over 2300 views. The standard deviation also increased to 561.387 from 474.93. Along with this change, the variable has as expectedly increased.

Shape: When comparing the differences or the growth from September to October in views of science related videos, the curve continues to be skewed but the values are much smaller. It looks close to a standard curve and less like a reverse j-curve, a change that was noticed when analyzing the October views curve after analyzing the September views curve. The curve is once again skewed to the right. The curve is unimodal.

Center / Location: The center of the curve can be determined by the mean or median. As with the other two graphs the mode was not a relevant value since in two of the graphs no values were repeated.  The mean is of 187.93 while the median is of 142.5.

Spread / Variability: Unlike the other two graphs, the range of this graph is substantially smaller. Since it only represents the viewers over a month this is to be expected. The range is barely over 500 viewers. There is a much smaller variance than the other two graphs for Science packets, and the standard deviation is also much smaller, having a value of 134.56

Statistics for Humanities Packets:

Shape: While there are also 30 packets for the humanities graph, the graph is substantially more skewed (to the right). One outlier is having a major impact on the spread of the graph. This greatly affects the shape as the curve also resembles a reverse j-curve. The curve is unimodal. It does not resemble a normal distribution.

Center / Location: For this graph the median is a better indicator of the center than the mean. The use of the median as the most accurate center is due to the degree to which this curve is skewed. This curve has a mean of 383.5 and a median value of 104.

Spread / Variability: The variance of this curve is greater than the variance of any of the Science packet curves. This curve has a range of 3733 with a maximum value of 3737 and a large standard deviation of 726 despite half of the values being below 104.

Shape: This graph is the most skewed of all graphs. It is also skewed to the right and it also resembles a reverse j-curve more than a normal distribution curve. The curve is unimodal. There is again a very clear outlier which may or may not be indicative of a trend. It is possible that very high quality or valued Humanities packets obtain a much higher visibility than the other Humanities packets. The collection of future packets in the future as the population increase make help answer this question.

Center / Location: As with the previous graph, the most adequate unit of center for this graph is the media (127 views). While with a normal curve the mean may be a more adequate statistical variable, in this case having such a distant outlier influencing the curve, the median is more indicative of the center than the mean (481 views).

Spread / Variability: The variability of this curve is the most extensive having a range of 4616 views. The variance is also the most extensive as well as the standard deviation. While half of the values are below 127, the standard deviation is of 895.90 views.

Shape: This curve is also much skewed to the right and also resembles a j-curve although to a lesser extent than the two other curves for Humanities packets. Since this curve represents the difference, it has a smaller center and range of values. As with the rest of the curve, this curve is unimodal. It does not resemble a normal distribution.

Center / Location: For this curve the median is again more indicative of the center than the mean.  The mean value is of 97.8 views, while the median is of 32.5 views. As with the other curves, the mode is not practically significant since there are few or no values repeated for all six curves.

Spread / Variability: Being a curve based on the difference or the change in views from September to October of Humanities packets its range, standard deviation, and variance are small than the other two curve for the Humanities packets. It has a range of 894 views and a standard deviation of 172.32 views.

Verifying Necessary Data Conditions (4 points)

The data analyzed in this assignment is grouped in two independent pairs. When conducting a t-test it is important to have a large sample size. The larger than sample sizes the more indicate they are of the population distribution and the smaller the sample error. Some samples of populations resemble a normal distribution while others do not. In the cases discussed above, most of them resembled a reverse j-curve, as the number of cases decreases quickly when plotted by the number of viewers. Another problem with the data collected is that there are strong outliers particularly when looking at the Humanities data curves. Despite having had a sample size of 30 (usually consider large) because of the high level of skewness, having more cases would have been beneficial. Having mentioned some of the problems with using this data, I will now test for significant relationships between the two variables, Humanities to Science packets, yet it is important to keep in mind that the data is skewed and while part of the selection was randomized, since items were selected from a website with its own classification rules, those rules may have negatively influenced the selection process. However, many other variables were controlled for: the data has been collected from the same site, the site became live and started counting views for both categories simultaneously, and the selection rules of one group were exactly the same as the selection rules of the other group.

Conducting a hypothesis test (10 points)

A one-sided independent t-test was conducted to test for significant relationships between the variables.

Based on the results of the hypothesis test, do you reject or fail to reject Ho?  Why?  Do you feel your results are statistically significant?  Are they practically significant?  What is the p-value?  Interpret the p-value in your own words. You discuss the results of your hypothesis test.  What was the p-value?  Interpret the p-value in your own words.  Based on the results of the hypothesis test, do you reject or fail to reject Ho?  Why? Are the results statistically significant?  Are they practically significant?  (4 points)

Because of some of the differences in the variance of the data, I did not assume that they are from the same population. As such, the p-values were calculated with “equal variances not assumed”. The p-values obtained are .149, .102, .028. Since this was a one-sided t-test, those values can be divided by 2, but this also divides the alpha-level from .05 to .025 for significance. According to these p-values we can reject the null hypothesis when we compare the difference in the total number of viewers from Sept to Oct as the p-value is of .014 which is to say that in 1.4% of the time the conditions needed for the null hypothesis to be true are there. The p-value or probability values indicate how likely we are of obtaining a test statistics as extreme as the one obtained in this test. The other p-values obtained .075 and .051 are very close to the alpha-level yet, the null hypothesis cannot be rejected as it could result in a type 1 error.

The null hypothesis is rejected based on the results of the third t-test, yet the null hypothesis cannot be rejected based on the other two results that compared the total views from October and September for Humanities and Science packets. Because of the extent to which the values were skewed more cases would been beneficial to finding out if the first two test could also be significant. They were both very close to another commonly used alpha value of .10. However, using an alpha-value of .05, only the difference between the total number of views from September to October between Humanities and Science packets was significant. This is partly visible in the means of both types of packets. Humanities packets had a mean of 97.8 while the Science packets had a mean of 187.9. However similar ranges of difference were also visible in the mean for the total number of viewers. Yet, we can only safely conclude that having a p-value of .028, it appears that they number of views of Science packets is increasing statistically significantly faster than the total number of views of Humanities packets. Therefore, because of how unlike it is for this values to happen without, we can reject the null hypothesis –  H₀: µ = µ₀  (Null Hypothesis) and accept the alternate hypothesis – H_a: µ₁ > µ₀ (One-sided – Alternate Hypothesis) which as a one sided test has a p-value of .014. More cases are needed to have be certain that the other differences are or are not significant. To avoid a type 1 error, I cannot reject the null hypothesis with the other two tests, but I may be committing a type 2 error and would benefit from increasing the number of cases.

These results are also practically significant. It is clear that some types of packages are visited more often than others and this difference is linked to their category. While this difference may also be linked to their rating, it seems that more individuals visit Sophia to view and possibly use Science packets. This may be related to the relationship between computer science and mathematics and more directly to the relationship of the Open Source Software Movement to the Open Education Resource Movement. While it is not possible to determine this relationship from the data, understanding this difference could lead some business to market Humanities materials more aggressively to try to differentiate themselves, or have the opposite reaction and stop or reduce the production of Humanities materials so that the quality of the resources that are more likely to be used increases, the quality of Sophia increases, and marginal, low-quality products are discarded.

• Based on the results of your hypothesis test, what kind of error could you have made?  Please explain, and indicate just how you might control for this kind of error.  (3 points)

As previously explained because of the skewness of the data, more cases would have helped in increasing the validity of the t-test results, yet despite not assuming equal variances, the results of the third test are clearly significant. Since I selected an alpha level of .05, the other two test with results of .149 and .101 were not significant. While the .149 value would not have been significant if the alpha level was of .1, the .101 result could have been seen as significant or could have been determined either way. Yet, it may have resulted in inappropriately rejecting the null hypothesis and committing a type 1 error. It is also possible that not enough cases were included and had more cases been included both of those values would have been significant. If this is the case, then type 2 errors may have taken place, yet without further testing, this conclusion cannot be reached. The only conclusion that can be reached from the data using a .05 alpha level is that the third test was statistically significant and therefore it is appropriate to reject the third null hypothesis.

• A confidence interval is automatically generated when you conduct a t-test.  Please indicate what this interval is and how it should be interpreted.  (2 points)

The confidence interval is used to estimate the reliability of the data or the relationship between the sample estimates and the population estimate. For this test, I obtained the confidence intervals mentioned below. The greater the confidence level (usually it is either: 90%, 95%, or 99%) the greater the range of possible values. Confidence regions can help indicated if there are likely sampling errors as well as help indicate if one estimate for a quantity is unreliable, and as such if there are also others that may be unreliable.

Conclusion and Summary (8 points)

Did you discover anything that surprised you when you analyzed the data?  Do you think the results would have been different if you had bigger sample sizes?  If you had to do the project again, how would you do it differently?

You summarize your project and include some mention of how you came up with your idea, how you collected your data, and what you found when you explored and analyzed your data.  (3 points)

After collecting the data and analyzing the samples, we can conclude that there is a relationship between the types of packets that are viewed by users most often. As previously mentioned, this may provide a market advantage for those that produce high quality Humanities packets, or more likely lead for business to consider expanding their Science packets since there seems to be a greater use and demand for these packets. This is not to say Humanities packets should generally be discarded, but only that currently they are less viewed than Science packets. This is not a surprise as Khan’s Academy, one of the inspirations for Sophia.org primarily focuses on Science packets. In addition, the university that began the Open CourseWare Movement by opening their courses to the public is also a Science packets oriented institution. Sophia.org is experiencing the same trend and it may be due to Science packets not being as contextual as Humanities packets. Humanities packets may be more political and site specific. In addition, Humanities instructors tend to not be as technological oriented and innovation oriented as Science instructors. Sophia.org itself may be capitalizing of this trend as they recently began offering a paid service of over 20,000 math only videos to supplement classroom and home school instruction. The findings of this study were helpful in illustrating this trend.

• You discuss any shortcomings of the methods you used to gather data.  (3 points)

Unfortunately, being only in its first public year, despite the rapid growth of the site, there are only a limited number of packets within the Sophia website. Because of the small number of packets, the packets are more indicate or more closely resemble the population, but they were selected via the ranking of the website instead of a randomizing process. While the categories were randomly selected, the site classification also influence the data selection process.

• You talk about how you would do the project differently if you were to do it over again (2 points)

If this project was conducted again, additional data would be gathered. In addition, I would communicate with Sophia.org so that they would allow me to access their site analytics. While I collected the data that was openly available in a per month basis, this data is collected every day by the Sophia.org servers. Having access to their servers and site analytics would increase the depth and quality of the study. Another path to follow is to possible compare a different site or more categories. However, as I experienced initially when constructing this study, the differences in how sites are organized makes it difficult to compare between sites. Waiting until more data is available would help to more comprehensively develop a study of Sophia.org.

College of Education and Human Development Graduate Students’

Opinions about Open Access Journals

December 2, 2011

Most traditional academic journals, also known as toll-access journals, are accessible to the libraries that can afford to pay an annual subscription fee to a series of companies that sell journal access packages. Journal articles and subscriptions to individual journals can also be paid by either institutions or students, yet despite the possible contribution of many articles to broader society, they are not openly accessible, and this is particularly a problem for individuals who would like to contribute despite having limited financial resources or access to a subscribed institution. While information and communication technologies (ICT) has increased the reach of information, journal subscription fees have risen sharply in recent years (200% over inflation), resulting in many universities, including top tier research institutions, reducing their number of subscriptions (http://righttoresearch.org/).

Unfortunately, as subscriptions are cancelled by universities, the number of journal articles that researchers at that particular university are able to access decreases. Therefore, publishing in expensive toll-access journals can lead to a number of consequences which are both detrimental to the scholar who publishes an article and to those that are hoping to learn from the article or build on its findings. It affects the writer in terms of the visibility and impact of his or her work, while it affects the reader in the possible duplication of efforts. It can also result in the inability of the researcher to produce the most comprehensive or beneficial literature review, ignoring points of view that would have complemented the writer’s research. Fortunately, more information is becoming openly available, albeit much of it of limited quality.

To reduce these problems increasingly journals have decided to publish their articles openly by becoming an Open Access Journal (OAJ) or by allowing authors to retain some author’s rights and for the authors to be able to place a copy of the article that is accessible to the public through either their personal blog or their institution’s library website. By publishing openly, individuals across society could quickly access the scholar’s work and the benefit from understanding its implications, yet why is it that OAJ account for only ten percent of the total number of journals, according to recent Study of Open Access Publishing (SOAP) statistics (http://project-soap.eu/)? Part of the reason for the limited number of OAJs is because of the difficulties that a journal can face when trying to change its business model. Having a high quality peer review system and distributing articles to readers is very expensive. Editing is a time consuming and expensive process and should be adequately remunerated. However, because of ICTs, the cost of distributing articles to readers has been significantly reduced to the extent that they are usually only a minor expense. The rapid decrease of cost per calculation has reduced the price of hosting a site with the bandwidth needed for a journal to even less than $20 a month, depending on the journal. Being aware of the decreasing costs of running a journal, it is hard to justify the cost inflation of journal articles. In recent years, some scientist have promoted OAJ as an alternate journal business model, and have begun to campaign for increase openness within academia.

In particular the natural sciences have promoted an open access policy. The Arxiv (http://arxiv.org/) for example has over 719,000 academic pieces that are publically available and are published only months after the findings are obtained, instead of the longer periods of time of one or two year of administrative and editing processes that are needed to publish in more traditional journals. Particularly in fields such as information technologies, including the College of Education and Human Development’s (CEHD) program in educational technology, if an article takes two years to become available, that article is likely already out of date, and as such it is no longer of the highest quality or usability (For example the iPad version 1 is less than 2 years old). Unfortunately, while the natural sciences have promoted openness to increase the rate of innovation and their productivity, the social sciences and the humanities have attached more strongly to the traditional toll-access journal system. These focus groups tried to answer the why to that statement. Why would fields such as education, which hopes to improve the lives of others through their work and are not as concerned as the natural sciences in protecting patents, continue to primarily publish in traditional toll-access journals? With Open Access Week 2011 approaching, the UMN libraries decided to host a series of events (http://z.umn.edu/openaccess2011) for which I was asked to discuss my involvement on Reconsidering Development, an open access student run journal based at the University of Minnesota where I work as an assistant web-editor (http://journal.ipid-umn.org).  With this in mind, to supplement my presentation, I organized a series of four focus groups to discuss in greater detail CEHD students’ opinions about Open Access Journals

Recruiting of Participants

An invitation was sent to students two weeks before the focus group with a flier (http://z.umn.edu/oa2011poster) which directed them to the Right to Research Coalition website (http://righttoresearch.org/) as well as a Doodle poll where CEHD graduate students could register to attend to one of the four focus group sessions. The three in-person focus groups were held Thursday, October 20, Friday October 21, and October 24. An internet focus group was offered so that participants could join from October 21 to the 24 and answer the same questions in an asynchronous way ( http://oafocusgroup2011.freeforums.org). Another online focus group which answered a very similar set of question had also been held online from October 11 to October 14. Having held a total of 5 focus groups with CEHD graduate students discussing their opinions of Open Access (OA), this study feels that various saturation points have been reached and a subsequent study would benefit more in asking CEHD faculty members instead of students about their opinions regarding OAJ.

Participants did not seem to have any problems using the Doodle poll (z.umn.edu/openpoll2011). A couple of participants commented that they were attracted by the statements on the flier and having visited the included website before deciding to attend. A concern when selecting participants was that, despite regularly using open resources on the internet, participants did not generally consider themselves knowledgeable enough in the subject to openly share their opinion. Based on this concern, it is perhaps not surprising that various participants were members of the Learning Technology (LT) program within the Curriculum and Instruction (CI) department in CEHD. Purposely selecting participants, and sending every single participant a personalized email likely contributed to the high attendance rate. While there were only limited times when participants could attend a focus group and an email with a flier was sent to less than 100 students, 14 students signed up to attend the face to face (F2F) focus groups, while 10 students signed up to attend the online version of the focus group. The online focus group was held through freeforums.com. This site allows me to create user accounts and provide users with anonymity. This was mentioned to them before participating. In contrast to the online focus group, anonymity is not possible within the face to face version of the focus group. Out of the 14 scheduled face to face participants, one did not attend, resulting in 13 participants or two focus groups of 4 individuals and one focus group of 5 individuals. The online focus group had only two responses. Eight of the 10 students did not respond to the questions. However, based on the tone of their initial email response it is possible that some of these participants did not intend to participate online from the beginning. A number of them had mentioned that they couldn’t attend in person but would try to participate online. This could be interpreted as a way of expressing that they will likely be unable to participate. One of the two who participated mentioned that the questions scared him at first because being able to see all of them he felt it would take more than the time he could afford to share to answer them. However, after completing them, similar to the face to face participants, he expressed having enjoyed the focus group. Each face to face focus group lasted for two hours.

Below I summarized their responses into themes that were brought up repeatedly throughout the focus groups. In addition, for a visual overview of the different themes that were discussed in the focus group you can see or download a PowerPoint from the following link: http://www.slideshare.net/fastfonz/presentation-open-accessweek2011. It includes embedded audio clips of participants for most of the questions.

Defining Open: How do you define open, in terms of information? What qualities would you include in what it is to be open?

Students had similar opinions about the meaning of open. To them it meant that materials would get reused. A journal would be available from a computer and a mobile phone. They felt that it an article was open to the public, then it would reach a broader audience. Open access journals were also seen as helping individuals who are not academics to publish, allowing also for individuals to become a knowledge dispenser, as well as a knowledge receiver. One of the participants expressed how Open Access Journals helped increase transparency in the knowledge development process; “I can access it anywhere”. While to some participants it seemed that openness was linked mainly to greater accessibility, to others it included some of the other elements discussed within the open content literature and the benefit of being able to redistribute, remix, reuse, and revise information. Most of the participants expressed not being familiar with topics such as open education resources, open content, open data, and open courseware, but that they had found them interesting and were supportive of them after understanding them. In the online forum they were asked to visit links that explained the concepts after answering the question, then they were asked a subsequent question based on that experience.

Extent of the Influence: To what extent is the ability to access information freely on the internet influencing the type of information you consume? (not at all, some (little), a lot, a great deal)?

When asked to what extent having access influenced their research and education, most of them expressed an unwillingness to pay to read the news. While many used the Chronicle of Higher Education and the New York Times, they used the free portions of the site or accessed them through the university. Despite her interest in the paper, a student expressed her unwillingness to purchase the Wall Street Journal. By being open content was more accessible and influential. In terms of their academic work, a couple of students mentioned that they would reorganize their project if they could not access a particular source or simply dismiss it and change it for a different one. For one student, if a site required subscription, then they decided that it was not worth examining. Students use what is available, and mentioned that without a UMN ID they would not be able to have the same access.

How is Access Important: To what extent is the lack of access to information a problem? (not at all, some (little), a lot, a great deal)?

This question was somewhat related to the one above. In general lack of access led to either students borrowing access from someone or for them disregarding the source and attempt to find a similar article. They expressed how at the University of Minnesota they have more access than at smaller schools. There also seems to be a growing lack of patience with internet articles, and a student mentioned becoming frustrated after losing twenty minutes trying to get through all the security and sign up requirements when accessing an article off-site. The need to sign up to access even if the article was openly available was problematic and a source of frustration for some students. However, they felt that not having access to one article was not necessarily a major problem since; they could read more articles and find other studies. There seems to be a problem with sorting out sources and finding the best pieces of information.

Impact of Open Access Outside of Academia

While it was not its own question, many students expressed a concern about the impact that open access could have outside of higher education institutions. One of the participants mentioned how her father was able to learn more about her mother’s medical condition thanks to Open Access articles. A student from Africa mentioned his frustration of only being able to share a few articles with students. With internet being a privilege where he worked as a teacher abroad he would print a couple of openly accessible articles and share them with his students. Various of the students attending worried about not being able to access articles once they graduated and how important and helpful it could be to access these materials as an educator, a researcher, or an employee of an NGO. Organizations with a smaller budget will be more handicapped as into what they can afford to purchase. A student from Asia mentioned how there is a black market for articles in his home country and how it was more important for him, and more beneficial for society, if people were able to pursue their interests and satisfy their curiosity instead of limiting themselves to whatever the local institution can purchase. As mentioned before, there was also a concern regarding what could be accessed at a smaller university and how it was probably impacting students’ quality of education.

Generational Differences: To what extent do you think there is a generational difference in the support for openness, and open access journals?

While students seem to think that professors were not against the idea of open access, they argued that they seemed to be less aware of how it works or less optimistic about open access journals’ potential. However, students did not think of it so much as a generational issue, but perhaps also a disciplinary issue. Students then mentioned some of their problems with open access, including the difficulty of finding quality articles. A student mentioned how her professor warned her against using open articles. Another student wondered if younger students who grew up with the internet would be supportive of the toll-access model. In general, they felt that professors within CEHD were not very supportive of open access journals but that they encouraged students to publish in traditional journals. Trying to improve their marketability and obtain a tenure track job, many doctoral students feel that therefore it may be best to primarily focus on toll-access journals because of their prestige and because of the recommendations of their professors. One student, however, argued that it was important for students and professors to know that they can reserve most of their copyright and that many publishers are allowing faculty members who want to to publish while retaining many of their author rights.

Sustainability of OAJ: To what extent do you consider Open Access Journals to be sustainable? What do you think needs to happen for them to be sustainable?

One of the most common critiques of OAJ was their quality. To many students this is linked to them being free and a sense that they would accept articles that would not be accepted elsewhere. For OAJ to be sustainable it is therefore important to develop a funding model that allows them to pay for the staff needed in order to retain the quality of the journal. While in many journals the editing staff is composed largely by some of its readers who work pro-bono, understanding the importance of editing the work of their colleagues, journals have other expense such as calling for articles and organizing who need to edits which article. In some cases reviewers are paid. For the quality to be maintained and for OAJ to be sustainable, one student argued that perhaps OAJ could imitate internet games that function under the 1% or 2% models and obtain valuable funds and human resources in this way. Other students wonder if OAJs could sustain themselves by asking individuals for donations. Overall, they felt that money needs to come from somewhere and that asking individuals to pay to publish does not seem fair, “they should pay us to publish”.

Public vs. Private: Is there a difference between privately sponsored research and publically sponsored research in terms of access by the public?

Whereas some companies sponsor their own research, participants felt that if the public is paying for research then it should be available to them. A few students argued, that in their opinion, the medical field had done a great job in becoming as open as possible. NIH grants require that findings are released publically. A couple of participants were aware of this change and felt that education research, and research in other fields should be open to the public.  A student joked about why restricting access to education research seemed ridiculous and to go against the accepted aims of the field; “We have to keep our secrets about teaching and learning, nobody can know.” One participant turned the question around and asked, “In terms of access, should anything be private?” while another participant asked “is there anything that shouldn’t be public?” In addition, another participant mentioned that even sensitive data should be public because criminals will likely obtain access regardless of official controls on access. One participant mentioned that if private research is not open then companies could simply discard trials that were not positive and publish only the trials that were beneficial for them.

To what extent do you feel openness increases the quality of scholarship? (strongly agree, agree, neutral, disagree, strongly disagree)

Concerns about quality were likely the most negative attitudes students had regarding Open Access Journals. “The internet is filled with junk” was a common opinion and filtering this junk was a common concern. A student from Eastern Europe mentioned how she had difficulties accessing articles back home. In one instance, she found an archive with articles, but as she read through it, she noticed that it had not been accurately translated. This concern with accuracy and reliability was very common and led to one student repeating the advice given to her by her undergraduate instructor not to trust open access journals. However, a few students pointed out that they had used open access articles for their recent presentations and how some open journals have the same level of quality as toll-access journal articles. On a related theme another student argued that “if everybody was doing it, then there would be quality control. You want your stuff to look good anyway.” Another student also pointed out that while some online publications may publish anything they obtain, other open access journals have a very traditional peer review system. Because of the ease with which a site can be hosted, a journal with low quality can exist despite having a low quality product, yet other open journals can and do have the same level of quality as traditional toll-access journals. However, participants felt that many OAJ do not have the same level of credibility as traditional toll-access journals and, when most or all of the major journals in a field are subscription based, it is more likely that the faculty members, particularly older faculty members, will continue to publish in and advocate the use of these journals.

Moral Obligation: To what extent do you feel opening or increasing access to information is a moral obligation for educators, for the CEHD?

This question originated organically from the first focus group and was asked to the rest of the focus groups afterwards. One participant repeated various times during the focus group that access to education and knowledge should be a basic human right. In her opinion, without open access we are contributing to social reproduction as only the institutions where individuals with wealthier backgrounds attend will be able to access the most recent and relevant information. In her opinion piracy sometimes is warranted because we all have a responsibility to serve the common good. Another participant argued that if people felt that “they needed to follow unethical avenues; we need to restructure the way information is accessed.” During the focus group, information piracy was discussed as a social ill rather than an individual ill. Participants concluded that they needed “to make it possible for anyone who is seeking information to do so ethically… reworking the way we think…” Because of the moral benefits of openness and its relationship to the institutional goals of the University of Minnesota and the College of Education and Human Development one participant felt that the university itself should lead this transformation by encouraging faculty members and students to use and publish in Open Access Journals; “Academic institutions, humans, have a moral obligation to connect with each other and tell their stories”.

Innovation: To what extent do you think open access encourages or discourages innovation?

For most students open access would increase innovation because researchers could more easily develop accurate literature reviews and build upon the work of others rather than reinventing the wheel. One of the participants, explained it in the following way: “You see, we are doing something funny, we are trying to discover really good things, but then we put them in a box and lock them up. There are very wonderful things that haven’t been read…. I tried to access this article, and couldn’t access it. I thought this guy did a very good job, but what does that mean, I just forget it. I can’t pay the 39 dollars.” In this sense he and other students argued that open access contributed to innovation. Other students argued that it would increase competition and, as a consequence, journals would have to find ways to be innovative in order to increase their audience. Another student mentioned that capitalism and innovation thrive on competition so there would not only be innovation but that would also allow them to generate revenue and become sustainable. In conclusion, “if authors aren’t innovative (and if publishers aren’t innovative), readers will gravitate to other authors/publishers” Related to innovation, another student felt that open acess would result in a greater number of voices that would be heard internationally. Currently the World Bank and other major international organizations publish their voice, their viewpoint openly, but other voices are more silent. Openness would increase the visibility of other voices and, as a consequence, more countries may enact innovative policies and be themselves more innovative.

Future: Projecting yourself two, five and ten years into the future what role do you see open access playing for academia and society?

When discussing the future of OAJ, a few students pointed to how the natural sciences are at the forefront of the Open Access Movement, while the humanities, social sciences, and fields such as education are behind the curve. They argued that this is common with these fields as they are less technology oriented. However, in the future, they seemed to think that open access would likely be more commonplace. One of the students mentioned the rate of change would depend on key players in the various disciplines and their choices. Regarding quality concerns, students generally agreed that over time the quality of open access journals would likely increase. They questioned each other about how this could be done. “How were the other journals doing it… I just don’t know but it’s doable.” To various students it seemed as if the rate of change could snowball if more and more prestigious journals changed their financial structure or if authors increasingly became worried about being less cited if they only published in toll-access journals. “If everyone is invested in it, there would be a quality control. You want your stuff to look good anyway.” According to various participants, open access will probably develop layers or levels and through this system, some layers would use terms that are accessible to all while other layers would be more complex in terminology.

Choices: Which one of these three journals would you choose as the site through which to submit your article and what are some of the reasons you would chose this journal (please indicate what would be the most important aspect in making this decision)?

Despite the frequently expressed support for open access, CEHD graduate students were mostly concerned with publishing in a high impact journal or a “quality” journal at least during the first years of their careers. After obtaining tenure, they felt that they could more easily publish in open access journals. This was a very common statement, even among those that felt that open access should be a right. One of the few students that preferred an alternate route argued that her alternate route was not just in terms of publishing but because of the “academic game” and that she felt that she could accomplish more with her career elsewhere. To her, education should be about serving people, but writing for journals rather than the broader public was, she felt, a poor time investment. While she had been interested in academia before, the linking of success to publishing in prestigious journal steered her away from the profession.

Most other students, however, planned on becoming academics or publishing in their future careers. It is therefore important to create a strong professional image. One student mentioned that she would likely look at what the tradition is in a field. In a field with well-established traditions, even if it saddened her, she would likely publish in traditional toll access journals. Students expressed their interest in supporting open access journals, but at the same time were conscious of how important it is strengthen their CV. They stated that perhaps, over time, as the general trend changes, they will be able to publish more in open access journals, thereby supporting the initiative while not hurting their careers. Should open access journals become more accepted and gain prestige, students felt confident about the future of open access publishing and their ability to participate in it; “A hierarchy of journal prestige always exists.  Researchers are always trying to get their articles accepted by the most prestigious journal they can.”

ICT and the Digital Divide – A Participatory Solution with a Dominican Flavor

11/29/2011

The Digital Divides – The Ownership of Knowledge in the 21^st Century

The “digital divide,” a term coined in the late 20th century, highlights the growing disparity in access to information and communication technologies (ICTs) between and within countries (Norris, 2001; Compaine, 2001; Warschauer, 2003).  This concept, its meaning, and its implications for states across the world have been topics of widespread research and debate since the beginning of the 21^st century (OECD, 2000; Attewell, 2001; Goldfarb & Prince, 2007; Katz, 2008; Trucano, 2005; Mar, 2004; Chapman & Mahlck, 2004). Within the United States and abroad, public libraries and public computing centers have attempted to diminish the gap between technology haves and have-nots, yet many still lack access to computers, broadband Internet, and even mobile phones (Krebeck, 2010; InfoDev, 2010; The Economist, 2010; The Economist, 2005; Richardson et al., 2000). In addition, while the use of mobile phones is growing; smart phones remain inaccessible for the majority of individuals in developing countries. It is unclear whether reducing these inequalities will help countries to “develop” and “modernize” or whether it will increase reliance on imports, and promote capital flight while perpetuating global inequities.

Despite the promise of ICT, various studies of programs implemented in OECD states and LDCs found the initiatives had a debatable impact, obtaining only marginally positive results (Trucano, 2005). Studies surrounding the overall costs of ICT programs are also limited and only 10 to 15% of the total cost of ownership is attributed to the initial set up of the program (ibid).

Warschauer (2003) criticized the discourse of the “digital divide” and contended that the term should not be seen as a binary conception. He argued that such a notion “can even be patronizing because it fails to value the social resources that diverse groups bring to the table” (pg 7). The relationship between different groups of people and technologies vary and the development of a new technology does not by definition imply a benefit to the society. While Warschauer (2003) reminds us that technology is not a panacea, the material realities of ICTs extend beyond the discourse and its semantics (Peet & Hartwick, 2009). Unequal access to ICTs helps to perpetuate the economic differences between the core and periphery states, as the periphery continues to be dependent on the industries and products developed as a result of the knowledge accumulated by the core (Peet & Hartwick, 2009).

One of the primary reasons for the difficulties faced by poorer states in catching up with richer states has been the rapid, usually exponential, rate of innovation in technology through improvements in circuit technology. Since 1965, Moore’s law, or the exponential rate of technological innovation, continues to be supported by quantitative studies (Schaller, 1997; Kurzweil, 2005). Countries have increased their access to technology worldwide, yet the “divide” is a dynamic concept that changes as new technologies develop. ICT programs traditionally include radio, television, personal computers, the Internet or sharing of materials through a network, and mobile phone technology (Trucano, 2005).

In a study of 179 countries, Norris (2001) illustrated that there were three main “digital divides”, one between countries, or a “global divide”, one between social classes, or a “social divide,” and one between those who use resources and those that do not, or a “democratic[CEHD1]  divide.” Access to ICTs could be categorized in a different number of ways, and Norris’ categories highlight some of the differing implications of the “divide” depending on how it is conceptualized. It is also important to note that, similar to the use of the term “globalization,” the division between “information haves” and the “have-nots” is not a new concept; rather modern technologies have accentuated historical trends (Friedman, 2007; Sheppard et al., 2009; Compaine, 2001).

ICTs allow people to connect instantaneously worldwide, flattening certain aspects of international trade and communication (Friedman, 2007). Individuals can now find information about a large number of subjects without going to a library. Technology has transformed society, and it will continue to do so for years to come, and at a very fast pace. As such, Friedman (2007), along with other writers (Steinmueller, 2001; Friedman, 2007; Norris, 2001), has encouraged countries to invest in ICTs. To some, such investments are seen as a “stage” or objective to be met for a country to modernize and develop.

The rapid growth of ICT industries and their implications for states have led to transformational changes in society (Moravec, 2009). The first Apple personal computer, Apple Lisa, was released in 1984, the Internet was created in 1990 by Tim Berners-Lee, and the first mobile phone was developed by Motorola in 1974. Within just a few decades, mobile phone subscriptions extended from 12.4 million in 1990 to 4.6 billion by 2009, the Internet has reached most places in the world, and “research firm Gartner, Inc., predicts that there will be 2 billion computers in use worldwide by the year 2014” (Katz, 2008; Heeks, 2008; ITU, 2010). Facebook, a social network site which launched only in 2004, already has an estimated value of over 30 billion dollars (Hardy, 2010).

In some fields, ICTs have brought very positive changes, such as improvements in disability learning tools, distance education tools, long distance communications, and media creation. In various areas of the world citizens are becoming amateur reporters, recording and blogging about events that repressive governments attempt to censor (Diamond, 2010). Yet despite a large number of positive possibilities, some of these technologies are still used principally for entertainment purposes, and their potential for societal transformation and development has not been fully utilized. As long as states remain primarily importers of technology, rather than developers and producers, ICTs will perpetuate inequality. When the technologies are fully appropriated and the local population not only modifies and improves on the technology, but generates new technologies, the “digital divide” and knowledge dependency will diminish. To reduce inequality, education policy experts should focus on increasing access to Open Education Resources (OER) rather than simply attempting to “catch up” with richer societies by emulating their current use of ICTs [CEHD2] (Downes, 2007).

The proper use of ICTs could help bridge the knowledge and information gap, and aid in achieving a “new stage” of development. States could leapfrog past other states through a not yet developed, advanced system of online education which builds and intertwines aspects of Open Education Resources such as OpenCourseWare, Open Source Software, Open Books, Open Access Journals, Creative Commons licenses, and the cumulative creations made through Open Resource Economics (Jonestone, 2005; Downes, 2007; Benkler, 2008). By following Ted Berners-Lee’s call to “raw data now” and taking advantage of free materials such as YouTube EDU, ITunes U, and Connexions, as well as the future development of more open and freely accessible universities, poor states could use the Internet to perhaps eventually provide higher education for all, lifelong learning for all, and possibly reduce the costs of obtaining a high quality higher education degree (UNESCO, 2009; Faber, 2002; Lubas et al., 2004; Baraniuk, 2006; Berners-Lee, 2009). [CEHD3]

By using ICTs to create a more open and equitable society, and increase the local development of knowledge, states could move towards the development of “critical consciousness” and create their own path toward the future (Freire, 1974).  The growing use of technologies will also allow for the strengthening of local voices. ICTs are increasingly allowing its users to “rip”, “copy”, “reuse”, “mix”, and “burn” (Baraniuk, 2006). As Napster and peer-to-peer sharing transformed the Music industry, OER may transform our understanding of education. In his recent book, The Tower and The Cloud, Richard Katz (2008) wondered: “if a 300-year-old institution like Encyclopedia Britannica [could] be threatened in five years by Wikipedia, [could] other aggregators of expertise (aka colleges and universities) be similarly challenged?” (Katz, 2008).    Wikileaks?   J

Yet, more than a challenge, OER offers an alternative. OER can help local communities preserve their documents, materials, and extend the reach of their limited human capital through initiatives such as HP Brain Gain, the African Virtual University OER initiative[CEHD4] , and other programs which focus on increasing access to education for all. Rice University Connexions initiative of freely accessible textbook materials allows for the customization of chapters, free digital access to quality educational resources, and printing of cheaper materials (Connexions, 2006).

This essay will address how the Dominican Republic (D.R.) is attempting to reduce the digital divide through the use of ICT, in particular through the building of Community Technology Centers (CTCs) across the country and through the use of OERs. Yet while the D.R. has invested significant amounts of money on the development of ICT technologies, limited improvements in their public primary and secondary education may result in ICT benefiting primarily a certain sector of the population, while the poorest members of society continue to be excluded from the best schools and ICT technology. This essay overviews the educational system in the D.R., recent ICT developments, and policies that could be implemented to ensure that current investment in ICT reduces rather than increase inequalities.

The Dominican Republic – Rapid but Limited Growth – ICT for Education

With 10% of the population receiving close to 40% of the country’s GNP in 2007 (San Roman, 2009), the high level of inequality in the D.R. raises questions as to whether expenditure on ICT, rather than on improving the quality of the primary and secondary education levels, represents the best way for the country to increase equality while improving its economic outlook[CEHD5] . Yet, as technology plays an ever increasing role in modern society, it could also be argued that in order to leapfrog into the future, the D.R. needs to prepare its workers for the increasingly knowledge and innovation driven nature of the economy (Moravec, 2008; Davison et al, 2000), attempting not to imitate the west, but develop in its own style. With a limited tradition of research and innovation, the D.R. could invest in the development of OER to not only increase access to quality education material for all, but also to promote innovative and more constructive ways of teaching and learning. However, without an increased investment in education, present reforms will likely have a limited impact within the poorest sectors of the population.

The high level of inequalities in the D.R. dates back to the colonial era. Despite governmental efforts, the educational system and economic system developed by European colonists made increasing economic prosperity and reducing inequality a difficult challenge for former colonies to overcome (Bulmer-Thomas, 1994). The D.R. and neighboring Haiti had in place a plantation system in which only a small number of whites, constantly fearful of a revolt, controlled a much larger black slave population, who were constantly monitored and whose education was not encouraged (Moya Pons, 1992). As argued by Rodney (1972), the colonizers were fearful of educating slaves as it would make them more difficult to control. Out of up to 400,000 “taino” natives that inhabited Hispaniola when it was “discovered” by Columbus in 1492 only 3,000 remained by 1519 (Ibid. p. 29). Most of the natives died from epidemics as well as from the harsh treatment to which they were subjected by the colonists.

To continue the economic production and “development” of the island, the Spanish crown quickly began to import slaves from Africa and organized the island’s economy along various agricultural plantations. According to local documents, by 1546, black slaves accounted for around 12,000 individuals and were controlled by a white population of only 5,000 (Ibid. p. 34). While over time most of the population intermarried, and 84% of the population considers themselves mixed or of African ancestry, vestiges of the colonial system remain (CIA, 2010).

Just a few decades ago, in 1937, the then Dictator of the D.R,. Rafael Leonidas Trujillo (1930-1961), ordered the massacre of over 15,000 Haitians for having migrated to the D.R. without authorization. It is believed that many Dominicans with darker skin, and who resembled Haitians, also lost their lives. Trujillo’s government encouraged Europeans to migrate and open businesses with the aim of increasing the business sector as well as whitening the Dominican society. As a consequence, to this day, the poorest citizens of the D.R. are often of Haitian origin and tend to be of darker skin than wealthier families (Prado, 2009; Lozano, 2008; Murray, 2005; Lilon, 1999).

Recent demographic trends as well as a transition to a service economy have also increased the challenges faced by the government in fostering development. In recent years, the Dominican economy has transformed from a primarily agricultural economy to a service economy. Primarily through the growth of tourism, the service industry has grown to over 66% of GDP. This change has been beneficial to the D.R.’s economy, which has grown steadily since the 1980s, having an average economic growth rate of 5.43% from 2007 to 2009 and a Per Capita GDP of $8,300 (CIA, 2010). Free trade policies have led to the improvement of aggregate economic statistics for most Latin America countries, including the D.R.(UN Millennium Project, 2005). However, in 2008, 42% of the population lived below the poverty line (World Bank, 2008). In 2007, 11.2% of the population continued to live with less than a dollar a day (UNESCO, 2010), and the country had an unemployment rate of 14.9% (CIA, 2010).

Rapid urbanization and population growth has contributed to the government’s difficulties in developing the human resources needed to deal with the demands for new skills from the changing economy (OECD, 2008). While in 1950 the D.R. had a population of 2.134 million, of which 23.8% were urban dwellers, by 2010 the population had increased to 10.169 million inhabitants, with 68.54% living in urban areas (CEPAL, 2010). Similar to other developing countries that rapidly urbanized, the D.R. was faced with a complicated development challenge, a challenge that could be best met over time by empowering individuals to be creative and critical thinkers through a student-centered, progressive and competitive education system.

Previous governments attempted to increase both quality and access to primary and secondary education through a number of reforms (OECD, 2008, p. 92). In 1992, the country approved a major education reform through a 10 Year Plan (Plan Decenal) which promoted: 1) modernizing the educational structure, 2) the introduction of new technologies, 3) the promotion of literacy, 4) strengthening vocational education, 5) increased investment in education and 6) increasing community involvement (p. 104). The government programs yielded various positive results. Among them, the literacy rate increased steadily from 60.8% in 1980 to 70% in 1990, 78.5% in 2000, and 81.6% in 2008 (World Bank, 2010). Through Plan Decenal, the proportion of children between 6 and 17 that were attending school increased from 71.3% in 1991 to 86.4% in 2002 (p. 10).

However, despite moderate success in some areas, several indicators remain below expectations. By 2008, the average enrollment rate for Latin America and the Caribbean in secondary school was 94%, while in the D.R. enrollment was only 80%. The literacy rate remains below the regional average of 91% at 88.2% and total public spending on education (% of GDP) is low, at a level of 2.2% in 2007, compared to a world average of 4.6% in 2006 (World Bank, 2010). Primary education enrollment rate is also lower than the regional average at 104% compared to 116%[CEHD6] . In addition, according to a 2001 assessment of education throughout Latin America  by PREALC, urban children in the D.R. performed worst, and rural children only outperformed children in Bolivia and Peru, for whom Spanish is often a second language (Murray, 2005). The limited quality of the public education system has led to the rapid growth of private education institutions. While there were only 24 private schools in 1961, there were at least 2,500 by the year 2000 (Murray, 2005).

In recent years, however, the despite the implementation of the second Plan Decenal (2008-2018), the government has not substantially increased its education expenditure, instead focusing to a large extent on increasing access to ICT technologies for the general population. The government’s investments in ICT technologies recently resulted in the First Lady of the D.R. receiving the World Information Society Award in 2007 from the United Nations. She was honored for her contributions to building a more equitable and inclusive society by empowering Dominican families to improve their living standards through access to ICTs at Community Technology Centers (CTCs) (Cedeño Fernández, 2007). However, it is hard to assess to what degree these technologies are reducing inequalities and influencing primary and secondary education.

Other recent technological initiatives include the establishment of the Technological Institute of the Americas, (ITLA) founded in 2000, the Dominican Telecommunication Institute (INDOTEL), established in 1998, the Cyberpark of Santo Domingo, founded in 2001, and the new Network Access Point (NAP) of the Caribbean built in 2008 (INTEC, 2010; INDOTEL, 2010; PSCD, 2010; Dominican Today, 2008; ITLA, 2010).

The increased investment and emphasis on ICT development has contributed to the rapid expansion of Internet use in the D.R. in recent years, increasing from 183.687 Internet accounts in 2006 to 508.603 Internet accounts by June 2010.  Based on the average users per Internet connection, INDOTEL believes that 33% of Dominicans, or 3,214,371 people, had access to the Internet by June 2010 (INDOTEL, 2010). The D.R. has a cell phone use rate of 0.91 cellular phones per person (INDOTEL, 2010), and mobile phone coverage is increasing in rural areas. In addition, through the rural connectivity program, INDOTEL hopes to bring broadband access to every town with over 300 inhabitants by 2012 (San Roman, 2009; INDOTEL, 2010). The telecommunications industry has grown by over 15% annually between 1997 and 2004 with most of the traffic from the D.R. is destined for the United States (Stern, 2006). With 1 million, or 9%, of Dominicans living outside the country, most of them in the United States, the D.R. received 3.477 billion dollars in remittances in 2009 (World Bank, 2010). Remittances are not only a major source of foreign revenue but they have also fueled the growing development of ICT as families are in favor of investing to increase communication with their loved ones.

Regarding ICT and education, the government has promoted ICT through CTCs, the Educando.edu.do online community, Red-Wan, Virtual Areas for Education (AVES), and OER and software development at ITLA (ITLA, 2010; Khelladi, 2003) The most popular initiative in ICT for Education has been the CTCs. CTCs originated from the Costa Rican Little Intelligent Communities (LINCOS) initiative which was built in partnership with the MIT.  The project’s innovative approach of using decommissioned shipping containers to house computer labs proved enticing to donors (Granqvist, 2003; Granqvist, 2005; MIT, 2001).

This model was appealing to donors, but it was perceived as foreign, temporal, and inconvenient to Dominican users. LINCOS originated in Costa Rica, but it was exported soon after to the D.R. where it was subsequently modified. Granqvist (2005) argued that the lack of participation by the community in the planning stage of the program resulted in designers ignoring the fact that the containers felt hot and crowded[CEHD7] , and that some of the software and manuals, including the operating system, were on some occasions not in Spanish (Granqvist, 2005). The container model was dropped and replaced with a traditional building during the Hipolito Mejia administration (2000-2004), and it was further modified during the second presidency of Leonel Fernandez (2004-2008) to also include meeting rooms and a radio station (Prado, 2009). They are also hoping to include libraries and other educational materials within the CTCs. Working in collaboration with the Technological Institute of Monterrey, a pioneer in virtual education and OER, the CTCs include a variety of educational software and material (ibid).

Originally, most of the software and manuals were in English and they were limited in quantity and scope, yet that has increasingly changed as the initiative has developed (Granqvist, 2005). By the spring of 2009, there were 49 operational CTCs (Prado, 2009) and the Office of the First Lady has set a goal of building 135 CTCs in the upcoming years (Cedeño de Fernández, 2005). Comprehensively, the ICTs for Education initiatives in the D.R. are slowly bridging the digital divide. More and more individuals are obtaining access to advanced technologies, in doing so, the D.R. is meeting the the last Millennium Development Goal (MDG) 8-8f, which promotes increasing access to telephone lines, cellular subscriptions, and Internet use per 100 population (UNDP, 2010). The likelihood that OER and other educational technology tools will improve over time and the constructionist nature of their design allow for individuals who were previously unable to study to have the opportunity to learn how to use computers, operate a radio, learn how to read, as well as use the educational software within the computer. [CEHD8] Such resources provide a one of a kind opportunity for the individuals that have access to these newly built centers (Lozano P., 2007).

However, are the centers visited by a large percentage of the population? Who visits them? Prado’s (2009) study showed that, despite being located in rural areas where there is a higher poverty rate than in the cities (54.1% live under the poverty line),  most of the users of the three CTCs she surveyed had a higher level of education than the average population (World Bank, 2010; World Bank, 2008; Prado, 2009). According to Prado (2009) , 61.5% of the CTC users in the locations she studied had completed high school, a level which is significantly higher than the national average in 2004 of 40%. In addition, 12.8% of the CTC users had a university degree, in comparison to the 3% nationwide enrollment rate in higher education in 2008 (pg 80).

Therefore, are these ICTs programs reducing or increasing the digital divide? Being unable to find a job with computer or IT skills in rural areas, individuals who become competent in ICT are likely to migrate to the cities. ICT also do not seem to be targeting individuals of Haitian descent. Prado’s (2009) studied showed that, as a result of anti-haitianismo, individuals were hesitant to label themselves as Haitian or of Haitian [CEHD9] descent (Prado, 2009). Similar to previous studies mentioned within the InfoDev’s ICT Knowledge Map (Trucano, 2005), studies of the D.R. and the effectiveness of ICT are not supported by education outcomes data[CEHD10] . After modifying the LINCOS project and increasing the participatory element of the CTC establishment process, individuals appear to be more supportive of the project (Prado, 2009; Granqvist, 2005) yet the initiative appears to be serving only a limited number of individuals, at high operating costs.

While CTCs and increased broadband access, as well as the introduction of educational programs through mobile phones, could, in the future, extinguish the digital divide and promote lifelong learning, current international indicators show that the D.R. public education system is failing to meet the MDG for universal primary education, while only a few students are benefiting from their ICT initiative. As such, the D.R. could pursue two courses of action; either increase, and perhaps as much as triple, the budget allocated to public education in an attempt to better national results, and/or increase access to ICT for every individual in the country by joining the OLPC initiative and providing every single child with a laptop and CTCs in all communities. In order to leapfrog and properly meet the challenges facing the D.R. in the 21^st century, a much greater national emphasis needs to be given to all areas of education, not just ICT. This effort should be linked to the national Dominican identity, by promoting athletic activities such as baseball, and music education after school programs similar to Venezuela’s “El Sistema”, but with a bachata or merengue emphasis. While there have been noticeable improvements during the past decades, the D.R. continues to lag behind in educational achievements compared to other Latin American countries (Murray, 2005). ICT development and reducing the digital divide are part of the solution, but to fully reduce the inequality that have plagued the D.R. since the colonial years, a greater expenditure on education and improved relationships with Haiti would be beneficial.

Very well written. The next great frontier for ICT research is to follow up with users on the impacts of ICTs. You aptly noted that this is missing in the field, but I hope you will help fill this void in the years ahead.

A

Works Cited

Baraniuk, R. (2006, August). Richard Baraniuk: Open Source Learning. Retrieved November 3, 2010, from TED Ideas Worth Spreading: http://www.ted.com/talks/richard_baraniuk_on_open_source_learning.html

Bulmer-Thomas, V. (1994). The Economic History of Latin America Since Independence. Cambridge: University of Cambridge.

Cedeño de Fernández, M. (2005). Launch of the Digital Solidarity Fund (DSF). Geneva: Oficina de la Primera Dana.

Cedeño Fernández, M. (2007, May 17). World Information Society Award 2007 Laureates. Retrieved 12 3, 2010, from International Telecommunication Union: itu.int/wisd/2007/award/cedeno.html

CEPAL. (2010). CEPALSTAT | Base de Datos y Publicaciones Estadisticas. Retrieved from CEPALSTAT: http://www.eclac.org/estadisticas/

CIA. (2010). The World Factbook. Retrieved from Central Intelligence Agency: https://www.cia.gov/library/publications/the-world-factbook/geos/dr.html

Compaine, B. M. (2001). The digital divide: facing a crisis or creating a myth? Massachusetts: MIT Press Sourcebooks.

Connexions. (2006). Connexions: Sharing Knowledge and Building Communities. Houston: Rice University.

Davison, R., Vogel, D., Harris, R., & Jones, N. (2000). Technology Leapfrogging in Developing Countries – An Inevitable Luxury? The Electronic Journal of Information Systems in Developing Countries , 1-10.

Diamond, L. (2010). Liberation Technology. Journal of Democracy , 69-83.

Dominican Today. (2008, August 12). NAP of the Caribbean. Retrieved December 4, 2010, from Dominican Today: http://www.dominicantoday.com/dr/technology/2008/8/12/29025/NAP-of-the-Caribbean-starts-operating-in-Dominican-cyberpark

Downes, S. (2007). Models for Sustainable Open Educational. Interdisciplinary Journal of Knowledge and Learning Objects , 29-44.

Freire, P. (1974). Education for critical consciousness. New York: Sheed & Ward.

Friedman, T. L. (2007). The World is Flat. New York: Picador.

Gajardo, M. (2008). Education for All Global Monitoring Report: Dominican Republic – Country Case Study. New York: UNESCO.

Granqvist, M. (2003). Assessing ICT Efforts in Marginalized Regions from a Critical Social Viewpoint – Learning from the Case of Lincos. Olistica, 1-6.

Granqvist, M. (2005). Looking Critically at ICT4Dev: The Case of Lincos. The Journal of Community Informatics , 21-34.

Hardy, Q. (2010, September 23). In Zuckerberg We Trust. Retrieved November 4, 2010, from Forbes: http://www.forbes.com/forbes/2010/1011/rich-list-10-technology-facebook-google-laws-zuckerberg-we-trust.html

Heeks, R. (2008). ICT4D 2.0: The Next Phase of Applying ICT for International Development. Computer , 26-33.

INDOTEL. (2010). Conectividad Rural de Banda Ancha – Segunda Etapa. Retrieved December 3, 2010, from http://www.indotel.gob.do/proyectos-indotel/proyectos-indotel/conectividad-rural-de-banda-ancha-segunda-etapa.html

INDOTEL. (2010). Quien es el INDOTEL. Retrieved December 3, 2010, from Instituto Tecnologico de las Comunicaciones: http://www.indotel.gob.do/conoce-al-indotel/conocenos/quien-es-el-indotel.html

INTEC. (2010). History. Retrieved December 3, 2010, from INTEC – Instituto Tecnologico de Santo Domingo: http://www.intec.edu.do/sobre/historia_intec.html

International Telecommunication Union. (2010). The World in 2010: ICT Facts and Figures. Geneva: International Telecommunication Union.

ITLA. (2010). Filosofía institucional. Retrieved December 5, 2010, from ITLA – Instituto Tecnologico de Las Americas: http://www.itla.edu.do/app2/index.php?option=com_content&view=article&id=248&Itemid=125

Jimenez, E., & Lockheed, M. E. (1995). Public and Private Secondary Education in Developing Countries – A Comparative Study. Washington D.C.: World Bank.

Katz, R. N. (2008). The Tower and The Cloud: Higher Education in the Age of Cloud Computing. Boulder: EDUCAUSE.

Khelladi, Y. (2003). Iniciativas públicas para la difusión social de las TICs en República Dominicana. Santo Domingo: Funredes.

Lilon, D. (1999). Propaganda y política migratoria dominicana durante la Era de Trujillo (1930-1961). Historia y Comunicacion Social , 47-71.

Lozano, P. (2007, August 07). Centros Tecnológicos Comunitarios PART 1. Retrieved December 3, 2010, from Youtube: http://www.youtube.com/watch?v=MN7qhxvOyRE&feature=related

Lozano, W. (2008). La Paradoja de las Migraciones. Santo Domingo: UNIBE.

Luxner, L. (1997, Sep/Oct). Caribbean Gateway to the Web. Americas, pp. 4-5.

McMichael, P. (2004). Development and Social Change: A global Perspective. Thousand Oaks: Sage Publications.

MIT. (2001). Lincos – Unwiring the World – The Vision. Retrieved December 6, 2010, from MIT Media Lab – Unwired: http://www.media.mit.edu/unwired/index.html

Moravec, J. W. (2008). A new paradigm of knowledge production in higher education. On the Horizon, 123-136.

Moravec, J. W. (2009). Technological Applications of Leapfrog. Minneapolis: Leapfrog Institute.

Moya Pons, F. (1992). Manual de Historia Dominicana. Santo Domingo: Caribbean Publishers.

Murray, G. (2005). El Colegio y la Escuela. Santo Domingo: Fondomicro.

OECD. (2008). Reviews of National Policies for Education. Paris: OECD.

Peet, R., & Hartwick, E. (2009). Theories of Development: Contentions, Arguments, Alternatives. New York: The Guilford Press.

Prado, P. (2009). Bridging Digital Poverty: Adoption of Information and Communication Technologies at Community Technology Centers in the Dominican Republic. Miami: University of Miami.

PSCD. (2010). About PCSD. Retrieved December 4, 2010, from PCSD – Parque Cibernetico de Santo Domingo: http://www.pcsd.com.do/English/about-pcsd.html

Sachs, J. (2005). The End of Poverty. New York: Penguin Press.

San Roman, E. (2009). Bringing Broadband Access to Rural Areas – The Dominican Experience. Santo Domingo: International Telecommunication Union.

Stern, P. A. (2006). Promoting Investment in Information and Communication Technologies in the Caribbean. Washington D.C.: Inter-American Development Bank.

UN Millennium Project. (2005). Investing in Development: A Practical Plan to Achieve the Millennium Development Goals. London: Earthscan.

UNDP. (2010). Goal 8: A global partnership for development. Retrieved December 4, 2010, from United Nations Development Programme – Millennium Development Goals: http://www.undp.org/mdg/goal8.shtml

UNESCO. (2010). UIS STATISTICS IN BRIEF – Education in Dominican Republic. New York: UNESCO.

World Bank. (2008, September). Dominican Republic at a Glance. Retrieved November 30, 2010, from World Bank: http://devdata.worldbank.org

World Bank. (2010). Dominican Republic Indicators. Retrieved December 4, 2010, from The World Bank: http://data.worldbank.org/country/dominican-republic

World Bank. (2010). Migration and Remittances – the Dominican Republic. Washington D.C.: World Bank.

World Bank. (2010). World dataBank: Dominican Republic. Retrieved from World Bank: http://data.worldbank.org/country/dominican-republic

Invisible Learning in the Community Technology Centers in the Dominican Republic

11/29/2011

A case study about how increased access to information through modern information and communication technologies is impacting the informal and formal education of Dominicans.

Chapter 1

–        A growing number of individuals lack access to a higher education. There are unmet expectations. Discuss statistics which mention that despite a growing access to primary and secondary schools worldwide, there is a lack of human capital and other resources needed to meet the demand that is currently present and will continue to grow for access to higher education institutions.

–        Information access is growing because of the growing ubiquity of certain ICTs. Explain how ICTs are not only spreading quickly as few other technologies have before, but also, that since they carry information with and through them, therefore they have a broader educational impact than those the spread of a other types of technology such as new biotechnologies or energy technologies, etc. ICTs increase connectivity, communication and the transfer of information.

–        In the Dominican Republic one can find both of these situations converging, happening simultaneously. The DR is in close proximity to the United States and this proximity influences their economy and technology adoption.

–        The Dominican Republic has spent funds to improve both situations, focusing on access to ICTs (Low educational investment and achievement when compared regionally to other countries. A high level of investment in ICT including broadband access, and rural connectivity)

–        One of the key programs implemented by the government in the Dominican Republic that has increased access to ICT are Community Technology Centers. Brief explanation of what they are, and how they have been increasingly localized over time.

–        This situation is not unique to the Dominican Republic or Latin America but rather most countries. Mentioning of how telecenters, CTCs, and other ICT projects which aimed at increasing access have been implemented in the region over the past decades.

–        What is ICT, explain this and other definitions, including OER, and also Open Access, information piracy, educational formation, invisible learning, CTCs, Moore’s Law, Prosumer (some will be explained within the text, but the most general ones and important ones will be explained in this section)

–        What has been the result of the government investment in the Dominican Republic, and what are some of the learning programs which are currently taking place in the CTCs and how is it impacting its users.

–        A discussion of Information. An overview of what is considered private, open, and pirated in terms of information and how access to information is increasingly seen by some as a human right.

–        A discussion of visible vs. Invisible learning, formal vs. informal learning. A question as to where or in which of these areas is the impact of ICT in the DR.

–        What educational resources are currently used and how?  To what extent are they localized? To what extent are they contextualized? What is localization to them? What is contextualization to them? Are they producing or just using? What are they producing?

–        A discussion of the study and what it aims to do through the use of ethnographic methods. How transferrable and relevant is it? Why is this study significant, and some expected limitations.

–        Brief Outline of all chapters (Overview – Goal, Theoretical Framework, Methodology, Conducting the Study and Analyzing the Data, Conclusion).

Chapter 2

–        Taking increased access to information as a given, while technology increasingly exerts a major influence in our lives; some technologies impact the ideas that we assimilate more than others, our formation, our education.

–        ICT and information debate. Different perspectives as the type of access to information that should be available and what has prevented greater openness to information. Quality concerns vs. access concerns. What is knowledge, and what is valuable information. What role does technology play in the dissemination of information? What is the directionality of the transfer?

–        Positivism and post-positivism scholars and a discussion of the impact of technology in education. How to some technology is seen as a primarily positive influence to education, both increasing access and opportunities. Modern ICT are seen as a way in which the world is increasingly interconnected.  Technology is seen as primarily a positive influence on society and the world, including a positive influence in terms of increased information diffusion.

–        Critical theorists and constructivist scholars and their views surrounding technology and education. Technology and information transfers are criticized and more specifically categorized. Geography matters and so does the local economy and needs of an area. Technology is a tool that can help at times to improve local conditions, including increasing access to information or protecting and diffusing local ideas. There are plenty of failed examples of poor implementation of access to technology and technology and education programs.

–        Classicists and critical constructivist scholars and their views of the impact of technology in education. While it is hard for anyone to deny that there is an increased access to information through technology in modern society and that it is greatly influencing youth, the impact of this influence is questioned as individuals are increasingly bombarded with information but having less time to reflect and form relationships and knowledge out of the information. The increased information may be affecting individuals’ ability to unplug or reflect on past experiences. Less information is internalized and there is also an erosion of local cultural values.

–        How knowledge is contextualized or localized and does this matter? Discussing ways in which ICT has helped and could hurt to preserve local knowledge. A discussion of the directionality of OER. Differences as to how open resources are, and how by being open they can be adapted and localized to different extents. Discuss two different paths taken by the open education movement in terms of information licensing and how while both increased access to resources, the way in which they can be localized is different.

–        Theories as to how contextualization, localization, and participatory ways of learning have the potential to empower the local population. How through the internet and other ICTs individuals can not only learn from each other but they can also share their own knowledge with the rest of the community and the world.

–        Discussion of what educational information is currently accessible thru the internet and how access to this information is important. Highlight that the study expects that the CTCs will have all three types (open, closed, and pirated) of information available. When discussing information, the study divides and discusses information as open, closed, and pirated. Brief overview of how pirated or illegal access to information and how its use is perceived varies by country. The important of knowing what information is being used for learning in the CTCs and what are they learning. Discussion of theories that argue that closed is better for economic growth and innovation and how close would also reduce the impact of some types of information and its erosion of local cultures worldwide.

–        Discussion of how to some, not all information should be openly available, but how high quality information that was paid by public funds is not currently available to tax payers and the rest of the world. Will diffusing this information help the developing world?

–        Theories and opinions of technology centers from the ICT4D literature. How have these centers impacted local areas in the past? Discuss the best and worst experiences and how they are perceived by ICT4D scholars. Hint at where the Dominican CTC project fits within this literature and how they are attempting to address several of the concerns mentioned below but how the extent to which they are succeeding and its impact changes rapidly and is not fully known.

Wikieducator.org – OER[YUN1]  Online Community – An Online Community Ethnography

11/29/2011

Increasingly interacting with digital media, spending thousands of hours watching television, surfing the internet, or playing video games, an ever growing number of individuals can be considered digital natives or digital immigrants (Prensky, 2010; Tapscott, 2008). As a result of this recent change, it is important to understand how these interactions affect individuals living in our time and space, their cognitive abilities and what it all means for the youth of tomorrow. These are among some of the topics that have recently risen to challenge the modern ethnographer. The rapid technological change witnessed by societies in recent years has led to an increasing disconnection between generations. Phrases such as the Net Generation, Digital Natives, or Google Generation are surfacing regularly in popular news channels and have been the subject of recent publications (Tapscott, 2008; Prensky, 2010; Rowlands, et al., 2008). Whether one considers oneself a late adopter or laggard, or an innovator or early adopter we all should have an interest in understanding these changes (Rogers, 2003; Kurzweil, 2006). Therefore, it is important for studies to be conducted that would help us to understand better what happens during the thousands of hours that digital natives spend exploring virtual worlds and immersing themselves in epic digital adventures[YUN2]  (Harry, 2005; Yee, 2006).

By allowing individuals to connect with anyone in the world who shares a similar interest, subcultures are able to thrive at a level only possible before in large urban settings.  Through the internet, individuals can code a program collaboratively, play video games internationally, live “second” lives, buy property within online environments, and develop online learning communities based upon a shared vision and mission such as is the case of Wikipedia and WikiEducator. Through websites such as Ning.com and Groups.Google.com, individuals can connect with likeminded people and change the method through which knowledge is produced. Some of these communities, such as Wikipedia, have been the subject of both intense praise and criticism in press and journal articles as a result of their innovative but controversial collaborative framework and organizational culture.

In a whirlwind of change, Wikipedia, has in a decade, become one of the most visited websites in the world. It encompasses millions of articles that anyone with internet access can create and edit. Wikipedia’s success has been to a large extent a result of its wiki structure, the charismatic leadership of “benevolent dictator” Jim Wales, and its consensus oriented decision making process (Reagle Jr., 2010). Wikipedia has risen to challenge even the highly regarded, professionally written Encyclopedia Britannica, first published in 1797 (Kafker & Loveland, 2009; Katz, 2008). The degree of impact Wikipedia has had as a freely accessible reference tool and the transformative power that results from collective or cooperative knowledge production warrants further study. According to the most recent Alexa rakings, Wikipedia was the 8^th most visited website on the internet from February to April 2011, used primarily by childless people under the age of 35 with postgraduate educations browsing at school or work (Alexa, 2011).

Since the success of Wikipedia, other Wiki websites have attempted to replicate its model. Currently there are many other popular wikis, and some of them, such as Wikia.com and WikiHow.com, have been the subject of recent scholarly research (Levine, 2006; Murley, 2008). This paper analyzes a different wiki community, WikiEducator, which has the potential to bring about a transformative change in education and the way in which youth will learn in the future. WikiEducator hopes to build a worldwide community of educators focused on the development of quality open education resources for K-12 and Higher Education. The success of WikiEducator and OER[YUN3]  would not only enhance education in developing countries but also in developed countries, by supplementing traditional education.

Through the development of Open Education Resources (OER), freely sharable educational materials that can be modified, remixed and redistributed, sites such as WikiEducator may play a role in improving access to high quality educational content and in meeting the goal of creating an increasingly affluent world (Baraniuk R. , 2006; Kamenetz, 2010; Sachs, 2005). Another possible impact of OER would be to reduce textbook costs by either replacing proprietary textbooks or increasing competition among publishers. OER could lead to the increased reuse, modification and redistribution of educational material, provide students with additional resources to supplement their course materials and improve the number and quality of lifelong learning resources.  For some, this type of resource could even potentially contribute to the development of an online only educational system where students would be able to obtain a quality higher education but pay only a small amount in fees. As Jim Taylor (2007) remarks OER are not “intended to threaten existing models of higher education provision, but to create a parallel universe capable of ameliorating the apparently insurmountable problem of meeting the worldwide demand for higher education.” Just “India alone would need nearly 2400 additional universities in the next 25 years – or roughly two new universities per week” to meet the demand for higher education (HE) as  more students finish secondary education or need to return to school in a search for a new career or simply for enjoyment (Daniel et al., 2007).

The objectives of people involved in WikiEducator vary by member, yet they all share the conviction of its founder, Dr Wayne Mackintosh, whose vision is to turn “the digital divide into digital dividends using free content and open networks” (WikiEducator, 2010). Similar to Wikipedia’s Jim Wales, Wayne Mackintosh shares some of the charismatic appeal and friendliness of a “benevolent dictator”. Yet, Mackintosh is only one of many scholars who have written about the transformative potential of OER.

Although the term OER was first coined by UNESCO in 2002, open educational resources have been available since educational materials (formal and informal) were first developed and freely shared with someone else. As social interactions fill our world with meanings, the first educational material was probably a story that was shared orally by members of the community. Hearing a story and retelling it with a slight modification is the equivalent of the remixing idea promoted by OER. More than just free of cost, OER are concerned with freedom of access and the ability to build upon previous works by modifying or remixing them (Baraniuk R. , 2006). OER today are associated primarily with digital resources yet, depending on the definition, OER also include many non-digital materials. For OER supporters, it is important for information to be available through a multiplicity of formats. One of the benefits of OER is their potential to increase the amount of high quality educational materials available online, and to reach individuals with different learning modalities, who can then access materials through a number of devices, asynchronously, at no or much reduced costs.

Since 2002 the OER movement has been spearheaded by the OECD and the Hewitt Foundation together with UNESCO. Various institutions such as MIT, UK Open University, Carnegie Mellon, Rice, Stanford, Yale, and Berkeley have developed, or promoted the development of, OER (OECD, 2007; UNESCO, 2011; Walsh, 2010).  Other organizations or online communities have also been pivotal in promoting OER, such as Saylor.org, Opencollegetextbook.org, Creativecommons.org, and Opencontent.org. Yet one of the few communities that has centered specifically on the development of OER content is WikiEducator.

This paper is a report of an ethnographic study of the WikiEducator community I conducted during the spring semester of 2011, as the precursor to a larger analysis and more in depth study of WikiEducator. This paper used a constructivist methodology in the hope of understanding what WikiEducator represents for its members and to understand some of the inner functions of this online community. As an exploratory study, this study also attempted to establish long term relationships with various members of WikiEducator through the use of semi-structured interviews and participant observation.

Value Premises

Whether we like it or not, bias is there, and despite attempts to reduce it, a researcher should be aware that his/her interactions will affect the results of the study. His/her interaction with participants will be unique and could never be fully replicated (Wolcott, 2008; Fetterman, 1989; Kouritzin, 2002). Because bias is unavoidable scholars stand to benefit and aid their readers by stating their beliefs openly (Myrdal, 1972). As a researcher, I find it difficult to separate the private and the public; writing is to me, as to many critical theory researchers, inherently political (Hanisch, 1970; Elliot, 2009; Crotty, 1998). As Foucault eloquently wrote; “knowledge is not for knowing; knowledge is for cutting” (Foucault, 1984). Through my writings, I hope to show both the limitations and capabilities of OER, yet personally I am interested in the success of this model and the reduction of HE costs worldwide. Having benefited to a great extent from the opportunities that were available to me through a result of both hard work and serendipity, I am aware that while I have succeeded others were not as lucky. Many individuals out there deserve the chance to explore their full potential, to have the opportunity to access higher education, obtain an advanced degree, and feel empowered through their education. To break the cycle of dependency, we need to work so that more and more individuals from low income economies can obtain this opportunity.  It is my personal objective to increase access to educational resources and, by doing so, reduce the achievement gap. Yet, while I am passionate about the possibilities of OER, as a scholar it is important to question assumptions and explore even counterintuitive ways to bring about the necessary changes. As OER scholars debate whether or not private primary schooling may have certain advantages over public primary universal schooling in terms of quality, even at low income levels, so too should the sustainability of OER and its financial model continue to be questioned (Wiley, 2007; Downes, 2007). One of the most frequently expressed concerns by WikiEducators is the need for increased government support and the limited amount of grants available from nongovernmental organizations (NGOs) or national governments. Other organizations are hesitant to support the initiative, citing fears that the spread of OER could endanger their job security. Another important concern relates to the distinction between universal knowledge, or “free culture,” and knowledge that should remain private to stimulate further innovation and  for national security purposes (Lessig, 2005; Kramer, Starr, & Wentz, 2009).

Methodology and Methods

This study conducts an online or virtual ethnography of the WikiEducator community by comparing it to the literature on online learning communities and online ethnographies (Palloff, 2009; Gee, 2003; Nardi, 2010; Boellstorff, 2008). While the digital world offers new possibilities, it also brings new challenges for ethnographic research due to indirect nature of the interactive space (Fetterman, 1989). Online environments, such as Second Life and World of Warcraft, offer their users the possibility to escape their physical environment and live in a world of fantasy (Boellstorff, 2008; Nardi, 2010). Researching on the internet raises a number of complications, such as the limitations of interacting through the internet, complications of obtaining access to information or observing behavior online, difficulties of real vs. constructed fictional identities and lack of context apart from text or voice. However, some of these concerns are being addressed through the increased capabilities of information communication technologies (ICT) as well as through the graphical representation or increased complexity of online environments (Bilir, 2009; Harry, 2005; Wittel, 2000) Online ethnographies will become increasingly relevant as more and more individuals spend time interacting with digital technologies (Correll, 1995; Hine, 2000). This study builds on prior literature by providing a window into an increasingly popular wiki environment (Friesen & Hopkins, 2008). WikiEducator illustrates the possibilities of open-source economics and how a small group of people can effect macro changes through online collaboration (Benkler, 2009; Reagle Jr., 2010).

Methods

To increase the reliability and transferability of the study, I triangulated my methods using participant observation, a literature review, interviews, and discourse[YUN4]  analysis of forum data (Wolcott, 2008). This enabled me to understand the community more deeply. By reading forum discussions as well as conducting interviews, I developed a deeper relationship with respondents. This paper focuses on my interviews and the participant observation aspects of the study.

For this study, I spent various hours a week visiting WikiEducator.org and related OER organizations. During my study the interrelated nature and support between different OER organizations was evident. Other websites that were visited include the OERU Google Group website (Pic 1), Scope OERU Forum (Pic 2), and the WSIS OERU forums (Pic 3). While these sites are managed separately and have their own community numbers, many of the members of one of the communities are also members of the other three. The other three communities visited include a forum for discussion, yet it is only in WikiEducator (Pic 4) where participants are developing content. During my study of the community I came to feel that I was becoming a part of it, transforming myself into a WikiEducator. As part of my participant observation, I wanted to participate in the development of resources and observe how they were developed. In addition, to feel more integrated within the community, I developed a WikiEducator profile page and became involved in the creation of OER, but I was unable to develop as much content as I originally intended. Collaboration when developing OER is primarily asynchronous and it seemed to be at a lower level than I had originally expected.

During the interviews, it was common for WikiEducators to mention having a personal relationship with Wayne Mackintosh and the close knit nature of the OER community, with many of its administrative members communicating regularly over Skype or via e-mail. As an online community the members are spread throughout different countries and the use of ICT technologies is needed for the coordination of events and the development of objectives that guide OER production. Despite of the high level of interest and possibilities for OER in the developing world, most of the members of WikiEducator lived in countries with[YUN5]  high income economies such as the USA, Canada, New Zealand, France, Australia, and the United Kingdom. However, there were also members from countries with low income and middle income economies, including Jamaica, Guyana, India, Trinidad, Samoa, Brazil, among others.

To understand the OER community better and, in particular, WikiEducator, I attended a number of OER / WikiEducator sponsored events, which included a planning meeting on February 23^rd and a five day course on Open Content Licensing for Educators (OCL4Ed). By attending[YUN6]  these two events, I was able to participate as a student in WikiEducator’s educational activities, and learn more about the OERU University initiative. The February 23^rd  meeting was broadcast live through UStream, an online environment that allows attendees to interact digitally with those in the meeting through the UStream chat or by posting messages in indenti.ca, an open social microblogging service similar to twitter. The OCL4Ed course allowed me to see firsthand the quality of their education materials and to become familiar with key texts on OER.

Participant Observation

The February 23^rd meeting shed light on the different connections WikiEducator has with the Commonwealth of Learning and UNESCO, as well as Athabasca University, Otagon Polytechnic and the University of Southern Queensland. The meeting included presentations from a number of professors including Jim Taylor, Phil Ker, Robin Day, David Porter, Paul Stacey and Wayne Mackintosh. The model to follow for the development of an OERU was discussed, as was the history of OER and the need to increase credibility. The concept of the OERU and whether or not the accreditation system would be free were two other major topics of debate. Contrary to popular opinion, OER are not necessarily free. While most of the content may be obtained from the internet free of charge, the OERU was primarily envisioned as consortium of universities that would, after accepting a commonly agreed upon standard for a particular degree, accept a standardized set of courses from any university and provide the students with a degree. Starting with three open universities in three countries, the OERU would originally cost students only a small administrative fee (Pic 5).  In theory, the project would allow anyone to take courses from a variety of places, to register and obtain a diploma or take the courses they have yet to complete to obtain the degree. This added flexibility would permit students to receive credits for courses they have taken in the past, or even in different countries. During the meeting they opened an identi.ca feed #OERU and after a few suggestions they decided to project microblog posts on a wall via a projector. However, despite efforts to include the voice of those attending via the internet, the ability to participate virtually was limited. The planning session also included a couple of closed meetings, yet, as can be seen in the agenda, most of the event was open to the public and the conclusions to those meetings were shared with virtual attendees when the general assembly reconvened[1]. A number of prominent OER figures such as Stephen Downes and Pheo Martin as well as 202 other members from 46 countries registered for the meeting, many of whom attended virtually. Various virtual participants later expressed their opinion about the event through their personal blogs[2].

At the beginning of the meeting, Sir. John Daniel talked about the need for focusing on innovation in only a few aspects at a time to increase credibility, and the need for an open but tough institution. Daniel, the CEO and President of the Commonwealth of Learning (2004-Currently), is considered the founding patron of WikiEducator. As the former Vice-Chancellor of the UK Open University (1990-2001) and having served as UNESCO’s Assistant Director-General for Education (2001-2004), he is enthusiastic about finding new ways in which to extend education to individuals who currently are unable to attend a university. Various countries now face a situation where millions of students are college ready, having benefited from the successful growth in attendance and graduation from primary and secondary education, but lack access to higher education (Taylor, 2007; Daniel et al., 2007). It is important to meet this challenge, but in a way that allows students to obtain a credible credential at an affordable rate.

Following the meeting, I visited a number of blogs from participants and familiarized myself with the resources available through WikiEducator.org.[3] I completed the wiki editing tutorial and edited a basic wiki page.[4] I then visited the page catalog, which includes links to all of the wiki pages available through WikiEducator.org and the number of editors who have worked on those particular wikis. With over 500 wikis between the letters A to C, it was evident that the site included thousands of wiki pages. The community currently has 19,484 users and holds regular Learning4Content workshops to enable visitors to “familiarize themselves with their new neighborhood” (Mackintosh, 2011). Various wiki pages indicated the multinational reach of the initiative. WikiEducators in Colombia, Argentina, and Africa, among other groups, had created a wiki for individuals interested in developing context specific resources and learning about courses for educators being taught in these locations. Considering knowledge as a common good, editors are forewarned that material published on WikiEducator will be licensed under a Creative Commons (CC) CC-BY license. CC-BY is currently the most open Creative Commons license, permitting the remixing, reproduction and commercialization of the content. The only two types of materials accepted by WikiEducator are CC-BY or CC-BY-SA. By adding the SA or Share-Alike condition to the OER, individuals are required to openly share any derivative of the original OER under the same, or a more restrictive license. Other institutions such as MIT have adopted a CC-BY-NC-SA license, requiring individuals to share any modified products under the same license and prohibiting the commercialization of materials.

Apart from visiting various OER resources and collaborating in their development, I also attended the OCL4Ed course for a week to experience WikiEducator as a course participant. After enrolling online free of charge participants were taught a number of basic OER concepts such as open licensing, creating and using an identi.ca account and the objectives of WikiEducator. Readings by OER advocates and videos on various key issues about licensing and the possibilities of OER were assigned. To aid participants in familiarizing themselves with various creative commons licenses, scenarios were presented and participants were asked to explain if the individual had violated copyright law and how that could have been prevented.

Another important objective of the OCL4Ed course was the development of a peer learning environment. The course began by asking everyone to share with the class their name, their country of origin, and their opinions about OER. By posting these answers on a Moodle site, others were able to comment and inquire about the member’s responses. Additionally, through identi.ca, participants were asked to share their opinions on the class topic in less than 144 characters. Posts that were particularly insightful were then retweeted by Wayne Mackintosh (Pic 6). Hosted in New Zealand but offered to participants worldwide, learning took place asynchronously and participants were able to share their thoughts about the course subject-matter in their own time frame and schedule.

Individuals could visit and learn from these materials any time of the year as all of the resources are accessible through WikiEducator. By encouraging multiple types of feedback OCL4Ed was a very effective and efficient course as individuals were able to obtain a good understanding of CC and the importance of open content, and the course provided for a high level of interaction between participants. While some individuals were more expressive than others in their tweets, there seemed to be a high degree of satisfaction with the course, eliciting comments such as  “this is the best course I have ever taken”[5]. However, as with other online courses, interaction with other class members was entirely textual and no interactive meetings with other class members were held. While in the future interactions in online classes may increasingly include the use of video conferences and provide for a more personal environment, online classes today, due to their asynchronous elements and distance between participants, allow some participants to feel more comfortable in expressing themselves more freely.

Other important elements of my study as an active participant included visiting the WSIS OER website and responding to the debate regarding whether or not OER should allow commercialization. Voting ended with 57% in favor of commercialization to 43% against it. (WSIS, 2011) Further investigation of this issue would be helpful and a future study should include additional observation of relationships in the wider OER community.

Interviews

After attending the meeting, in an attempt to become better acquainted with the community, I used the list of 226 attendees to construct a database of participants’ names, organization, and email address as well as their administrative position, their blog (if available), and their country of origin. Following the development of this list, an email message was drafted and individuals were subsequently contacted for a 30 minute interview. This process[YUN7]  allowed me to meet 13 different WikiEducators and OER supporters.

A first email was sent only to the 23 individuals who registered to physically attend the OERU meeting. This email included not only the purpose of my study but also a brief disclaimer about my personal views and value premises (Annex 1). Receiving only one hesitant response from 23 emails the letter was shortened, removing my value premises. While the lack of responses to the first email was discouraging, it was not unexpected as many of these individuals hold high level administrative positions and are also some of the most prominent members of the community. The second letter (Annex 2), sent to 226 people, resulted in a higher response rate with 13 individuals being interviewed through Skype.  Participants were able to sign up for any date in March, April and May where there were times available (Pic 8).

Surprisingly, several of the individuals who responded were administrators or directors of different OER programs. The rest of the individuals who were interviewed were OER producers and university professors at their particular institutions. Respondents included Cable Green (Director of Global Learning – Creative Commons), Abel Caine (UNESCO – OER Project Administrator), Joyce McKight (SUNY Empire State College, April 2011 WikiEducator of the Month), Richard Heller (Director of the People’s University), Steve Forester (LCO Ojibwe Community College, Consultant), Pheo Martin (Director of the Realizing Education’s Potential Institute (REPI)), Joan Garfield* (College Open Textbook Communities)[6], Simon Yalams (Professor at the University of Technology, Jamaica), Anna Gruszczynska (C-SAP – University of Birmingham OER initiative), Sean Linton (PhD Student – Otago Polytechnic), Jane Park (Creative Commons Education Coordinator), Dennis Taylor (HIRAM College – Co-Director STEM Program), Benjamin Steward (Autonomous University Aguascalientes) and David Porter (Executive Director – Bccampus.ca, British Columbia). Having scheduled interviews through an email, without prior personal contacts, the project obtained a respectable 5% response rate. While five percent of individuals responded to my request for interviews, another five percent communicated with me to express an interest in scheduling an interview at a later date or contributing to the research project, and 90% of those emailed did not reply or participate. As individuals are bombarded by emails, 10% represents an acceptable response rate yet, for an open community, their cooperation with the study was not unusually high.

Some of these interviews were subsequently transcribed and emailed to the participants for further clarification. It is the project’s objective to develop a reciprocal relationship with participants and, once they are finalized, I will email all of the transcripts to the participants. To prevent the unexpected loss of data, interviews were recorded through three electronic[YUN8]  devices simultaneously, a digital recorder (Olympus DS-61), a desktop screen / video recorder (Camtasia), and a virtual video recorder (Audacity) (Pic 9). On more than one occasion one of the recording devices experienced a minor malfunction. Having three recording devices increased reliability. The interviewees were asked whether they wanted their interviews to be private or public and under what license they could be shared. With one exception, all of the participants interviewed were comfortable with the open publication of the interview and the use of an open license without first reading the transcript.

After transcribing the interviews and obtaining further feedback or clarification from the interviewees the interviews were coded according to themes. Some of these themes are discussed in the following section. As mentioned by one of the interviewees, despite a degree of philosophical homogeneity among participants, members who were interviewed had differences of opinion regarding the goal of OER, the future outlook of OER, whether OER is a social and/or international movement, their sustainability, and ways in which the community could conduct further outreach and raise awareness about OER.

The interview questions were analyzed by two graduate students  and a faculty member from Curriculum and Instruction after the original draft and modified to increase their effectiveness as well as to single out questions that would be of greater interest to the participants. A list of questions can be read in the annex section (Annex 3). Apart from recording the interviews and analyzing differences of opinion and similarities through the interviews, the study also hoped to find out how open the community is to outsiders, what  it takes to be accepted as a member of the community and whether members would be willing to support a researcher from a closed institution? During my study, I was not questioned as to why I was studying OER and WikiEducator, rather respondents were sympathetic and keen to help a PhD student interested in their subject area. They encouraged me to play a stronger role in the community in the future.

Preliminary Findings (Pilot Mini Ethnography – Spring 2011), Openness, OER, and WikiEducators:

Defining the term “open resource” and the degree of openness of OER and WikiEducator are questions of heated debate within the OER community. Various individuals interviewed were unsure which open license they regularly used for their materials. In addition, the licenses recommended by major OER producers varied. While WikiEducator recommends the use of CC-BY licensing which does not require remixed materials to be shared or to be limited to non-commercial endeavors, the United Kingdom’s (UK) Joint Information System Committee (JISC) promotes the use of CC-BY-NC-SA for most materials, and the CC-BY-NC-ND license for ‘sensitive’ resources[7].  Despite the support for open licenses among OER advocates, I noticed that several OER supporters have published books that sell for a price similar to that of an average paperback book. The books were not expensive, but it symbolizes the balance that individuals have to make between their ideological beliefs and their financial reality.

An inclusive and diverse movement, OER are not all produced under the same license. WikiEducator, through their OCL4Ed course, has attempted to promote the advantages of CC-BY over other licenses. By providing individuals with the freedom to use the resource as they see fit, OER follow closely Stallman’s quote of Open Source Software (OSS) being more like free speech than free beer (GNU, 2010). With the goal of increasing society’s creative output and the amount of knowledge accessible to all, Mackintosh hopes that others will also decide to share their remixed or original OER under the CC-BY license yet individuals are free to choose the license they are most comfortable in supporting. For WikiEducators, OER should contain four essential freedoms: the freedom to use a resource, to adapt it to your needs, to help your neighbor (by sharing copies), and to help your community. Focused on the practical and improving the living conditions of not just the user but the community, WikiEducator hosts courses and hopes to reduce barriers for people who cannot attend Higher Education courses or obtain credentials.

Sustainability

A major concern of my informants was the need for greater government support. According to Joyce McKnight (SUNY), without the support of the government, various OER programs will likely experience financial difficulties. However, Cable Green (CC) explained how the disappearance of a particular OER, due to lack of maintenance or a server crash is likely to happen at times, but the community will likely either replace the lost material, or have uploaded it to a different site before the server went offline, lost or erased the data. Yet, grants for OER projects are increasingly competitive and it is likely that some programs will lack the financial resources to sustain their OER , as has been the case with many dot com initiatives.

During my interview with Anna Gruszczynska (JISC) she mentioned the likelihood of her program losing financial support at the end of this year. While the project has received funding for two years, the inability to be self-sufficient financially may result in the loss of their expertise. While she believed that OER created by independent producers will continue to be developed without financial support, larger scale initiatives could expand rapidly if only they were given the right type of incentives. While all of the individuals interviewed expressed their desire to continue producing OER and wished to spend more time developing these resources, full time faculty members and many other educators already have a very busy schedule. If administrations were to offer benefits for teachers to publish in open access journals or develop open education resources, their quantity and quality could grow exponentially. Abel Caine (UNESCO) argued that concerns of educators regarding a loss of job security from the growth of OER is a “red herring,” or misconception, as many students would continue to prefer living the college experience, going out and socializing, and living away from home to exclusively using online resources. In addition, there is a large unmet demand of students who are capable but unable to attend a higher education institution (Daniel et al, 2007; Taylor, 2007).

Jane Park (CC) reiterated that recent policy changes, such as the US government recently requiring educational resources produced with a $2 billion grant to be published under a CC-BY license are steps in the right direction that will hopefully continue (DOL, 2011). Similar pressure from national governments and grant makers in the future would provide the OER movement with much needed support and legitimacy, increasing the chances of obtaining a critical mass of supporters, and reaching a tipping point (Rogers, 2003).

Networking and Relationships

From the interviews, it became apparent that many participants communicated with each other on a regular basis. Unlike in larger networks, it seemed that the most active members of the OER community were regularly in touch with each other. New information communication technologies (ICT) have clearly played a role not only in making it possible for almost 20,000 individuals to develop OER through WikiEducator alone, but also to build a sense of community and camaraderie. A recent survey of WikiEducator members showed that 60.5%   had found out about the site from a friend or a referral, 19.3% had found the site through a search engine, 2.7% through a social tagging site, and 20% through other ways (WikiEducator, 2008). It appears that the open nature of the movement, in particular its leadership, has aided its early expansion.

Unlike more traditional academic fields, OER scholars are few and they appear to hold minor or mid-level management positions within their institutions, making it difficult to influence high level administrators. In addition, while it was MIT which first opened its courses to the World Wide Web, and was soon followed by Yale and other top tier US universities, members from these institutions do not seem to be present on a regular basis within any of the OER communities I studied. Having started my study by visiting the OCW Consortium website, it quickly becomes clear that while the universities within the consortium are producing some of the most valuable content within the OER movement, their online forums are not very active. WikiEducator seems to have, and be increasingly developing, this interactive environment yet, as David Porter (BC) stated, the quality of WikiEducator resources has still not reached the desired level. However, just as the internet and wiki technologies proved to be essential to the success of Wikipedia, these technologies can also enable WikiEducators have a broader societal impact. The ability for anyone with access to the internet to use OER from any place in the world, reduces the number of individuals who are needed for the movement to develop a critical mass.

This will be particularly helpful for the success of OER as their supporters are growing but limited in number in comparison to the broader population. Out of 7 billion people, what can be achieved by 20,000 members? Cumulatively, much more that most would expect! However, while some respondents considered OER to be part of a social movement, most of them asserted that OER is not a commonly known term, and has only moderate support. Abel Caine (UNESCO) envisions a steady growth in the use of OER and its support, yet he does not envision its development or use growing exponentially. David Porter emphasized the need to promote the reuse of OER, increase quality, and encourage new members to advocate for OER. In his view, OER communities are led by various charismatic individuals who, while they have been essential for the growth of OER, have also not provided much of a space for new voices to be heard.

The “benevolent dictator” is considered an influential aspect of other Free Culture initiatives such as Wikipedia, Linux, and OSS. While there are OSS initiatives without a charismatic leader, some of the most successful projects do have a key figure who plays an important role in guiding and developing the initiative. Functioning as CEOs these members hold considerable influence and contribute to the overall success of the initiative. This seemed to hold true for WikiEducator, according to my respondents. All of the individuals who were interviewed knew or had spoken to Wayne Mackintosh on some level, and had connections with various other OER members.

Apart from the regular exchange of ideas between different members of the OER community, David Porter mentioned that there is a high level of philosophical homogeneity among supporters. This was seconded by Joan Garfield*, as well as Joyce McKnight (SUNY), and Pheo Martin (REPI). Yet, Pheo Martin and others emphasized that, given the small size of the OER community, this degree of homogeneity is not surprising. However, as the community expands, it will be more likely for subcultures or subsets of the community to develop. As of now, the number of developers and advocates is small and their sense of community has increased the sustainability of the OER movement by constructing a network of knowledge and support. Surprisingly, despite the collective spirit of WikiEducator, several of the individuals interviewed mentioned that resources are currently developed primarily by individual contributors and cooperation is not as common as one would expect.

Conclusions from Interviews and Participant Observation

After conducting 13 interviews and being part of the community for most of the spring 2011 semester, it became apparent that WikiEducators, despite sharing a belief in the importance of developing OER, have different motivations for joining the community. Reasons given ranged from having been hired by an institution supportive of OER to having a desire to make education free for all and developing resources during their spare time. Despite my interest in learning more about the importance of WikiEducator for developing countries, only one participant from a developing country was interviewed. Simon Yalams (Jamaica Technology University) expressed a strong interest in OER, but had not been able to spend as much time developing them as he had hoped. He saw the need for more resources to be developed locally, but was primarily interested in increasing their use and support across Jamaica. With 47 countries represented in the OERU planning meeting it would be beneficial to analyze WikiEducator from a multicultural perspective. In regards to sustainability, it would be interesting to see if they will have financial difficulties in the upcoming years, or whether the open model will continue to sustain itself. The WikiEducator website generates between 8 and 10 million online hits per month. Developing a sustainable financial model will require a substantial amount of donations and grant awards. To meet this challenge, they have developed packages for individuals and organizations to buy a membership and be part of the OER Foundation (OERF). Members did not answer whether they thought WikiEducator itself was sustainable, but rather discussed economic problems in terms of the broader OER movement. In general, WikiEducator is a very open and community oriented environment. They have concerns over quality, but they have continuously increased the number of resources available and participating members, and they are confident about meeting the quality challenge over time. Reaching a critical mass of contributors and supporters is key to increasing adoption and to the overall success of the initiative. Opinions about the future of OER and WikiEducator or whether OER was a social and/or an international movement varied, yet respondents generally agreed that the community will continue to grow over time. A very close community, OER have benefited from inter-institutional cooperation and support. Having members from a long list of countries and places, but being a project of the Commonwealth of Learning, whose countries communicate primarily in English, parallel WikiEducator projects in different languages may not be as successful. Further study will help clarify some of these questions.

Conclusions – Research Framework

While this preliminary study provided me with a broad understanding of WikiEducator and the OER community and the relationships between different institutions supporting OER, asking questions that deal with more specific details would be helpful in producing a more nuanced and in-depth study. Additional interviews would enable me to find out more about certain issues, as well as increase the number of members from other countries who participate in WikiEducator, adding their perspectives to my analysis. Aiming to provide the reader with the thickest description possible, spending 9 months to a year participating in WikiEducator would help me to develop deeper and more nourishing relationships with community members who may then be more open to sharing further details about their experiences. Perhaps I would eventually obtain access to some of the people in key administrative roles within WikiEducator.

Thank you WikiEducators for your help and collaboration in making this idea a reality!

Works Cited

Alexa. (2011, May 07). Wikipedia.org – Wikipedia. Retrieved May 07, 2011, from Alexa.com – The Web Information Company: http://www.alexa.com/siteinfo/wikipedia.org.

Baraniuk, R. (2006, August). Richard Baraniuk: Open Source Learning. Retrieved November 3, 2010, from TED Ideas Worth Spreading: http://www.ted.com/talks/richard_baraniuk_on_open_source_learning.html

Benkler, Y. (2009). The wealth of networks: How social production transforms markets and freedom. New Haven: Yale University Press.

Bilir, T. E. (2009). Real economies in virtual worlds: A massively multiplayer online game case study, Runescape. Atlanta: Georgia Institute of Technology.

Boellstorff, T. (2008). Coming of Age in Second Life: An Anthropologist Explores the Virtually Human. Princeton: Princeton University Press.

Correll, S. (1995). The Ethnography of an Electronic Bar. Journal of Contemporary Ethnography , 270-298.

Daniel, J., Kanwar, A., & Uvalić-Trumbić, S. (2007, 6 15). Mass Tertiary Education in the Developing World: Distant Prospect or Distinct Possibility? Retrieved February 20, 2011, from Commonwealth of Learning: http://www.col.org/resources/speeches/2007presentations/Pages/2007-massTertiaryEd.aspx

DOL. (2011, January 20). US Labor Department encourages applications for Trade Adjustment Assistance Community College . Retrieved January 23, 2011, from United States Department of Labor: http://www.dol.gov/opa/media/press/eta/eta20101436.htm

Donner, J. (2008). Research Approaches to Mobile Use in the Developing World: A Review of the Literature. The Information Society , 140–159.

Downes, S. (2007). Models for Sustainable Open Educational Resources. Interdisciplinary Journal of Knowledge and Learning Objects , 29-44.

Elliot, A. (2009). Contemporary Social Theory. New York: Routledge.

Fetterman, D. M. (1989). Ethnography: Step-by-Step. London: SAGE.

Foucault, M. (1984). The Foucault Reader. New York: Pantheon .

Friesen, N., & Hopkins, J. (2008). Wikiversity: or education meets free culture movement: An ethnographic investigation. First Monday .

Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York: Palgrave Macmillan.

Geertz, C. (1973). The interpretation of cultures. New York: Basic Books.

GNU. (2010, November 12). The Free Software Definition. Retrieved April 25, 2011, from GNU Operating System: http://www.gnu.org/philosophy/free-sw.html

Hanisch, C. (1970). The Personal is Political. Notes From the Second Year: Women’s Liberation: Major Writings of the Radical Feminists .

Harry, D. (2005). Bridging the Gap: How Offline Relationships Affect Online Sociality in World of Warcraft. Needham: Olin College.

Heeks, R. (2008). ICT4D 2.0: The Next Phase of Applying ICT for International Development. Computer , 26-33.

Hine, C. M. (2000). Virtual Ethnography. London: SAGE.

Kafker, F. A., & Loveland, J. (2009). The Early Britannica: The Growth of an Outstanding Encyclopedia. Oxford: Voltaire Foundation.

Kamenetz, A. (2010). DIY U: Edupunks, Edupreneurs, and the Coming Transformation of Higher Education. White River Junction: Chelsea Green Publishing.

Katz, R. N. (2008). The Tower and The Cloud: Higher Education in the Age of Cloud Computing. Boulder: EDUCAUSE.

Kouritzin, S. (2002). The “Half-Baked” Concept of “Raw” Data in Ethnographic Observation. Canadian Journal of Education , 119-138.

Kramer, F. D., Starr, S. H., & Wentz, L. (2009). Cyberpower and National Security. Washington D.C.: Potomac Books Inc.

Kurzweil, R. (2006). The Singularity Is Near: When Humans Transcend Biology. New York: Penguin.

Lessig, L. (2005). Free Culture – How Big Media Uses Technology and the Law to Lock Down Culture and Control. New York: Penguin Press HC.

Levine, R. (2006, September 6). New Web Sites Seeking Profit in Wiki Model. New York Times .

Mackintosh, W. (2011, May 5). EL4C50 – WikiEducator Learning4Content Workshop . Retrieved May 5, 2011, from WikiEducator: http://otago.wikieducator.org/Learning4Content/Workshops/eL4C50/Workshop_schedule#Getting_started_.28Pre-workshop.29

Murley, D. (2008). What Second Life Taught Me about Learning. Law Library Journal , 787-792.

Myrdal, G. (1972). How Scientific are the Social Sciences? Journal of Social Issues , 151-170.

Nardi, B. (2010). My Life as a Night Elf Priest: An Anthropological Account of World of Warcraft (Technologies of the Imagination: New Media in Everyday Life). Ann Arbor: University of Michigan Press.

Naughton, B. (1993). Deng Xiaoping: the economist. The China Quarterly , 491-514.

OECD. (2007). Giving Knowledge for Free: The Emergence of Open Education Resources. Paris: Organization for Economic Cooperation and Development.

Palloff, R. M. (2009). Building Online Learning Communities: Effective Strategies for the Virtual Classroom. Hoboken: Jossey-Bass.

Prensky, M. (2010). Teaching Digital Natives: Partnering for Real Learning. Thousand Oaks: Corwin.

Prensky, M. (2010). Why You Tube Matters? The Horizon , 124-131.

Reagle Jr., J. M. (2010). Good Faith Collaboration: The Culture of Wikipedia. Cambridge: The MIT Press.

Rogers, E. M. (2003). Diffusion of Innovations. New York: Free Press.

Rowlands, I., Nicholas, D., Williams, P., Huntington, P., Fieldhouse, M., Gunter, B., et al. (2008). The Google Generation: The Information Behavior of the Researcher of the Future. Emerald Insight , 290-310.

Sachs, J. D. (2005). The End of Poverty: Economic Possibilities for Our Time. New York: Penguin Press.

Schaller, R. R. (1997). Moore’s law: past, present and future. Spectrum – IEEE , 52-59.

Somekh, B., & Lewin, C. (2006). Research Methods in the Social Sciences. London: SAGE.

Tapscott, D. (2008). Grown Up Digital: How the Net Generation is Changing Your World. New York: McGraw-Hill.

Taylor, J. C. (2007). Open Courseware Futures: Creating a Parallel Universe. e-JIST , 1-9.

UNESCO. (2011). Shifting Focus to Open Education Practices – Opal Report 2011. Paris: UNESCO.

Vest, C. M. (2006). Open Content and the Emerging Global Meta-University. EDUCAUSE Review , 18-30.

Walsh, T. (2010). Unlocking the Gates: How and Why Leading Universities Are Opening Up Access to Their Courses. Princeton: Princeton University Press.

WikiEducator. (2008). WikiEducator Survey for New Accounts. Internet: WikiEducator.

WikiEducator. (2010, December 24). WikiEducator:M and E Overview. Retrieved April 20, 2011, from WikiEducator: http://wikieducator.org/WikiEducator:M_and_E_Overview

Wiley, D. (2007). On The Sustainability of Open Education Resource Initiatives in Higher Education. Paris: OECD.

Wittel, A. (2000). Ethnography on the move: From field to net to Internet. Forum: Qualitative Social Research , 23 Paragraphs.

Wolcott, H. F. (2008). Ethnography: A Way of Seeing. AltaMira Press.

WSIS. (2011, May 6). Should OER favour commercial use? Retrieved May 07, 2011, from WSIS Platform of Communities: http://www.wsis-community.org/pg/debates/group:14358/phase/251476/252291

Yee, N. (2006). The Demographics, Motivations and Derived Experiences of Users of Massively Multi-User Online Graphical Environments. PRESENCE: Teleoperators and Virtual Environments , 309-329.

Annex #1 – Original Letter Sent to OER and WikiEducator Supporters

To: Mark Brown

Dear OER and Wikieducator.org supporter,

My name is Alfonso Sintjago, I am a PhD student from Venezuela studying at the University of Minnesota and I am currently conducting an ethnographic study of the OERU – Wikieducator.org initiative (http://wikieducator.org/User:Sintjago). Apart from engaging in the community as a participant observer and analyzing forum data through discourse analysis, I am hoping to conduct interviews regarding OERs and investigate how individuals become involved in this initiative, their aspirations, their outlooks, and other personal opinions.

I hope to record video conferences with users, OER advocates, developers, and other community members and share the interviews through the use of a CC-BY license. These interviews may be podcasted, vodcasted, and placed on a YouTube channel for increased viewing. The interviews could also be private and anonymous if this is in the interest of the participant. I appreciate your time and collaboration. Please contact me if you are interested in scheduling an interview. My schedule is currently flexible and I am available most days.

To increase transparency, I wanted to clarify my personal beliefs and possible biases. I am currently promoting the increased use of OER in Latin America and development of them from educators in the United States in institutions such as the University of Minnesota, as well as through academic communities whose work focuses on development issues or public studies. I am also interested in promoting the development of OER by academics who have left developing countries, by promoting a brain gain or the production of “knowledge remittances”. However, I am interested in learning about your opinion and experience, and listening to the multiple influences, perspectives and voices about OER, its local and global implications.

These conversations have the flexibility to be private or open to accommodate different opinions and desires. Participants will have the opportunity at the beginning and the end of the study to express their desire for the information to be considered private, confidential and anonymous. I truly hope you are able to participate. While I plan on doing my personal remix with materials captured through the conversations, by being CC-BY, this material could later be used in a number of additional ways. Willing your participation, this material could also later be expanded through focus groups organized according to type of interaction (student, developer), region, or mixed groups to further discuss some shared topics, goals, and experiences.

I hope you are able to share with me sometime in the upcoming weeks your motivation, your experience and your thoughts as an OER developer.

Thank you for your time,

Alfonso Sintjago

352-359-4026

sintj002@umn.edu

Do not reply to this email. To respond to this invitation, click the button above or the link below.
http://my.timedriver.com/af3248798sd98u89uu98uu98agsdf  – Powered by TimeTrade

To: Dennis Taylor

Annex #2 – Modified Letter Sent to OER and WikiEducator Supporters

Dear OER and Wikieducator.org supporter,

My name is Alfonso Sintjago, I am a PhD student from Venezuela studying at the University of Minnesota, USA and I am currently conducting an ethnographic study of the OERU – Wikieducator.org initiative (http://wikieducator.org/User:Sintjago). Apart from engaging in the community as a participant observer and analyzing forum data through discourse analysis, I am hoping to conduct interviews regarding OERs and investigate how individuals become involved in this initiative, the broader OER movement, their aspirations, their outlooks, and other personal opinions.

I hope to either take notes during the meeting, or record the audio or video from the interview, depending on the wishes of the participant. The interview can be private and anonymous if this is in the interest of the participant. If the participant wishes, the recorded audio, video or written notes from the interview can be posted online and shared through the use of a CC-BY license.
I appreciate your time and collaboration. Please contact me if you are interested in scheduling an interview. My schedule is currently flexible and I am available most days.

I hope you are able to share with me sometime in the upcoming weeks your motivation, your experience and your thoughts as an OER developer.

Thank you for your time,

Alfonso Sintjago

352-359-4026

sintj002@umn.edu

skype: fastfonz

Do not reply to this email. To respond to this invitation, click the button above or the link below.
http://my.timedriver.com/af3248798sd98u89uu98uu98agsdf

Powered by TimeTrade

Annex #3 – Original List of Questions for Semi-Structured Interviews

OER Ethnography Study – Questions (1^st Draft)

Thank you for deciding to contribute to this study. Below are some of the questions that we may discuss during the interview. The interview will be semi-structured and I am also interested in learning from you what you consider relevant. Feel free to expand on any point or make suggestions about other areas to explore. These questions are only an outline of potential topics. The questions will be refined and modified after a predetermined number of interviews to better address relevant issues. Some of the questions will not be addressed due to time constrains and availability.

Thank you for your collaboration,

Alfonso Sintjago

How and when did you first become involved with OER? How has your involvement with OER changed over time? What attracts you the most about OER?

If the answer is different from the previous question, how and when did you first become involved with Wikieducator.org and the OERU initiative?

How does OER influence knowledge and technology transfers? locally and globally what, in your opinion, is the goal of the OER movement? Is the OER a social movement?

Do you consider Wikieducator.org and OERU an international movement or an international initiative? Why or why not?

What are some of the greatest obstacles facing OER? Should every country participate in the development of OER?

Should OER be produced in least spoken languages as well as major languages?

Would you say they are currently being developed in a large number of languages, which languages are most prevalent? How important is contextualization in the production of OER?

What license should OER be produced under (if Creative Commons, what type of Creative Commons)? What license do you publish most of your work under?

Are you currently employed? Does your employer support the development of OER?

How important is strong support from the university administration for the development of OER?

How important is grass roots support of local communities for the development of OER? Is Wikieducator.org sustainable? Which other OER supporting websites are, in your opinion, sustainable?

What are some obstacles that limit you from spending a greater amount of time developing OER?

What are some of the greatest obstacles the OERU and the WikiEducator initiative face? What are some of the initiative greatest strengths and greatest weaknesses?

What are some limitations of OER? Wikieducator.org and the OERU initiative?

Are most people aware of what OER means or are they passive users of OER?

What leads a person to become more active in the community? How does the community strengthen itself?

Is Wikieducator.org doing enough in terms of outreach? Is the OER community doing enough in terms of outreach?

How are the wikieducator.org and OERU community different from the OER community at large?

Should the term OER be a household or commonly known term?  Have you taught your children about OER? Are your family and friends supportive of OER?

Are most of your friends aware of what OER are and are they supportive of the idea?

Have you met people within Wikieducator.org? Have these relationships extended between the online community? What has been the nature of these relationships?

To what extent do you consider OERU and Wikieducator.org to be an online community? What has limited it from growing?

If you do not mind me asking, how do you identify yourself; political conservative, liberal, independent?

How do you envision the OER movement to be in 2 years, 5 years, 10 years? What role with the OERU and Wikieducator.org play in the future of OER? What role does OER play within ICT4D?

How many hours do you devote a week to the development of OER? How frequently do you visit Wikieducator.org or affiliated websites? Which other websites do you visit?

Which OER sites do you frequently visit?  How did the OERU idea develop? What has been your level of involvement with the OERU initiative?

What do you think is the greatest barrier for other universities when considering whether or not to join the OER movement?

What has the OER community done to increase awareness of OER? What other steps need to be taken?

What to you is the ultimate goal of OER? Are OERs sustainable? Why or why not?

Please explain your work within OER and what led you to choose this approach when developing education resources? What is your average day like? Have you recently participated in OER courses?

If there was one variable missing from the OER that is most hindering its growth and expansion, what would that be? What is the greatest piece missing in the OER movement?

Annex #4 – Modified List of Questions for Semi-Structured Interviews

Most Frequently Asked Questions (2^nd Draft)

How and when did you first become involved with OER? How has your involvement with OER changed over time? What attracts you the most about OER?

What are some of the greatest obstacles facing OER? Should every country participate in the development of OER?

How important is strong support from the university administration for the development of OER?

What are some obstacles that limit you from spending a greater amount of time developing OER?

What leads a person to become more active in the community? How does the community strengthen itself?

Should the term OER be a household or commonly known term?  Have you taught your children about OER? Are your family and friends supportive of OER?

Are most of your friends aware of what OER are and are they supportive of the idea?

Have you met people within Wikieducator.org? Have these relationships extended between the online community? What has been the nature of these relationships?

If you do not mind me asking, how do you identify yourself; political conservative, liberal, independent?

Which OER sites do you frequently visit?  How did the OERU idea develop? What has been your level of involvement with the OERU initiative?

What do you think is the greatest barrier for other universities when considering whether or not to join the OER movement?

What has the OER community done to increase awareness of OER? What other steps need to be taken?

What to you is the ultimate goal  of OER? Are OERs sustainable? Why or why not?

Please explain your work within OER and what led you to choose this approach when developing education resources? What is your average day like? Have you recently participated in OER courses?

Minnesota Principal Licensure Program – Cost / Benefit Analysis

11/30/2011

Advancing one’s education is correlated to an individual financial lifetime earnings, and increases the number of ways in which an individual can contribute to society. While going back to school is seen by some as a cost, it should more accurately be described as an investment. As this examples shows, were an elementary school teacher having a six of years of work experience and a MA degree to decide to go back to, even if this person were not to obtain the desired job, in this case, that of being a principal, the individual would still increase their earning by now qualifying to a different scale in the Wayzata teacher salary scale. In this example, the teacher in question originally had an MA and 30 additional graduate credit units, if that teacher were to obtain an additional 30 credit units, that individual would then be moves to the MA+60 pay scale. In average, a teacher changing pay scales from MA+30 to MA+60 would benefit from a pay increase of ($40442 – $42116 [before discounting or BD]) $1674 dollars (BD) a year.[1] Since the employee in question was on the 6^th year of work when he or she took a paid sabbatical year, being paid at MA+30 ($48,212 [BD]), and coming back on his or her 8^th year, being paid at MA+60 ($54,239 [BD]), the individual at first value seems to increase their earnings considerably even if he or she did not started working right away as a principal. If the person came back to work and was immediately transition to working as the acting principal, the difference would be even greater, starting at $106,984 (BD) which is more than twice the salary the person would have received if he or she had simply stayed as a school teacher in the MA+30 bracket (pay for year 8^th – $52,565 [BD]). However, when a raise is received is impacted by the discount rate that is used to calculate the depreciation of his or her earnings throughout the years.

Looking more closely at the question, in this study the researcher uses both a 5% discount rate and a 3.5% discount rate to calculate the possible changes in earnings over time. When calculating the discount rate the study uses the present value formula to find the discounted value of a future amount (P= A/(1 + r)ⁿ)^. Both 3.5% and 5% are consider acceptable discount rates by most of the related literature. Taking into account a discount rate, as well as an interest in the individual potential savings, and the cost of obtaining 30 more graduate credits, what are the cost benefits for the elementary school teacher of going back to school? Should the elementary school teacher go back to school and obtain the licensure to become an elementary school principal? What other variables that are usually not analyzed in terms of financial cost should be included as financial costs for this calculation? While a principal generally earns more than a teacher, the teachers also work more days in the year, and have to deal with different types of stress, something that may also have a cost for the individuals, and be one of the variables that the is most influential in their decision. Below are a set of assumptions that were made when developing a response to this problem.

Assumptions:

–        Discount Rates – To answer this question, two different discount rates were used. While other values such as 6%, 3%, 10% may be used by some to calculate the changing value of salaries, this evaluator chose 3.5% and 5% as its discount rates. The reason for choosing these two values is that they are both within the values that are considered acceptable and they provided the researcher with the possibility of seeing if the best possible choice for the teacher changes when the discount rate is modified.

–        Savings and Interest Rates – While saving rates change depending on an individual’s income and currently most savings and stocks are owned by the wealthier members of the United States society, this solution sets the saving rate at 10% per year. This saving then earns a 7% interest per year. While these values are arbitrary, the interest rate of saving was purposely set higher than the income depreciation rate.

–        Credits Per Semester – In this example, to reduce the complications of attending to summer school, the student decided to take 15 credits per semester. To do so, the student took the maximum allowed credits per semester under the basic tuition rate (14) and then paid for the extra hour a semester to then take 15 credits a semester and finish his or her 30 credits after 2 semesters.

–        Salary Steps – Every year of work and even during the sabbatical year when the teacher attended school, the instructor advanced a salary steps. Salary steps are there to rewards seniority, but also to help account for depreciation and the inflation of the cost of living. While during a sabbatical, a Wayzata teacher earns only 50% of their salary, and sabbatical years are generally awarded sabbatical for 1 to 2 years, in this case, while the instructor only went to graduate school for a year, after that year, in term of salary steps the teacher was now on the 8^th, rather than the second half of the 7^th salary step.

–        Health Care – Health care choices are regularly based on complex decision that take into account that needs of that individual as well as other direct family members (children, spouse). Because of the different health care options available to both a state employee and a graduate student, such as signing up for one of the student health care coverage or continuing to pay the Wayzata district teacher health care policy, for the purposes of this paper, it is assumed that this represent only a minor loss for the teacher, who would obtain less benefits from their employer during their sabbatical year and will have to pay an extra $800 that year for health care coverage.

–        No Stress Related Changes (Illness) – In this study the researcher assumes that as the teacher transitions towards working as a principal, that there will not be any additional work-related health risks, and the level of stress will stay constant, despite there being possible real world changes. Since this individual had seen becoming a principal as a possibility since early in their career, he or she was as mentally prepared for this job as he or she had been as an elementary school teacher. The teacher perhaps felt even less stress as an administrator than he or she had previously as a teacher.

–        From Teacher to Principal –While it is likely that in real life, the teacher may not be able to find a job as a principal right after graduating, and that he or she may instead go back to teaching, or working as an assistant principal or school administrator for a few years, it is assumed for the purposes of this report, that the teacher will begin to work as an elementary school principal in Wayzata right after graduating. In addition, the teacher has a 100% probability of obtaining this job after graduating.

–        Housing and Transportation – During the teacher’s sabbatical year, the teacher moved to live with his or her brother and attend the University of Minnesota. While his or her brother allowed the teacher to stay in his home for free, this individual still has to pay the mortgage to her home in Wayzata and decided not to rent out the place. The driving distance from the teacher’s home in Wayzata to the elementary school where he or she works is the same as from the person’s brother’s house to the University of Minnesota. There are no increases in either transportation or housing costs for the future student. The teacher was planning to take an initial trip to Minneapolis whether or not he or she would attend the UMN to visit his or her brother.

–        Demographics – In this example the individual in question is 30 years old, single but planning to marry his or her partner in a couple of years. The teacher does not have any children, and the teacher’s partner is quite busy and travels regularly for work. The teacher’s partner does not feel that the teacher’s return to graduate school will detrimentally affect their relationship. As such, there are limited relationship costs and these costs are not considered in this example.

–        Year Zero – There are a few places were year zero could have been set and then used as the basis to calculate the discount rates, and interest rates. If the teacher had been planning for the whole six years that he or she would go back to school, then the cost of school would have to be discounted from year zero (or six years before enrolling in school). Another possible year zero could be after the teacher has worked for six years. In this scenario, the costs of schooling are not discounted by the first six years. However, for this study, the researcher argues that the teacher had been planning for six year in returning to and going back to school, as such year zero is consider to be the first year the teacher worked as an elementary school teacher in Wayzata.

–        Cost to Society and Wayzata – to reduce the external impacts of the teacher’s decision, the study assumes that had this teacher not been admitted to the university, that a different student would have taken this student’s place as there are generally more applicants wanting to attend the university than the University of Minnesota can provide access to. Because of the low elasticity of demand, there is not a cost to society, but rather a transfer of cost from one individual to another. In this study, the researcher makes a similar argument for the Wayzata school district. The district knew of an upcoming vacancy for some years as a principal was planning to retire. Another instructor in this school would have been given this opportunity, if this teacher did not receive training to fill the newly established vacancy. As such, because both events, a new student and a new principal would have taken place regardless of whether this teacher filled that position, this analysis focuses on the decision making and choices of the individual instructor, considering the impact of further educating him or herself to primarily impact their personal condition rather than society in general.

–        Leisure Time – When calculating the cost of increasing one’s workload, it is difficult to know whether the time that the person was not originally working in had any value, because of this, while there are 7 additional weeks that an elementary school principal has to work in comparison to an elementary school teacher, it is hard to know to what extent is this time valuable. Since it is difficult to measure the contribution of leisure time and its opportunity cost, the researcher decided to exclude the value of leisure time from the calculations by arguing that were it not for these events the teacher would have lost a lot of his or her money gambling instead of being productive. While leisure time costs were not included in the calculation, it can be a very important variable. For more information about the financial impact of leisure time visit TABLE 9.

–        Interactive Spreadsheet – Modifying some of these assumptions would substantially impact the results. The attached spreadsheet is interactive; therefore if the reader is interested in finding out the impact of other interest rates, saving rates, and discount rates, they can modify the assumptions as needed to the attached spreadsheet.

Analysis and Results

To calculate the possible incomes, this research used the income tables that can be found at the Wayzata school board website (http://www.wayzata.k12.mn.us/Page/6925), basing the salary differences between the choices available to the teacher on the salaries for principals (from the principals’ contract) and the teacher salary differences from the resources provided by the professor who commissioned this study as well as the (WEA teachers’ contract) also available through the Wayzata school board website. Once a teacher joins he enters into a contract with the school board that details the salary steps and possible income that the teacher could generate were he or she to stay in this job for the remainder of their career. The attachment provided by the professor who commissioned this study provided a salary schedule from 2006-2007. This salary schedule was used to calculate the starting salary of the teacher in question. Since the teacher will have six years of teaching experience by the summer of 2012, it can be estimated that this instructor began to work in Wayzata on the school year 2006-2007. Therefore this scale is appropriate for the teacher in question. Since the teacher will graduate from the University of Minnesota after the 2012-2013 year, for this study the researcher uses the current principal salary as a proxy, but it is likely that the upcoming contract will include a minor modification to the salary base to account for inflation. To calculate the possible salary if the teacher does not obtain a principal position, in this study the researcher analyzed different possibilities, finding the differences between the yearly wage difference between the current MA+30 contract and MA+60 contract and added that difference to the 2006-2007 contract provided by the professor who commission the study, to find out what salary would be obtained if the teacher did not obtain a principal’s position. The tables were the average difference between MA+30 and MA+60 salaries was calculated is TABLE 1. The first seven years of an MA+60 salary were calculated, but they are not included in TABLE 4, and TABLE 5 because regardless of the outcome, the salary is the same for the instructor during the first 7 years (including the year of study) since the instructor has yet to improve his education credentials.

TABLE 2 provides the reader with an explanation of what cost were included when the student decided to stop working in Wayzata for a year and attend a licensure program at the University of Minnesota. As explained within the assumptions, healthcare was set at an increase of $800 a year, there were no changes to the transportation costs, and according to the Wayzata teacher terms of contracts, teachers are allowed to ask for a professionally related sabbatical to improve their education as long as other teachers are staying at the school and have not also requested taking an educational sabbatical. Since the school had planned to allow one of its current instructors to become a principal, the societal cost and district cost of training this principal are not the result of the teacher’s decision but would have happened regardless; therefore this study focuses on the personal cost of a teacher in making an investment and continuing their education. The costs of tuition were based on the current tuition cost for the University of Minnesota graduate school (http://onestop.umn.edu/pdf/tuition_grad_2011-12.pdf).

Since the teacher will be taking 14 credits each semester, the teacher decided to pay for an additional credit and take 15 each semester to avoid going to summer school and finishing the program as quickly as possible. TABLE 3 includes very similar information to TABLE 2 with the difference that the values are discounted by 3.5% (TABLE 3) instead of 5% (TABLE 2). Since the person is attending the university in what would be their 7^th year of work and the first year the salary was not discounted, the discount rate was 1.05^6 and 1.035^6 respectively. The costs of attending the university for a year are very different if the values are discounted by *1.34 or *1.23. Taking into account that the teacher will be taking 15 semester credits as well as working volunteering for 320 hours which at the cost of $17.5 an hour is equivalent to $5600 (BD) a year, the cost of going to school for a year $56,960.33 (BD) or $42,507.71 (5% Disc) or  $46,309.21 (3.5% Disc). It is expected for the values discounted by 3.5% to be larger than the values discounted by 5% since the values discounted by 5% lose their relative value quicker. An important assumption made by the researcher in this study which is included within TABLE 2 and TABLE 3 is that the person did not lose income despite now working 47 instead of 40 weeks since the person used to spend their additional time gambling and losing a value comparable to what he or she would have earned otherwise (additional details in TABLE 9). The total number of weeks a principal works and an elementary school teacher works are based on current Wayzata contract details. During the year the teacher goes back to school, he or she loses half of their income; this was included within the total costs of obtaining the degree.

The next tables that were generated were TABLE 4 and TABLE 5, these two tables account for the devaluation of salaries (5% or 3.5%) and the different possible salaries (MA+30, MA+60, Principal No-PhD, Principal with a PhD). There is a possibility that the teacher does not obtain the principal position and earn only the salary for a teacher with an MA+60 credits, or perhaps the teacher when completing those 60 credits was able to finish a PhD after transferring credits from the MA and having some courses exempted. The other important variables that are calculated in this table are the annual savings and the interest rate of these savings. While the first year the 10% yearly savings did not gain any interest since the money was added to the saving account at the end of the year, after the first year, the money the teacher added to the savings and the money that was already there gains a 7% interest rate per year (*1.07^n). The money added at the end of the year, does not gain an interest that year. The 7% interest rate was set to be higher than both discount rates (3.5% and 5%) [dividing by 1.03^n and 1.05^n respectively) yet this may not necessarily be the case.

TABLE 6 includes what the savings and earnings would have been without a discount rate. This table is only added for reference purposes, since it is important to discount all values on a yearly basis. TABLE 4 and 5 include a very detailed illustration of the different earnings per year. At the end of the table values for a 35 year career are calculated. All of these values are then taken for the calculation of the net benefit analysis (total benefit – total cost), and the benefit-cost analysis (total benefit/total cost). Earnings (without taking into account the cost) are also compared using the option of no going back to school as having a value of “1”. According to this comparison, both taking into account savings and not taking into account savings, the earnings comparisons result in (for 5% discount) – MA+30 1 (BD) or 1 (5%) or 1 (3.5%), MA+60 as 1.01 (BD) or 1 (5%) and 1 (3.5%), principal no PhD as 1.76 (BD) or 1.66 (5%) or 1.69 (3.5%), and principal with a PhD as 1.76 (BD) or 1.66 (5%), or 1.70 (3.5%). This is to say that someone who became a principal will earn 1.76 times what someone that did not go back to school over the course of a 35 year career.

However, when taking into account the cost of going back to school, following the assumptions that are made by the researcher in this study, it can be concluded that unless the teacher obtain a job as a principal, it would not be worth it to go back to school. The benefit-cost ratios indicate this situation. When calculating the benefit-cost of programs, the researcher divided the benefits by the cost. These results can be found in TABLE 7 and TABLE 8. As indicated in these tables, this resulted in the following values: MA+60 – 0.38 (BD) or .00 (5%) or .08 (3.5%), principal with no PhD – 28.84 (BD), 15.24 (5%), 18.14 (3.5%), principal with PhD – 29.08 (BD) or 15.37 (5%) or 18.51 (3.5%). These calculations show that for every dollar invested on obtaining the licensure and the 30 graduate credits needed, if the person does not obtain a job as a principal, then for every dollar invested in obtaining the licensure the person obtained a return of less than half a dollar (.38) or .00 (5%) or .08 (3.5%) per dollar when discounting. However, if the person obtained a job as a principal, whether or not he or she also obtained a PhD, the teacher obtained either $28.84 (BD) to $29.08 (BD) per dollar invested or $15.24 (5%) or $15.37 (5%) or $18.14 (3.5%) or 18.51 (3.5%). The return on the investment seems to be marginally bigger if the person obtains a PhD or not, and the differences between not becoming a principal become larger as the discount rate decreases (from 5% to 3.5%).

Conclusion

As indicated in these results since the researched assumed that the instructor would obtain a job as a principal when he or she returns to Wayzata, it can be concluded that it would be beneficial for the teacher to make the investment to obtain a University of Minnesota principal licensure. While not calculated here, the person may also save money as he or she may would stop gambling. Since the cost of the additional seven weeks of work the person would have to work were not calculated because of this personal problem of this individual, a sensitivity analysis would benefit from calculating how the additional weeks of work negatively impact the gains of going back to school. Due to the subjective nature of how much these weeks affect their additional opportunities; the researcher chose to not take them into account. To illustrate how these values could have impacted the final benefit-cost values please refer to TABLE 9. While it was not calculated for this project, it is likely that the opportunity cost will substantially impact the benefit-cost ratio of becoming a principal. Once again, however, this is only relevant if the person working as a principal does feel that he or she has less time for other endeavors rather than wasting this additional time procrastinating, or pursuing hobbies that may be detrimental to the teacher’s health. It is difficult to quantify leisure time as an opportunity cost because of the disparity between what individuals do with their leisure time.

Sensitivity Analysis

Within the sensitivity analysis, it is important to also consider the impact of savings and interest gain on a person’s savings and investments. To illustrate how saving an interest rates affect this ratios, additional calculations were made using a 10% a year savings rate and a 7% yearly interest rate. This calculations can be found in tables 4,5, 6, and 7. In TABLE 4 and 5 the reader can observe how every year, 10 percent of the income generated was saved and it would later provide a return of 7%. The total values after a 35 work career for savings vary greatly by income. These are the amounts that could have been saved and were earned as a result of a 7% interest rate over the years: MA+30 – $769,868.61 (BD), or $462,405.84 (5%) or $528,377.07 (3.5%), MA+60 – $766,626.32 (BD), or $456,373.00 (5%), or $522,652.96 (3.5%), Principal with no PhD – $1,237,819.13 (BD), or $682,122.49 (5%) or $799,985.71 (3.5%), Principal with PhD – $1,241,848.21 (BD), or $684,050.19 (5%), or $802,353.74 (3.5%).  These values, depending on the scenario can account for close to a third of the benefits obtained by the teacher. When analyzing TABLE 7 and TABLE 8 the reader can find the net benefits, and the benefit cost ratio, and the earning comparison of all 4 groups that were compared in this study.

To anyone using this study as the foundation for their decision as to whether or not to take the courses to become a principal, they must take into full considerations the assumptions made by the researching in this study and consider which ones should be modified. Assumptions vary depending on the time and place, therefore by editing the attached spreadsheet, a local community or a region can independently decide what is most important to them and modify the variables or their weights accordingly.

Bibliography

Levin, H. M., & McEwan, P. J. (2001). Cost-Effectiveness Analysis: Methods and Applications. Thousand Oaks: Sage Publications.

University of Minnesota. (2011, September 20). CEHD – Administrative Licensure. Retrieved November 5, 2011, from University of Minnesota : http://www.cehd.umn.edu/olpd/grad-programs/Adm-Licensure/default.html

University of Minnesota. (2011, August 15). Tuition Rates. Retrieved October 30, 2011, from One Stop Student Services: http://onestop.umn.edu/finances/costs_and_tuition/tuition_and_fees/index.html

Wayzata Public Schools. (2011, September 12). Employee Contracts. Retrieved November 15, 2011, from Wayzata Public Schools: http://www.wayzata.k12.mn.us/Page/6925

Sophia.org – A Collaborative and Accessible Online Learning Community

Alfonso Sintjago – Intro to Evaluation – Project B: Evaluation Context

Oct 21/2011

Object Summary

            Sophia.org provides students with an online social learning environment, where students and instructors can develop relationships with one another, and contribute to a robust social network capable of rapid expansion and fueling innovation. The program, located physically in Minneapolis, is accessible from anywhere in the world where there is free access to the internet. Their community is everywhere. However, having primarily English resources, most of Sophia’s audience is in the United States, and more specifically, in the Midwest. Sophia has various programs including free access to community developed educational videos and resources that can be rated by anyone who is a part of the community, a learning management system which integrates social media, the use of groups, and forums to further explore concepts and ideas, and Sophia Pathways, a program with over 20,000 educational videos where an institution can purchase a computer enhanced math rapid assessment program. Sophia hopes to become a key provider of digital educational content, as well as a major player in the rapidly expanding market of computer enhanced rapid assessment programs.

Procedures Used to Gather Information

            The majority of the information was gathered by visiting the Sophia.org website. Sophia.org regularly updates an online blog and has produced various web based interviews where their founder and CEO Don Smithmier explains the different functionalities and capabilities of Sophia. In addition, other resources were gathered during a site visit, where I met with the Vice President of Academic Outcomes, Dr. Angiers, and further discussed the functionalities, mission, and future of Sophia.

Purpose of the Evaluation

            This evaluation primarily focuses on determining to what extent the program goals and objectives are being followed. While the program is currently in its beta stages, the website is already available to the public. As an online website attempting to construct a community, Sophia’s staff believes that through the use of site analytics as well as written or spoken feedback that they will be able to increasingly meet the demands of their users. In this way, Sophia will likely continue to be a developing model. Feedback is received any day during any time of the day. Since the website became accessible, there have been no attempts to measure or assess the program’s success in increasing students’ knowledge. This may be partly due to the early stage of the program and also because of the role it plays within informal learning. It is difficult to assess how much a particular website contributes to learning unless an evaluation of content retention is included within the online modules. Since Sophia disseminates information rather than evaluates student retention, it will be difficult to obtain a better idea as to how helpful the program is to students, apart from through data on page visits and feedback forms. Sophia currently has a limited number of resource packets. As a collaborative web platform, this program would benefit from increasing their number of contributors.

By evaluating site analytics data including the number of page views, length of stay in the site, and the level of interaction between participants, an evaluation of Sophia demonstrate the potential of the program and the possibilities for the platform. Evaluating the platform’s effectiveness is useful for furthering the impact of the program through marketing and the establishment of new partnerships. While Sophia.org is freely available to anyone with an internet connection, its platform helps to attract users to a fee-paying service, Sophia Pathways. Sophia Pathways then generates the revenue needed for the sustainability of the program. Through marketing and by offering a unique and aesthetically pleasing experience to users, learners will be more likely to come back. The competition and the market for informal online learning platforms is likely to remain volatile for years to come. Without growth and the constant acquisition of new customers, Sophia will likely be unable to sustain itself over time. The list of failed innovative online educational projects is extensive.

Learning from the CEO’s successful experience with Capella University, and having an understanding of the possibilities for future growth in the market, Sophia is well situated to play an increasing significant role within that emerging industry. Some scholars have quantified trends and projected a work environment where informal education increasingly plays a role. As more and more students participate in online courses, and supplement their education informally through the internet and other resources, the greater the user bandwidth and opportunity for Sophia will become. Sophia’s emphasis on the customization of modules and the possibilities for different instructors to explain the same subject in a limitless number of ways is another reason why they could greatly benefit from expanding. Being able to understand why its user base has expanded and how this expansion rate could be increased are two ways in which an evaluation of Sophia could benefit this innovative startup company.

Evaluation Stakeholders/Audiences and Their Concerns

            Sophia’s CEO, Don Smithmier, would be the primary stakeholder of this evaluation. Smithmier is the founder of Sophia and it was because of his vision that the current Sophia executive board was assembled. After leaving his position as a VP for Capella University, he has devoted his interests and efforts to a number of innovative startup companies. Sophia is one of his most visible projects. The success or failure of Sophia will impact the perception of his management capabilities and innovative vision. The non-profit element of Sophia is not generating substantial revenue and, by remaining free from advertisements, without donations or the rapid growth of Sophia Pathways its sustainability is questionable. Because of this, Don Smithmier has conducted a broad marketing campaign and attended numerous events both online and in person.  Don Smithmier has an understanding and familiarity with the stock market and has an interest in maintaining a positive image of the project to encourage further investment into his company.

Vice President of Academic Outcomes, Dr. Angie Eilers, focuses on the importance of providing both rigor and flexibility though the Sophia platform. Having written a number of articles regarding educational reform and currently advocating Sophia’s capabilities as an instructor and educational tutor, she would stand to lose some credibility as an education reformer if the project fails to improve its audience’s academic performance. Does Sophia help to make its users more prepared to succeed academically? Sophia’s uniqueness as a channel where individuals of different skill levels can find a video to learn from is an example of Sophia’s platform’s flexibility, yet having a relatively small number of videos, it is currently not possible for an individual to use Sophia as their only educational supplement.

Senior Vice President of Business Development, Allison Gate, has been a successful entrepreneur in a number of businesses, including transforming a business from a startup to having an over 600 percent increase in sales and eventually being acquired by GE health care. With extensive experience as a business leader in the private sector, the transition of Sophia’s learning platform into a financially successful company is her primary responsibility and concern. Functioning on a subscription rather than an advertising model, becoming a financially sustainable business is not directly linked to increasing its overall number of visitors. As businesses which offer free services have experienced, despite having an extensive and growing audience, it has been difficult for websites such as YouTube to generate a profit despite the use of advertising. Offering a similar service without the inclusion of ads, increasing the number of visitors may also not result in a financial profit for Sophia.

The Chief Operating Officer, Steve Anastasi, is primarily responsible for the development and improvement of the Sophia platform. He worked on developing the Best Buy e-commerce platform and has over 25 years of experience. His experience led him to develop a successful software company that was then acquired by a larger firm (but he continued to work for it as an IT director). If the Sophia platform fails to appeal to the general public because of the interface capabilities, or browser incompatibility problems, as well as glitches or bugs in the code of the site, Steven Anastasi would be most directly linked with these difficulties. With his ample experience Sophia hopes to develop a platform that is not only stable, but also attractive to internet users of different age groups.

The Vice President for Program Management, George Tattersfield, was a co-founder of Guaranteach, an adaptable learning environment where students are given new, more difficult math assignment at a pace that matches student improvements. The financial sustainability of Sophia partly depends on the appeal of Sophia Pathways which was developed primarily from content previously generated by Guaranteach. This includes over 20,000 educational videos. Apart from students’ possible educational gains from using Sophia’s freely available educational materials, Sophia Pathways’ personalized and adaptable learning program has been purchased by a number of schools and could be used by parents of homeschooled children to increase the quality of their children’s learning experience. Negative assessment of Sophia Pathways reflects negatively on the perception of Sophia as a whole, but also on the Vice President for Program Management, since he was involved in the development of this platform and has been in charge of adaptation to meet the needs of different educational institutions.

The website’s visitors, users, and content producers are also stakeholders. As volunteers or beneficiaries, they could migrate to a different online environment if their needs are not met by Sophia. Producers could share their educational videos through Youtube or a different channel if they felt it would be more beneficial. These stakeholders participate in the project because of its flexibility and accessibility. Sophia allows every learner to become a teacher, and every teacher to become a learner. If an evaluation of Sophia were to highlight the benefits of the platform to its users, particularly their benefits in terms of educational gains, Sophia may increasingly attract more users and instructors, enriching the appeal of the site to other future instructors and students. As websites go “viral”, and suddenly grow at a rapid rate, Sophia.org could quickly become a main destination for internet learners. A positive evaluation could have a major impact in the appeal of Sophia to internet users. An evaluation could also help highlight the ways in which Sophia is not meeting the needs of a particular type of online learner or the needs of instructors, acting as a way in which to have a better understanding of the variables that could be improved in order to be more appealing to some current, and possible  future, users. The success of the free components of Sophia is probably the best way in which Sophia can increasingly become visible and then it will be more likely that its private services will be purchased by schools and learners. Many internet businesses function on the 1% or 2% model, where most of the users are free riders and only 1 or 2% pay for additional features. This model has resulted in a sustainable revenue stream for many businesses. It is possible that a similar percentage increase from non-users or public users to private users may be all that is needed to meet the sustainability of goals of Sophia.

Apart from these stakeholders, society in general and learning technologies programs would benefit greatly from learning from Sophia’s experience and to what extent it has captivated the attention and minds of its users. As a rapidly changing learning technologies environment, an analysis of the success or failure of programs over time can help us to better understand both the future of educational business through the internet, and the learning experience of internet users .If successful, Sophia could rapidly expand or similar sites could be developed. Much of this impact depends on whether Sophia is portrayed positively or negatively in the evaluation.

Primary Intended Users of the Evaluation

Don Smithmier, the CEO and founder of Sophia, would have the greatest opportunity to benefit from this evaluation. As an innovative CEO and having ample experience with DOT COM economics and education, Smithmier is aware of the constant change and improvements needed from internet business in order to remain relevant in a rapidly changing cyberspace. Having to change or modify certain aspects of Sophia which were not as positively evaluated as others would lead to long term improvements to the Sophia platform. Whether the evaluation is dissatisfactory to the main players or satisfactory, as a beta project Sophia can use these findings to further strengthen its program, and through these improvements, capture a larger audience and increase its sustainability. Outside of Sophia, the evaluation would be of interest to academics who are trying to better understand the future of online learning.

Evaluation Questions

The questions below have been written to address the concerns of Sophia’s stakeholders.

1. How quickly is the number of users to the site growing? At what rate are visitors opening an account after visiting? At what rate do they return to the site after starting an account?
2. Once a visitor has an account, how often do they visit other producers’ packages? How many packages do most producers make? Are producers and users asking for information about Sophia Pathways?
3. To what extent are the users of Sophia Pathways satisfied? Are the students’ math scores improving faster through Sophia Pathways than without it, according to national and regional standardized tests? Are students expressing high levels of frustration when using the system?
4. Where are most of the visitors arriving from? Is there a particular advertising system that is working more successfully than others? What types of resources do most of the visitors visit? Which site organizational tool are they more likely to use? Do they stay on a particular page for more than 10, 20, or 30 seconds?
5. Do visitors visit various resources after landing on the site? Do they seem to find one that meets their expectations or do they leave the site soon after browsing and searching for a couple of times? Are some key words more successful than others? Are some resources being regularly searched for but are not currently available?
6. When interviewing package producers or instructors, what are some of the reasons they create educational packages? What is their motivation? Are their expectations being met by Sophia?
7. What are the costs of Sophia’s nonprofit program in comparison to the cost of Sophia Pathways? Which one has a higher user satisfaction rate? What percentage of Sophia Pathways partners are renewing their contracts?
8. Are students and producers experiencing technical problems when using the site? This includes bad links, browser error messages, and difficulty using the site with a particular browser.

Proposed Role of the Evaluator

Sophia will be evaluated by an external evaluator. As an external evaluator, Alfonso Sintjago has extensive expertise in evaluating online learning sites. We are possibly currently experiencing a second DOT COM bubble. With this in mind, the evaluator will carefully analyze what Sophia offers in comparison to other competing programs. The evaluator does not have a stake in the outcome of the evaluation as he does not currently work for a competing company and is primarily interested in learning about the differences between these programs. He currently works for the University of Minnesota as an Instructional Technology Fellow and has a teaching background in Learning Management Systems (LMS). He is well versed in using various systems of analysis and categorization to identify what type of LMS would be most useful and user friendly for a particular population of learners and instructors.

Having a strong interest in the possibilities for improving education by creating educational videos, he has visited various sites that use similar delivery formats. His extensive knowledge of LMS and video enhanced education should aid the evaluator in writing a useful report about Sophia that could highlight both strengths and weaknesses. As an external evaluator, while he hopes Sophia is successful as an enterprise, he is not reliant on the support of Sophia’s leadership team and his employment situation will not be impacted by the content of the report. If a member of the leadership team loses their job, it would not have any impact on the evaluator’s ability to conduct the evaluation. This is one of the benefits of being an external evaluator, as the evaluator’s job security is not tied to the success of the organization.

Despite being aware of the limited use Sophia is currently receiving from educators throughout the United States, the evaluator is also aware of the possibilities of the platform and its innovative nature. Reading both negative and positive feedback, Sophia has become a useful virtual space in which educators can share their knowledge with their peers and all types of students. As an advocate of the capabilities of digital technologies to positively influence education, the evaluator’s general orientation toward the program is inclined to be positive, as he believes in the merits of the program’s infrastructure, goals, and objectives. Despite having a strong appreciation for both qualitative and quantitative data, most of the evaluator’s prior work was of a qualitative nature. Nevertheless, the evaluator is aware of the importance of analyzing the various statistical indicators that have been stored and categorized by the site analytics. However, in his evaluation, he will combine the use of focus groups and interviews with an analysis of the data collected when visitors arrived at the website and their actions were recorded by the system. The focus groups and interviews will allow the evaluator to have a more in-depth appreciation of the stakeholders’ conscious concerns. Despite the evaluator’s lack of experience in conducting formal program evaluations, he has completed college coursework in evaluation, statistics, focus groups, and other research methods. He has also completed a number of courses in the learning technologies department and has an extensive history of working with educational technologies and Learning Management Systems.

Constraints of the Study

The greatest difficulty in evaluating Sophia will be to find out why many of the individuals who visited Sophia on one occasion decided it would be better to not participate and create an account on the website. Most of the individuals who could be contacted for interviews, online focus groups, and surveys are those who are currently members and have shared their contact information online. However, a large number of visitors did not get to this stage and very few of them filled out the feedback form explaining the reasons why they were not planning on joining the community. Obtaining information from individuals that had an interest but quickly decided it was not what they expected would be helpful in understanding some of Sophia’s current deficiencies and decide whether some of them can, and should be, addressed.

With a small staff with a large responsibility, it is difficult to obtain access to Sophia’s employees to discuss in detail what they consider their roles to be and learn from them what they consider to be problems and successes of the program. Another possible limitation includes the reality (in contrast to the expectations) of how much analysis will be possible using their site analytics. While they current collect extensive usage data, some of which is displayed on every Sophia package webpage, and more detailed information is expected to have been collected by the site, it is uncertain how much of this data will be sortable and accessible to the evaluator. It is possible that some of this data may have been lost in a prior server update and the anticipated comparison of usage over time will be limited by the periods and quantity of data that are currently available. Another constraint will be the timeline of the evaluation. Despite the need for an extensive evaluation, the researcher will only have time to fully address some of these questions. Choices regarding what questions to pursue will have to be made, limiting the attention that can be given to other valuable and interesting questions. Depending on the data and the extent to which it was already pre-sorted by the computer system the statistical analysis will be easier or harder to complete satisfactorily. If other individuals volunteer or decide to contribute in other ways to the evaluation, a larger number of questions about Sophia may be successfully answered.