#LAK12 upcoming EDUCAUSE learning analytics focus session

The announcement about the ELI 2012 Online Spring Focus Session on learning analytics arrived in my inbox yesterday. Yet again, the reference to libraries:

"The analysis of this data, coming from a variety of sources like the LMS, the library, and the student information system, helps us observe and understand learning behaviors in order to enable appropriate interventions. "

Because EDUCAUSE's focus is the management and use of and leadership in information resources, librarians are a significant portion of the group's constituency. So I'm glad, in fact, that librarians are explicitly listed in the focus session announcement as a target audience.

However, since the sharing of user data (at least personally identifiable) would seem to be against the librarians' professional code of ethics, I'm still stumped as to how libraries have been placed as possible participants in the learning analytics sphere. Don't get me wrong...I'm glad to have been asked to the party, but I'm not sure I'm going to want to dance. Some libraries are involved in analytics (for example, the Library Impact Data Project in the UK, which I learned about from commenters on this blog) and I'm curious as to how their projects work and still honor values such as intellectual freedom and user privacy.

#LAK12 Gamifying SNAPP

In a recent blog post over at David Jennings' Quickened with the Firewater blog David asks: "What would happen if we put learners in charge of analysing their own data about their performance?"

Here's how it could happen with SNAPP, which is a software tool that helps online instructors analyze the communication patterns among students who use the LMS' communication tool:

• the instructor would allow communication to happen using the LMS, perhaps providing some task guidance or parameters to inspire the online discussion.



• The network visualization (like the one at the right, from the SNAPP group) could be shared with the students, who would use the visualization to help them think critically about how communication unfolded and what they individually added to the discussion.



• Learners could then be challenged to change the visualization by changing their own communication patterns as a group, thereby gamifying the system.

Learners would gain insight into their group communication processes and play with different communication roles. This might be especially useful in a management or communication course, but such insights would be valuable to anyone no matter their course of study. But what learners might also gain, in addition to learning about communication principles, is insight into how such systems can be manipulated, and how the systems might also be used to manipulate learners -- both worthy learning goals.

P.S. thank you Shane Dawson, for a fantastic presentation yesterday about SNAPP for the learning analytics massively open online course!

#LAK12 Word Cloud of Data Privacy & Ethics Chat

We had a lively online chat during Erik Duval's presentation on privacy and ethics in learning analytics.
Here are some details that you can see on the word cloud:
--"brother" as in Big...some of us brought up the "Big Brother" theme when the question was asked, "What do you worry about with sharing your data?"
--Lies vs. truth: is it considered lying when you present yourself as other-than-you? Is it your responsibility to present the truth about yourself online?
--transparency is a concern: who can see the data? who can see the models?
--power: who has it in learning analytics? who doesn't? Do teachers, learners, or administrators have power in the system?
--We discussed Google and Target as two corporations who are mining our data for marketing insights.
--medical: How are learning analytics issues similar to issues encountered with personally identifiable medical information?

#LAK12 Do Computer Scientists Do Science?

Dragan Gasevic's presentation about evidence-based semantic web shows that the software engineering field is beginning to adopt the paradigm of evidence-based practice (EBP) which has already been increasingly adopted in medicine, nursing, education, social work, and other human services fields.

In the evidence-based paradigm (and it indeed is a paradigm shift, especially for the medical field that birthed it), the randomized controlled trial is considered to be the study type offering the strongest possible evidence to support a hypothesis. (Note, however, that not every research question lends itself to an RCT. In software engineering, there may be other study designs which are more appropriate.) Systematic reviews and meta-analyses of many randomized controlled trials represent even stronger evidence. As its name suggests, a systematic review requires a systematic and methodical search of the literature in order to present an overall synthesis of results from the highest-quality studies that can be located. A meta-analysis goes several steps farther in that you would take the results of several related studies (all of them RCTs, or cohort studies, or case control studies, etc.) and pool the data, with the effect of creating one large study which can then be analyzed. Both systematic reviews and meta-analyses are important contributions to a field. Although we might not think of them as empirical scientific studies in themselves, they synthesize the entire body of empirical work that has been done on a topic to date. This synthesis is more than the sum of the results of the component studies.

The state of software engineering, as Gasevic and others seem to point out, is that most so-called evidence in the field consists of case studies or even simply expert opinion -- both at the very bottom of the EBP evidence hierarchy. The higher-level empirical studies that have been performed are often with small n, thereby decreasing the power of the studies to detect a statistically significant intervention effect. This is similar to the situation in other fields that have begun to adopt EBP. Gasevic shows that the study design methods, sampling methods, and data collection methods of published papers in software engineering are lacking in quality. If science is the application of rigorous methods to hypothesis testing, then is this a situation wherein computer scientists & engineers aren't practicing much science at all?

#LAK12 Semantic Web: Glimpses of Understanding

What I understand about the semantic web is that it:

-- relies on metadata that codes the metaphysical identity of a piece of data and how it relates to other things or concepts. Is it an image? a person? an idea? An author? This reminds me of bibliographic cataloging in libraries. ("The Semantic Web: An Introduction")

--exists on a very small scale currently ( Tim Berners-Lee's TED talk).

--would allow us to find and visualize relationships between any two bodies of information, whether the information is a person or an image or a body of raw data.

--would allow the implementation of learning analytics on a much more far-reaching scale.

This is what I understand and it isn't much, but what I don't understand is a lot. I really did not understand the "semantic web: an introduction" paper, nor the specifics of Hilary Mason's talk, but I found it fascinating none the less, and was particularly pleased that she used a disease-related example to illustrate Bayesian statistics. I made a connection from that to the concept of evidence-based medicine which I've also been learning about in the past couple of years.

Still need to watch Dragan Gasevic's presentation from last week. Perhaps it will make all things clear.

#LAK12 Knewton Love

Thoughts from my viewing of the Knewton video:

Flashback to 1992: I'm teaching at a private high school in Memphis, Tennessee. It's my first year of teaching at this school AND my first year of teaching high school. With 5 sections of students in two subjects, finding time to simply deal with the paperwork, classroom discipline, and preparing a new lesson plan for each day is a challenge -- let alone individualizing instruction for my students. A parent contacts me about her son. Without being overly accusatory, she tells me that one of the reasons she placed him in this school is that she hoped he would get some specialized instruction, but that's not happening. He's a gifted student, and now he is bored. I feel frustrated because just devising a single lesson plan to reach the average student is challenging enough; there's no way (I felt at the time) I can meet his needs too.

Knewton would have helped. It represents a way to use LA to do what all good teachers should be doing, but many don't have enough time to do:

--group students by learning preference/style, rather than by ability. This allows faster learners to learn more by teaching their peers.

--identify "study buddies" for students based on specific concepts plus learning preferences. Again, this allows students who have a mastery of a concept to learn more by sharing their mastery, plus students who still need to learn a concept are surrounded by more potential teachers. This would allow teachers to implement peer teaching in their classrooms.

I did notice at least one red flag: towards the end of the video, the narrator mentions that one of the benefits of Knewton for publishers is the establishment of a "lifetime relationship with students" allowing them to develop rich data on a student that could not be "shared, mined, or pirated." There's the catch! And raises the question, who really owns this data? The student or the system designer?

#LAK12 Community colleges: the perfect candidates for LA

As illustrated by Vernon Smith's presentation on Rio Salado's implementation of LA, community colleges would serve as the perfect testbed/incubator/adopter of learning analytics. Here's why:

1) Community colleges are bursting at the seams. Taken as a group, they represent a very large population = large "n" for doing analytics.

2) Community college faculty, for the most part, are there to teach, not to do research. There is a pragmatic focus on student success. Therefore, there would probably be more buy-in at the faculty level for LA.

3) You have a larger "n" of "at risk" students. Percentage-wise, the"at-risk" students represent a larger chunk of the community college population since the barriers to entry (such as price and high school performance) are lower than those at a four-year college/university. Therefore, you would probably get more return on investment with implementation of LA due to increased retention of these students.

4) In terms of human development/capacity, the ROI would probably be much greater at the community college level too, as many CC students are first-generation college students. Retaining a greater number of these students would have huge positive implications for society at large. (especially if education helps them see the need for changing the system status quo)

In contrast, I work at a medical school where we have a huge barrier to entry and a population of only about 145 students per cohort. This is a small "n" and I don't believe retention is an issue at all. "Completion" is not our concern. Improved learning is our concern, but in terms of return on investment for American society, I think money spent on LA at the community college level would yield a greater payoff. (unless, that is, the federal government and medical schools start aggressively pursuing a different student population in hopes of solving the huge problem of lack of access to health care of rural and urban Americans...)