Irene van der Staaij, author of the Statistical Manual Learning Analytics
On Friday February 14th, the zone Secure and reliable use of learning analytics has launched the Statistical Manual Learning Analytics at SURF in Utrecht. We interviewed Irene van der Staaij, author of the Statistical Manual, on the importance of a statistical manual for learning analytics and how it will contribute to the quality of higher education.
The Statistical Manual Learning Analytics was launched on February 14th. Why is this important?
“A lot of work is currently being done on learning analytics. At universities of applied sciences and universities, student populations and how they move within institutions and programmes are examined. It is about which choices they make in their studies, which courses they take, what steps they take around bachelor’s and master’s degrees. This is often done based on tables or graphs of the data, in which for example a difference between two groups can be seen. More is needed to ensure that the observed difference is also statistically significant. That is why Statistical Manual is important.”
How will higher education actually benefit from this? How does it improve the quality of our education?
“You can ask yourself many policy related questions that you could answer using learning analytics. Sometimes you see a correlation within the data, but you are not sure whether you can draw your conclusions with certainty. How you can answer your questions with the statistical manual depends on the size and composition of your data. The manual will mainly help to test relations at the level of student groups. For example, if you see that certain groups are experiencing significantly more study delay at a certain point, you can organise study counseling in a more targeted manner.
Also, you can measure whether there is a difference in the number of hours a student studies before and after a tutor interview. Another example is that you can test certain courses, for example a course for study behaviour, in which you teach students how to study effectively. A study advisor can ask questions such as: Do students study more or less after such a course? Or do students feel that they can handle their studies better after taking the course?”
Who will use the Statistical Manual and how will they do that?
“Anyone who works with learning analytics within a higher education institution can use the manual. We have written the manual for people who have had statistics in their own education and therefore have a certain basic knowledge. In the manual we repeat the biggest and most important questions for working with data.
We designed the statistical manual after the example of the Statistical Manual of the Amsterdam UMC. We have made an overview of the most commonly used statistical tests with an explanation of how these tests should be used. Using the manual starts with a few basic questions: What type of data do you have? Is it numeric? Is the data normally distributed? And when it comes to categories, is it ordinal or nominal? What kind of comparison do you want to make? Do you compare groups? These steps are taken in advance, so that you choose the right test.
For each test, a case study from higher education is provided to illustrate how it can be used and for what purpose. It also describes what the conditions are for using each test and what a logical way of reporting is.
The strength of the manual is that we also provide code for each test in multiple programming languages: R, Python and SPSS. So we do not only describe the thinking steps leading to each test, but we also offer the associated code in the programming language you prefer.”
You are the author of the Statistical Manual Learning Analytics. How did you get started?
“Writing the Statistical Manual Learning Analytics initiated from the Acceleration Zone Learning Analytics of the Acceleration Plan. First, I collected all kinds of cases, together with my colleagues, that can be answered using learning analytics from higher education. We assigned these cases to tests and we worked out the manual for each test. This was followed by an extensive review process by colleagues from higher education who are involved in the acceleration zone and VU Analytics. Scientists and data scientists from universities (including VU Amsterdam and Erasmus University Rotterdam) and universities of applied sciences (including Leiden University of Applied Sciences and Hanze University of Applied Sciences) participated in the review process.
This diversity in the background of people who worked on the manual brought a nice dynamic. I have a background as a teacher of mathematics in secondary education and I have worked with statistics in neuropsychology. While developing the Statistical Manual it sometimes occurred that someone with a mathematical background looked differently at how I wrote down the information at first, for example by simplifying the information slightly. Then it often comes down to the question: are we going for the exact mathematical correctness, or for the usability of the manual?
We started with a number of tests and worked them out. On February 14 we have launched the Statistical Manual in Utrecht. Now we are extending it with more tests that can be done. We will continue to work on regressions and other analyses. Ultimately, we strive for the most complete set of useful statistical tests to support institutions that want to get started with learning analytics in the best way we can.”