Data is never neutral. This is my social science background talking. It’s made me suspicious! Or should that be critical? Not everyone agrees but I’ve always distrusted the ability of stats to tell the full story.
This week it was announced Hans Rosling has died. A sign of the internet age is the videos we leave behind. This link to a TED Talk (2006) The best stats you’ve ever seen begins with his trademark introduction ‘I’m a statistician – No – don’t switch off!’
Rosling set out to show the changing world through the visualisation of data. The concept was simple. Most good ideas are. Publically funded statistics exist but are not presented in ways which are educational and accessible. Rosling founded the Gapminder organisation to create software linking data with presentation tools, thereby making it visible and searchable or in this own words – liberated. Helped more than a little by a narration owing more to a sports commentary than traditional academia, graphs have never been so entertaining or eye-opening. Mission accomplished.
Over the years Rosling moved from overhead projector, with his trademark stepladder for reaching the high parts, to more sophisticated forms of digital touch screen representation. The technology was wizzy but somehow wasn’t the same.
I saw Rosling present a couple of times. Mostly on the international health and social care arena where he spoke about the world and what really matters; fertility rates, child mortality, family planning, distribution of income and the power of social change. There were always a number of key messages. Data is better than you think; there may be an uncertainty margin but the differences revealed are larger than any weakness. Data can be structured e.g. revealing the importance of context an highlighting diversity, sometimes within single countries. Most relevant to educationalists, Rosling maintained problems are not caused by ignorance but through preconceived ideas.
Data is big business and higher education has not escaped from the lure of using stats to review and refine the student experience. Within institutions the VLE dashboard and NSS (National Student Survey) have been used for some time to wave red flags. Now the TEF is bringing data analytics to the forefront. The relationship between NSS scores, figures from HESA (Higher Education Statistics Agency) and DELI (Destination of Leavers from Higher Education) and teaching excellence is still open for debate but there’s no denying how ‘Learning Analytics’ is now positioned centre-stage.
All my initial reservations about statistical data have come back. It’s one thing to collect and group figures into charts and tables but useful interpretation depends on wider issues such as identifying what you want to know and why you want to know it. Counting the times a student logs onto a VLE or walks into the library tells us little about the nature of their activity or quality of engagement.
The biggest concern is the rhetoric. The Bricks to Clicks report tells us data has “enormous potential to improve the student experience at university” while the Jisc report Learning Analytics in Higher Education offers analytics as a tool with many functions. These include quality assurance and quality improvement, boosting retention rates and assessing and acting upon differential students outcomes – to mention a few.
We’ve been here before in the early days of education technology which promised much with regard to enhancement but with little evidence of improvement. Deterministic approaches see technology as the agent of change rather than focusing on the cultural context in which it’s positioned. Today it seems there’s an increasing risk of data being seen through a similar determinist lens.
Education developers and researchers want teaching interventions which produce the most effective learning environments. As it stands, I’m not convinced the collection, measurement and interpretation of all this data for the TEF will produce any meaningful information about what we really want to know. The Learning Analytics movement needs someone like Hans Rosling to challenge preconceived ideas and find ways to interpret data which are innovative, useful and accessible.
It would also be worth asking if the data we have is from the most appropriate sources in the first place.