TEL-ling tales – where is the evidence of impact?

open laptop with the word learning on the screen

Research is complex. It can be a messy business, but it matters. Higher education revolves around research and student degrees yet when it comes to the REF, pedagogical research in HE has a poor showing. A recent HEA funded investigation found critiques of submission quality* while back in 2002, Jenkins described it as having Cinderella status. A paper by the HEA researchers (Cotton, Miller and Kneale, 2017suggests pedagogical research in HE remains the Cinderella of academia.

If pedagogical research in HE is struggling for recognition where does this leave the field of education technology or Technology Enhanced Learning (TEL)? The critiques are plentiful** so where is the evidence of impact?

digital-tech-pixabay

I have great respect for TEL colleagues and wearing my curiosity hat, I headed off to a closed learning technology mail list. Citing Surowieckis ‘wisdom of crowds’, I invited members to point me to evidence of TEL enhancement of learning and/or teaching.

I don’t know what I expected. Maybe references to the OLDS mooc project, a NMC Horizon Report, the OU Innovating Pedagogy series or anything from the Jisc elearning projects.  Maybe the application of a model like Laurillard’s conversational framework or her work on teaching as a design science, how Salmon’s Five Stage model of e-moderating was used or Garrison and Anderson’s Community of Inquiry.  At home I have an old fashioned plastic box full of printed papers from my TEL research literature reviews, some by well known names and others less so but all with a variety of methodologies and results. Admittedly, much is aspirational – revealing potential for scaling up rather than broad adoption, but they represent hope. So I thought it might be useful to scope the most popular ‘go to‘ pieces from others and collate them for sharing.

red question mark on a keyboard

The response was not quite what I expected. Maybe I asked the wrong question. Maybe my view is different and maybe this is a Hull issue – in the nicest possible way! As Philip Larkin said, we ‘re on the edge of things rather than the centre and being on the edge can give you a different perspective. Whatever the reason, there were lots of ensuing discussions, some tweets and a couple of blogs – all showing a variety of reactions – Show me the Evidence by James Clay and In Defense of Technology by Kerry Pinny – but no links.  There was also an #LTHEchat invitation ‘Establishing an evidence base for TEL’  which will take place on Twitter, 3rd May, 8.00-9.00 (diary date!) If the questions were wrong at least they generated some positive consequences.

tweetchat-tweet small

I think Kerry was closest to my position when she described asking questions as scholarly practice. If we’re not research-informed and engaged how do we know if we’re having impact? Familiarity with the literature and taking time for critical reflection is about thinking academically and we work within academic environments where TEL is promoted as an enabler and enhancer of student-learning. Pedagogical research might not be scoring 10 out of 10 with the REF but it’s our daily bread and no reason to ignore what’s out there or not adopt a scholarly approach to evidencing our own practice – in particular with TEL matters. Institutions are investing huge amounts of money into digital platforms supporting learning and teaching but less into supporting staff to develop the digital capabilities and confidence to use them.

media-studies

It’s now twenty years since the Dearing Report into the future of higher education which preceded the arrival of the VLE. Since those early days we’ve shifted from a read-only environment to user generated content, file sharing, mobile devices, social media, apps, virtual reality etc etc yet there’s still disparity of adoption and a widening divide between the innovators and those yet to climb aboard the TEL train.

What came out of the discussions (and what I see every working day) was how resistance to TEL remains high. Also it’s clear what’s missing includes the time, space, reward and recognition for staff engagement. We’re grappling with this at Hull and to make our case to SMTs requires evidence of impact on student learning and staff well-being. To find the evidence we need the research.

So where is it?

What do other TEL people use as their rationale for TEL matters?

magnifying glass


footnotes

* critiques of pedagogical research in higher education include small sample sizes, localised research not capable of wider dissemination and limited contribution to theory. This is similar to the examples of critiques of TEL shown below.

** examples of TEL critique

 ‘Our analysis of articles published in two leading journals [these were the Australasian Journal of Educational Technology) and ALT-J (since renamed Research in Learning Technology)] found…poorly conceived or poorly applied methodologies, limited reference to theory, weak results, incomplete descriptions, uneven presentation of data and overblown and unsupported claims of impact and importance.’ (Gunn and Steel, 2012:11)

‘….where the potential of technology to transform teaching and learning practices does not appear to have achieved substantial uptake, this is because ‘the majority of studies focused on reproducing or reinforcing existing practices.’ (Kirkwood and Price 2012: 24)

‘The majority of papers published in BJET and the other educational technology journals are in the form of small-scale, unconnected trials and applications which can have little influence on policy making.’ (Latchem, 2014: 2)


images from pixabay except tweet from #lthechat


 

 

Let’s get digital or not?

wine and cake pixabay

The Friday blog-habit is proving hard to break! If only I could be as strict with the Friday evening wine or Saturday cake. As in not having them.  The blog is almost a reverse addiction. Usually we go for instant gratification rather than delayed and blogging is definitely in the second category. It’s rewarding when posts get liked or quoted but that often comes days and sometimes weeks after the event!

On reflection, maybe the idea of pausing over the summer wasn’t so good after all. If the habit is established why stop? Exactly my approach to the Friday evening wine and Saturday cake. Why break something which works so well!

panopto logo

The days when August was the time for catching up and preparing for the new academic year are well and truly gone. Not only are we launching a new VLE in September, I’m also working on the policy document for Panopto and preparing staff development activities to introduce teaching with video (thanks Gemma Witton @gemmawitton from the University of Wolverhampton for the inspiring Panopto conversation this week)

The digital capabilities framework. continues to underpin everything I do. So far this year we’ve piloted the Jisc Discovery Tool and run the Digital Storytelling workshops. The TEL Team and the Library are now having regular catch-ups to discuss all things digital and I’m curating a ‘Sharing Practice’ resource center to demonstrate interesting and effective use of technology to support the student experience.

black and white cartoon, one dog tells anthother on the internet no one knows you're a dog

In LEAP there are Academic Practice Advisers and TEL Advisers. Unfortunately we’re divided by geography which reinforces the lack of opportunities to get together and discuss how maybe we should all be one and the same?  The minute you say the ‘technology’ word  those who see themselves as non-techie self-exclude yet we are all involved with learning and teaching. I want to ‘rebrand’ digital capabilities. I’m concerned the word ‘digital’ is getting like ‘technology‘ and the phrase ‘digital capabilities framework‘ is almost doomed before it begins. So what are my options?

pixabay education

I like ‘digital scholarship‘.  The HEA have reviewed the scholarship of teaching and learning (SoTL) and last year I attended a Colloquium event discussing the importance of being research informed and engaged in teaching practice. There are tensions over the meaning and evaluation of ‘teaching excellence’ but the TEF remains an opportunity to revisit institutional support for pedagogical research. Anyone supporting a VLE will be familiar with the persistence of transmissive approaches with emphasis on knowledge consumption rather than construction. We need to talk.

Time, space and rationale (as well as reward and recognition) are all essential prerequisites to change. Maybe the TEL Team could have a monthly ‘digital scholarship’ meetings over coffee – cake – or lunch – to discuss key papers and pedagogies relating to TEL – as well as ideas to publish and promote our work so yes, scholarship is a possibility.

There’s just one problem – I’m still using ‘digital’.

Should my new approach be with or without the D word?

Any suggestions?