TEL-ling tales – where is the evidence of impact?

open laptop with the word learning on the screen

Research is complex. It can be a messy business, but it matters. Higher education revolves around research and student degrees yet when it comes to the REF, pedagogical research in HE has a poor showing. A recent HEA funded investigation found critiques of submission quality* while back in 2002, Jenkins described it as having Cinderella status. A paper by the HEA researchers (Cotton, Miller and Kneale, 2017suggests pedagogical research in HE remains the Cinderella of academia.

If pedagogical research in HE is struggling for recognition where does this leave the field of education technology or Technology Enhanced Learning (TEL)? The critiques are plentiful** so where is the evidence of impact?

digital-tech-pixabay

I have great respect for TEL colleagues and wearing my curiosity hat, I headed off to a closed learning technology mail list. Citing Surowieckis ‘wisdom of crowds’, I invited members to point me to evidence of TEL enhancement of learning and/or teaching.

I don’t know what I expected. Maybe references to the OLDS mooc project, a NMC Horizon Report, the OU Innovating Pedagogy series or anything from the Jisc elearning projects.  Maybe the application of a model like Laurillard’s conversational framework or her work on teaching as a design science, how Salmon’s Five Stage model of e-moderating was used or Garrison and Anderson’s Community of Inquiry.  At home I have an old fashioned plastic box full of printed papers from my TEL research literature reviews, some by well known names and others less so but all with a variety of methodologies and results. Admittedly, much is aspirational – revealing potential for scaling up rather than broad adoption, but they represent hope. So I thought it might be useful to scope the most popular ‘go to‘ pieces from others and collate them for sharing.

red question mark on a keyboard

The response was not quite what I expected. Maybe I asked the wrong question. Maybe my view is different and maybe this is a Hull issue – in the nicest possible way! As Philip Larkin said, we ‘re on the edge of things rather than the centre and being on the edge can give you a different perspective. Whatever the reason, there were lots of ensuing discussions, some tweets and a couple of blogs – all showing a variety of reactions – Show me the Evidence by James Clay and In Defense of Technology by Kerry Pinny – but no links.  There was also an #LTHEchat invitation ‘Establishing an evidence base for TEL’  which will take place on Twitter, 3rd May, 8.00-9.00 (diary date!) If the questions were wrong at least they generated some positive consequences.

tweetchat-tweet small

I think Kerry was closest to my position when she described asking questions as scholarly practice. If we’re not research-informed and engaged how do we know if we’re having impact? Familiarity with the literature and taking time for critical reflection is about thinking academically and we work within academic environments where TEL is promoted as an enabler and enhancer of student-learning. Pedagogical research might not be scoring 10 out of 10 with the REF but it’s our daily bread and no reason to ignore what’s out there or not adopt a scholarly approach to evidencing our own practice – in particular with TEL matters. Institutions are investing huge amounts of money into digital platforms supporting learning and teaching but less into supporting staff to develop the digital capabilities and confidence to use them.

media-studies

It’s now twenty years since the Dearing Report into the future of higher education which preceded the arrival of the VLE. Since those early days we’ve shifted from a read-only environment to user generated content, file sharing, mobile devices, social media, apps, virtual reality etc etc yet there’s still disparity of adoption and a widening divide between the innovators and those yet to climb aboard the TEL train.

What came out of the discussions (and what I see every working day) was how resistance to TEL remains high. Also it’s clear what’s missing includes the time, space, reward and recognition for staff engagement. We’re grappling with this at Hull and to make our case to SMTs requires evidence of impact on student learning and staff well-being. To find the evidence we need the research.

So where is it?

What do other TEL people use as their rationale for TEL matters?

magnifying glass


footnotes

* critiques of pedagogical research in higher education include small sample sizes, localised research not capable of wider dissemination and limited contribution to theory. This is similar to the examples of critiques of TEL shown below.

** examples of TEL critique

 ‘Our analysis of articles published in two leading journals [these were the Australasian Journal of Educational Technology) and ALT-J (since renamed Research in Learning Technology)] found…poorly conceived or poorly applied methodologies, limited reference to theory, weak results, incomplete descriptions, uneven presentation of data and overblown and unsupported claims of impact and importance.’ (Gunn and Steel, 2012:11)

‘….where the potential of technology to transform teaching and learning practices does not appear to have achieved substantial uptake, this is because ‘the majority of studies focused on reproducing or reinforcing existing practices.’ (Kirkwood and Price 2012: 24)

‘The majority of papers published in BJET and the other educational technology journals are in the form of small-scale, unconnected trials and applications which can have little influence on policy making.’ (Latchem, 2014: 2)


images from pixabay except tweet from #lthechat


 

 

The problem is not ignorance, it’s preconceived ideas

https://pixabay.com/en/binary-code-man-display-dummy-face-1327512/
https://pixabay.com/en/binary-code-man-display-dummy-face-1327512/

Data is never neutral.  This is my social science background talking. It’s made me suspicious! Or should that be critical?  Not everyone agrees but I’ve always distrusted the ability of stats to tell the full story.

This week it was announced Hans Rosling has died. A sign of the internet age is the videos we leave behind. This link to a TED Talk (2006) The best stats you’ve ever seen begins with his trademark introduction ‘I’m a statistician – No – don’t switch off!

Rosling set out to show the changing world through the visualisation of data.  The concept was simple. Most good ideas are. Publically funded statistics exist but are not presented in ways which are educational and accessible. Rosling founded the Gapminder organisation to create software linking data with presentation tools, thereby making it visible and searchable or in this own words – liberated. Helped more than a little by a narration owing more to a sports commentary than traditional academia, graphs have never been so entertaining or eye-opening. Mission accomplished.

Hans Rosling presenting on a stepladder

Over the years Rosling moved from overhead projector, with his trademark stepladder for reaching the high parts, to more sophisticated forms of digital touch screen representation. The technology was wizzy but somehow wasn’t the same.

hans-rosling-digital

I saw Rosling present a couple of times. Mostly on the international health and social care arena where he spoke about the world and what really matters;  fertility rates, child mortality, family planning, distribution of income and the power of social change. There were always a number of key messages. Data is better than you think; there may be  an uncertainty margin but the differences revealed are larger than any weakness. Data can be structured e.g. revealing the importance of context an highlighting diversity, sometimes within single countries. Most relevant to educationalists, Rosling maintained problems are not caused by ignorance but through preconceived ideas.

USB PLUGS
https://pixabay.com/en/network-connector-network-cables-494651/

Data is big business and higher education has not escaped from the lure of using stats to review and refine the student experience. Within  institutions the VLE dashboard and NSS (National Student Survey) have been used for some time to wave red flags. Now the TEF is bringing data analytics to the forefront. The relationship between NSS scores, figures from HESA (Higher Education Statistics Agency) and DELI  (Destination of Leavers from Higher Education) and teaching excellence is still open for debate but there’s no denying how ‘Learning Analytics’ is now positioned centre-stage.

All my initial reservations about statistical data have come back. It’s one thing to collect and group figures into charts and tables but useful interpretation depends on wider issues such as identifying what you want to know and why you want to know it. Counting the times a student logs onto a VLE or walks into the library tells us little about the nature of their activity or quality of engagement.

digital number display
https://pixabay.com/en/nixie-tube-electronics-voltage-1501592/

The biggest concern is the rhetoric. The Bricks to Clicks report tells us data has “enormous potential to improve the student experience at university” while the Jisc report Learning Analytics in Higher Education offers analytics as a tool with many functions. These include quality assurance and quality improvement, boosting retention rates and assessing and acting upon differential students outcomes – to mention a few.

We’ve been here before in the early days of education technology which promised much with regard to enhancement but with little evidence of improvement. Deterministic approaches see technology as the agent of change rather than focusing on the cultural context in which it’s positioned. Today it seems there’s an increasing risk of data being seen through a similar determinist lens.

magnifying glass
https://pixabay.com/en/magnifying-glass-hand-glass-magnify-29398/

Education developers and researchers want teaching interventions which produce the most effective learning environments. As it stands, I’m not convinced the collection, measurement and interpretation of all this data for the TEF will produce any meaningful information about what we really want to know. The Learning Analytics movement needs someone like Hans Rosling to challenge preconceived ideas and find ways to interpret data which are innovative, useful and accessible.

It would also be worth asking if the data we have is from the most appropriate sources in the first place.

2017 looks like an interesting year

2017

2017 will be an interesting year. The Doomsday Clock moves closer to midnight. Penguin order a 75,000 reprint of Orwell’s 1984. Council for the US President invites debate on the nature of ‘alternative facts’ while the alleged influence of social media during the US election suggests ‘critical digital literacies’ warrants attention.

Should Media Studies be taken more seriously? Narrowly escaping Gove’s bonfire of the subjects the MS GCSEgives students the chance to develop a critical understanding of the role of the media in daily life’  while MS A level includes ‘understand and evaluate how meanings and responses are created’ alongside ‘develop and formulate… its influential role in today’s society.’ The only thing wrong with a well taught Media Studies programme is it isn’t mandatory!

media-studies

It’s not too far fetched to suggest truth and knowledge are under attack. The internet is awash with speculation and fake news masquerading as truths. To counter balance this we may need to re-frame ontology and epistemology in everyday language, encourage examination of media beliefs and interrogate preferred sources of news – be it the BBC, Daily Mail or Twitter. Words like ideology and discourse can be challenged but should at least be discussed. A baseline dose of Foucault with Orwell’s 1984 and Huxley’s Brave New World wouldn’t do anyone any harm.

Following the US election I blogged about the media in The Truth is out there somewhere . Today, just 8 weeks later, it seems even more important to address issues of truth and knowledge. Regardless of personal politics, the US election is a wake-up call.  Who do we believe and why? Suroweiki’s Wisdom of Crowds and Gladwell’s Tipping Point have supported the development of Communities of Practice – where relevant knowledge is situated within the connections we make  – and these concepts remain true; the risk is when wisdom and tipping points are based on fakery rather than evidence.

community

2017 will be a busy year.

I’ve begun to let go of wider commitments to create space for the PhD and shhh…. complete the dissertation for my p/t creative writing degree which no one knows about so far! The timing is not perfect but when it comes to doctoral research there’s no such thing. A time-table is mapped out for the PhD transfer process to the University of Northampton, a week with NVivo booked for April and I’m loving the educational research conversations with my supervisors, Director of Studies and fellow students.

On the work front there’s an exciting new focus on Learning Design. This will inform development of a digital capabilities framework for staff who teach and support learning and I want to develop the world of the TEL-People. The only downside is withdrawing from projects like #creativeHE and it was really difficult to put #poetryfeedHE to one side – albeit temporarily – but something had to give.

Lastly – the allotment lies waiting – as it did last year, and the year before that. At some point I need to find my allotmenteering hat and wear it again.

So here’s to 2017.

Did I say it looks like an interesting year? 🙂


 

 

Reinventing lurking as working #socmedHE16

img_1983

Last week I attended The Empowered Learner; the 2016 Social Media for Learning in Higher Education Conference (#socmedHE16) at Sheffield Hallam University.

The Keynote for last year’s conference was Eric Stoller with his amazing Star Wars effect opening.  You can make your own this Christmas.  Eric was always going to be a hard act to follow and the conference organisers didn’t try. They offered a Key-Not instead.

img_1988

A Key-Not translated as an activity. We were divided into four groups depending on the colour cup we’d chosen. Sneaky – I didn’t see that coming! The task was to use a social medial tool(s) and collaboratively build a resource for the ’empowered’ learner. My group – the Yellow Custard Stirrers – used Adobe Spark to show how to set up a Facebook Group and invite participants. We won! Well done fellow Stirrers – may your custard never go lumpy!

img_1989 img_2017

The Key-Not was followed by two traditional style presentations. After the frenetic activity of the previous hour, it felt strange to be back in passive audience mode. The mobile devices came out and people slipped back into isolation from each other while remaining connected to the virtual. What saved it – for me – was these were two of the best presentations of the day.

I want to blog about Andrew Middleton’s presentation separately. ‘Social Media as a Critical Future Learning Space’ resonated on lots of different levels.

a6fce5dc-575b-40c2-8957-b63c7b52aa43

I blogged about space earlier this year in a post titled Simultaneous Existence and want to go back to the subject, in particular the ssignificance of interstitial space.

74c79f79-6c0e-4e56-b52d-0e5fdb7d1445

The first presentation by Sarah Honeychurch was about lurking. Now I lurk, you lurk, we all lurk but the word has negative connotations. Traditional definitions include sinister, threatening and unpleasant while its latest linguistic incarnation in relation to discussion forums suggests to lurk is an incorrect or inappropriate thing to do.

I want to re-imagine lurking as working.

In these days of information overload through TEL, email, cloud computing and social media, we are mostly not waving but drowning I would suggest just being there online – long enough to register what’s happening before moving on to the next task – is about as much as anyone can manage.  If we re-invent lurking as less something negative, more a positive affirmation and recognition that we managed to get there in the first place, we could then change attitudes to ‘didn’t we do well!’

There’s lots of ways learning and teaching in HE use social media e.g.#lthechatTLC webinars#creativeHE community – and I’m sure these are all places where lurkers lurk, simply to keep up to date and check they’re not missing anything useful. Social Media is the single most valuable network of curated content which can be customised by choosing who to follow and which events to attend – even if it is in a lurking capacity. To lurk is better than not being there at all. It really is time for a linguistic turn.

laguage matters inside a blue speech bubble

Language matters. I don’t like to hear training or skills being used in relation to my TEL work and try to avoid the words lecture capture. Words like these carry connotations which don’t sit well with the objectives of enhancement and innovation which sit within my own interpretation of TEL.

So here’s to lurking as working. Remember – I lurk, you lurk, we all lurk. To lurk is a coping mechanism. It means we care enough to make the time to log on and check what’s happening in our own spheres of interest while also – apart from anything else – not everyone wants to be in the digital spotlight. Lurkers should be proud of their background activity and online bloggers, tweeters, and activity creators be pleased to have them there. A silent audience is better than no audience at all. Remember – as the email goes quiet and the festivities begin – it’s good to lurk and don’t let anyone tell you otherwise.


postscript

On the BBC News today people are advised to take a break from social media over the Christmas period as lurking may make them miserable and depressed <sigh>


 

The Empowered Learner #SocMedHE16

Social Media Conference Banner

Friday morning. 6.41 train from Hull. Heading to Sheffield Hallam University. It’ll be dark and cold but well worth it to attend the second Social Media for Learning in Higher Education Conference ‘The Empowered Learner

I’m helping facilitate a ‘socially mediated workshop about developing a social media workshop’ The repetition is deliberate and the workshop will be using the UCISA Social Media Toolkit as a baseline. The Toolkit offers a useful guide for universities using social media tools. what ever the reason; learning, teaching, research or administration – it preempts some of the questions which might be asked and contains a wealth of advice and support from those who’ve already tipped their digital toes in the social media waters.

UCISA Social Media Toolkit front cover

The rest of the programme looks interesting – as always, the perennial problem is selection – which to choose and which to miss.

The Keynote has been retitled Key-Not. The rationale for this intriguing name will be revealed on the day. If you can’t gt to Sheffield there’s an online option. The conference website says ‘If you are free between 9.15 and 11.15, will be online and like a challenge you are invited to participate directly in our online version of the Key-Not.  Please email socmedhe@shu.ac.uk with ‘Key-not’ in the subject line and we’ll fill you in.  Otherwise, watch out for a million tweets in the morning, and keep an eye on this page.’  The Twitter hashtag is #SocMedHE16 and some of the sessions will be periscoped – see the conference website for further details.

Yesterday there was an announcement. To coincide with the national event, this year’s conference would also celebrate the Great British Christmas Jumper.

Ooops – I don’t have one.

The closest I get is a little Ode written last month when colleagues were starting to discuss the annual CJ – so in the spirit of Christmas Jumpering, and the absence of one of my own (not to mention taking advantage of social media!) I include it here.

Have a good conference everyone.


ode-to-a-christmas-jumper


 

It isn’t technology it’s learning design

communication-pixabay

This began as a PhD reflection but turned into a blog post because the issues matter. We need to talk. What has gone wrong? Quite a lot.

December 1 was the first Graduate School Research workshop day. I thought joining in remotely would be a motivator. I’m self-funding so every opportunity for contact is welcome. Via Collaborate and a fixed camera I followed the slides and presentations during the morning but couldn’t join in the activities. In the afternoon the sound was lost. I’m hoping the recording will be ok and wondering when it will appear on the VLE. The cognitive connection has already faded.

What is meant by the phrase ‘online distance learning?’ What did I get from passively listening and watching on my laptop? Not a lot to be honest. The common model of distance learning is still a delivery one. Recorded lectures are seen as progression and if you build in some formative MCQ then Hallelujah – you have an online course.

My PhD includes mandatory research and ethics modules. They’re produced by Epigeum, so expensive and considered gold standard. I’ve sat alone in my room clicking through linear screen after screen of content in order to take the test at the end. It’s lonely and my learning is surface recall rather than any deeper approach achieved by cognitive understanding via critical reflection or discussion with colleagues.

What has gone wrong with the promise of student centered, interactive collaborative learning – online? How can the principles of Social Learning Theory be applied to what is fundamentally learning in isolation?

digital-tech-pixabay

In the THES last month there was a piece called Mass Learning must mean web based study It claimed the elements exist to make online learning happen, but ‘institutional inertia’ creates lack of progress. Thinking this might refer to the invisibility of on-campus digital divides or lack of recognition of diverse digital capabilities I read on. Technology has its problems I’m told – and here the piece links to Distance and Discontent the Downside of Digital Learning – but it will continue to evolve, solving all the negative issues as it does.

There are barriers such as the need for more teaching hours (at last – acknowledgement it requires additional resources rather than less to build and run effective online learning environments) plus new forms of examination and inclusion of ‘the broader social and cultural benefits of higher education’ but hey the piece goes on – none of these are insuperable. No. The problem is the university itself. Over the past decades they’ve continued to expand their physical presence at the expense of their virtual one, to a point where they can no longer afford to go online. As the author says, Turkeys don’t vote for Christmas but – don’t you know – technology is still the solution!

turkeys-pixabay

As if this were not depressing enough, the Distance and Discontent piece offers two further narratives of online education failure. Against a sector which still shouts about the transformative power of digital environments, something isn’t fitting. The rhetorical promise of e-learning solutions continues to be promoted in headlines, straplines, Jisc-speak and conference halls. In the meantime research and anecdote speak of digital depository models of VLE usage, empty discussion forums and neglected project sites return broken links and 404 errors.

So often over the years I’ve seen digital layers added onto existing face-to-face practice. It rarely creates effective online learning because it isn’t about the technology, it’s about learning design.


images from pixabay.com 

the truth is out there somewhere

image of a magnifying glass over the word truth and the words lies appearing beneath the glass

Critical digital literacy should be embedded throughout the higher education experience. We all need effective ways to tell the difference between truth and lies, not just for ourselves but those around us. In 1970, Alvin Tofler called our information explosion the Third Wave, the next greatest social movement following the Agrarian and Industrial ages. What would he say if he could see us now – not waving but drowning in information overload!

Yet the quantity is the least of our problems. It’s the quality which matters. New genres have appeared, in particular since since Brexit and Trump.

Da da!

introducing

Post Truth and Fake Truth.

image of the word truth as a jigsaw with missing pieces

They sound similar but there’s a difference. Post truth, most often used in connection with politics, appeals to emotions rather than presenting factual evidence. With Post Truth, what is true is secondary to getting that emotional hit, appealing to the personal and turning it into political action. Fake Truth or False Truth is another way to describe spin. Also known as Fake News/False News, it describes not so much the misinformation but the spreading of it via social media. Like Chinese Whispers, the story changes, getting further away from the original sources, picking up more emotional overtones as it travels on through digital space and time.

black and white image of a pile of books demonstrating different genres

A genre is born when new ways to structure and present information are created. Genres can be different styles of creative writing such as the thriller, detective or horror novel or it can be categories and styles of non-fiction news. Today we have what could be called genres of lies; deliberately false information masquerading as truth with the sole purpose of persuasion.

George Monbiot writes about the misinformation machines. He claims huge amounts of money are spent on setting up international and corporate think-tanks, bloggers and fake citizens’ groups. Their objective is swaying the hearts and minds of the electorate over big issues like immigration, employment and climate change. (Monbiot also refers to Trump and hyporeality which sounds to me ike Baurillard’s hyperreality nightmare come true – I think this may be is next week’s topic sorted!)

Falsity is not new. The internet has always been full of lies as has the world of advertising. At the beginning of the 20th Century, Edward Bernays applied the psychoanalytic ideas of his Uncle Siggy to persuade young women to smoke and increase the popularity of the colour green. His techniques were called Public Relations or Propaganda, depending if you were on stage controlling the show or in the audience watching it. Century of the Self by the brilliant documentary film maker Adam Curtis tells how America learned to take control of its population. Using archive footage, he tells the story of how Bernays, nephew to Sigmund Freud, laid the foundations of mental manipulation by the media, showing how ‘desire’ was created and blurred boundaries between truths and lies were established.

Control of the media equates with control of the people. George Orwell portrayed this as ‘Big Brother‘ in the novel 1984 and showed how deliberately  vague or meaningless language was used to conceal the truth in his essay Politics of the English Language.  In Understanding Media The Extensions of Man, (1964) Marshall McLuhan predicted the medium as well as the message would influence attitudes and behaviors while Neil Postman claimed we would be Amusing Ourselves to Death (1984) as the platforms of the public sphere were taken over by cable tv’s multiple channels leaving no place for discussion and critique of political discourse.

Were these writers prescient? Do we recognise the world they predicted?

digital divide with a page and an ipad

Early founders of the internet claimed it was a tool for social democracy because it offered equal access to information. Instead we have digital exclusion as the new but invisible category of social and economic discrimination. The development of user generated content via sites like Facebook and Twitter was hailed as a tool for the revolution, giving voice to minority groups and bestowing powers of resistance and subversion. Instead, we have a mess.

image showing social media logos from pixabay

For vast swathes of the population, social media has become the single source of truth. Mobile digital media supports speed swapping of news, presented in soundbites and video clips. Adjective heavy headlines and sensational straplines frame news stories telling the reader how to emotionally approach them. Reality TV confuses truth and fiction, magazine industries are built on ‘true’ confessions while multi-channel news is invaded by false news stories. As well as Monbiot, this weeks’ Guardian also has Roy Greenslade on Post Truth and the art of lies citing Barack Obama and his observation the morning after the US election that how the ‘new media ecosystem‘ of social media means ‘everything is true and nothing is true‘.

It seems this is the week for talking about truth.

But of course, after reading all this, you may not believe a single word I have said.