Some NHS e-Learning about an NHS MOOC – My Microethnographic Artefact for Block 2

If only we had anything this cool…

Here is a link to my Artefact. Since the MOOC I looked at was for a MOOC developed by the NHS, primarily for NHS staff and since I work as an e-Learning Developer for an NHS Trust, I decided to make my artefact in the style of one of the thousands of e-Learning courses that the organisation produces every year. Please don’t let this put you off, in this case the form of the artefact is in many ways part of the message I’m trying to convey 🙂

Artefact Link

Crouchipuss has uploaded some data

MOOC Data – all personal identifying info removed.

If anyone is interested in the data that I collected for my microethnography then these sheets contain the raw figures. My analysis went on to about a dozen sheets where I looked at this raw data in a number of different ways.

Created February 28, 2020 at 11:24AM
Folder path: /MScEDC Spreadsheets

View Google Sheet

Looking at the 1st week of my MOOC – caution lengthy post with graphs…

 

So, what did my intrepid expedition into the AI for Healthcare: Equipping the Workforce For Digital Transformation MOOC unearth in the first couple of weeks? As mentioned before, I’m using Week 1 of the MOOC for my microethnography. The MOOC runs over 5 weeks, so taking a snapshot of the first week, using data collected at the end of that week seems the fairest way to look at how the community started off.

Who knows or dares to dream what treasures we might find in this uncharted territory?

In terms of community it is difficult to judge just from my own experiences of the MOOC. I can see that people Like and reply to some of my posts, and I’ve left replies for some others. However, the format doesn’t lend itself to prolonged conversations. To see if my experience is broadly similar to others’ I have collected a list of every known user (those who have posted or replied to a post in week 1 or on the initial Hello thread). I have then collected data on how many posts each known person has made on each of the topic threads, how many replies where made on each post and by whom, and how many Likes each person received on their posts and comments. I know that this methodology is a crude way of measuring community. It doesn’t take into account the nature or usefulness of the posts or replies (for instance, replies that just state “well said” or “I agree”), but hopefully we can get a rough idea of how much the students interact through these figure. There is also no way of looking at Lurkers (people on the MOOC who never interact) as there is no way of finding out the total number of students on the MOOC. I would suggest that there is a subcategory of Lurker – the Like Lurker, who only interacts with others through Liking posts or comments. I can’t prove the existence of these elusive people, as there is no way of knowing who has awarded a Like, but I think it is a reasonable assumption based on my experience of Social Media.

Base Figures

    • Known Users (Week 1 and Hello thread) = 240
    • Total number of Posts = 514
    • Total number of replies = 74
    • Total number of Likes = 503
    • Users who posted at least once during Week 1 = 183 (76%)
    • Users who received replies = 51 (21%)
    • Users who left replies = 47 (19%)
    • Users who received Likes = 123 (51%)
    • Users who replied but never posted = 6 (3%)
    • Users who’s posts never received Likes or replies = 59 (25%)
    • Users who received replies but never replied to others (except on their own posts) = 38 (16%)
    • Users who gave replies to others but never received any for their posts = 28 (12%)

These figures tell a story. While most known users (76%) posted in Week 1, relatively few replies where made, with most interaction taking place through Likes.

I decided to drill a bit further into the data to see if there where any patterns. For instance, are posts, replies and Likes spread evenly through the Known Users set, or are some students more engaged in forum activities than others? It could be that we have a small group of highly engaged students who largely spend their time on the MOOC Posting, replying to and Liking each other’s contributions, with a larger group showing some engagement and then the pool of Lurkers?

Analysis of Post numbers

Looking at how many posts each Known User created.

As we can see, the majority of people made relatively few posts. 90% of students posted 5 times or less across the 12 topics looked at. So, 205 of the 514 total posts (40%) of the posts where made by 25 people (10%). This tends to support the idea that there is a core group of students who are more heavily engaged with the MOOC and that the emerging community may be driven by the comments made by this group of “influencers” who have a lot of control over the direction of conversation.

Analysis of replies and Likes

For this analysis I thought it would be useful to look at not just the number of replies each person received, but also how many they received per post. This should give us an idea of how successful our “influencers” are in controlling the conversation. A similar analysis of Likes was also carried out as this could be a better measure of how many people agree with the posts made by each person, while replies would tend to examine people disagreeing.

Raw data graph on number of replies per post
Raw data graph of number of Likes per Post

As we can see there are a couple of peaks in these graphs. Looking at the posts in question, one was a post that was “pinned” to the top of a thread by the course leaders. Since the posts are usually displayed in order of most recent, this blip could be caused simply by exposure, or it could be caused by the post being particularly insightful (which would reasonably be why it was pinned in the first place). The smaller anomaly was a post referencing a particular source of potential funding for projects, so it was naturally of great interest to the learners from within the NHS who would need to fund any AI applications that they intend to develop. I’ve reproduced the two graphs without these peaks so that we can look more closely at the trends.

Replies per post with anomalies removed
Likes per post with anomalies removed

As we can see, other than a small peak further along, the number of replies per post is relatively small and concentrated to those people who post often. However, the pattern for Likes is more uniform with those who post less frequently receiving a similar number of Likes per post to those who post often. In fact there seems to be a trend where those who post less frequently receive more Likes per post. There could be several reasons for this. Perhaps the people who post often are posing questions. Perhaps their views are contentious. Perhaps those within our proposed “influencer” group comment on each other’s posts, seeking them out to add to. Without considerable data gathering it is not possible to see which of these is correct, or if it is a combination. What we can see is that students are clearly reading the contributions left by their peers (remember, we don’t know if Likes definitely come from those who post, or if they also come from Lurkers). Not only can we see that students are reading a lot of the posts, but we can see that they are slightly more inclined to Like posts from those who are less prolific. There could be many reasons for this. Perhaps those who post less often only do so when they have a good point to make. Perhaps the usual way that posts are presented chronologically means that some posts get more exposure (on the front page) during times when more people are using the MOOC. Again, further data would need to be collected to arrive at any solid conclusions.

Conclusions

So what can I conclude from my examination of Week 1? It would seem that the MOOC has a large group of students who are engaged in the threads (although we can’t know this as a percentage of the number of users on the MOOC as we don’t know the total number of users). Within this group, there is a proportion (10%) who are heavily engaged in the threads (posting often and receiving replies and Likes in line with the number of posts). There is a larger group on the fringes of this community who involve themselves more rarely, but who’s contributions are either equally or slightly more highly regarded by the majority of users. The reason why we have this split is not clear, it could be down to personality type, experience of other online learning and MOOCs, familiarity with other online communities (for example social media), or it could be due to their reasons for attending the MOOC – it was designed for use by those within the NHS, and is very focussed on that organisation. Those attending who work for the NHS will naturally have more of an opinion on how the subject matter relates to their role, and may have insights that those attending the MOOC for personal interest do not. It would be interesting to see how these patterns play out in later weeks of the MOOC, or to see how they compare to a different type of digital course. It would also be interesting to see if there is any pattern in community engagement based on the background of the students (some data was collected on those users who are part of an NHS organisation or who have a medical background from the Hello thread before Week 1). I think that given the time constraints and the hours that it took to collect the data set so far, I will follow up on the last one of these ideas and look at patterns based on student history.

Week 5 and a bit of Week 6 – How on earth can you measure “Community” (or “Engagement” in general)?

Could a Mind Meld be possible over the internet?

These are pretty key questions if we’re putting together a Microethnography to look at a “Community” of people on a MOOC. I suppose we could look at community in terms of interaction. If we imagine a scale between 0 = no interaction between Learners (for example a MOOC with no way for students to communicate) up to say 10 = some sort of Vulcan Mind Meld between all the students, then we could look at the features that make important landmarks along that scale (students can post, students can reply, peer review etc). That would be one way to judge “Community”, but it would still be a long way from being an objective measure (for example, would we consider an active forum on an LMS to be superior or inferior to student discussion taking place on Twitter or Discord? What are we basing this on?).

Learners are mostly separated and working individually with the “content”.

The MOOC that I’m taking part in for this assignment is quite limited in terms of possible interaction between users. The course is broken up into weeks, with a number of topics in each week. Each topic then has some “content” (text, images, video, links etc) and a single discussion thread where users can post their thoughts and reply to those posts (for a while I couldn’t see replies as being connected to the initial posts, but this turned out to be a setting that I had unwittingly changed). It is also possible for students to “Like” posts and replies. However, there are no guidelines about how to use the discussion part of each topic, so mostly they’re just a set of individual musings without any connection to each other.

To try and get a sense of the Community on my MOOC I’m going to measure 3 things. Firstly the number of individual posts made on each topic, by each known user. Then I’ll look at how many replies are made, by whom and on who’s posts. I will also look at the number of “Likes” each person receives and map this to each topic as well.

So much data to gather and sort. Perhaps I need to develop some cyborg parts to cope?

This is going to take some time and a great big Excel file with multiple sheets 🙂 To stop the whole thing from becoming unmanageable I’m going to restrict myself to collecting data on the Week 1 topics only. I’m also going to restrict myself to posts and replies made during the period 9th Feb to 15th Feb (the first week of the course). This is a MOOC, and people join after the first week. There would be a problem with constantly having to update my data if I don’t set a specific snap-shot of time.

Who knows or dares to dream what treasures we might find in this uncharted territory?

If this method gives me interesting insights I can then return to the MOOC and either map a different week, or compare Week 1 as it was on 15th Feb with how it looks further down the line. Both of these could be potentially interesting as the first option would show how the course settles down (presuming that a lot of people who drop out do so after the first week) while the second option could show differences between people starting a MOOC on the start date and those catching up (for instance do people playing catch up seem to post more or less? How far back do they go to leave replies – is it just the first page or do they read the lot?).

Week 4 – MOOC decisions and overview

Slightly delayed post as the first day for the MOOC I chose was today.

For our Micro-ethnography task I’ve chosen a MOOC that looks like it will tie in not only with the EDC course itself, but also my work role and personal interests, so triple the goodness 🙂

This is the MOOC

The title of the MOOC is: AI for Healthcare: Equipping the workforce for digital transformation it is on FutureLearn and is written and lead by a team from The University of Manchester and Health Education England (HEE). The MOOC is free for anyone, but is largely aimed at an NHS audience.

The initial look is quite appealing, but I’m not particularly convinced by the format for the first week. It looks a bit disjointed, and the discussion seems to be one long thread on each page, which makes it difficult to keep up. Admittedly this is only day 1 of the course, so maybe all I can expect is a set of randomly fired out comments from some of the students. A lot of comments so far quite definitely instrumentalist in their outlook, and also quite a lot of people happy to Black Box the whole subject area.

Who knows or dares to dream what treasures we might find in this uncharted territory?

I have my butterfly net, sandwiches wrapped in foil and a flask of weak lemon squash for my expedition…