Monday, November 20, 2017

Using data (analytics) to inform (e)Teaching and (e)Learning 15th APMEC main conference symposium

Main conference symposium

Poh-Sun Goh (NUS YLL SoM)
Sergio Hernandez-Marin (Google)
Lim Wee Khee (NUS ISS)

Some ideas which will be discussed by the panel at the symposium:

"Learning analytics (LA) can be combined with data from Assessment (both formative and summative), faculty and peer/student observations and feedback, to inform both eTeaching (for faculty), and (e)Learning (for students). LA can provide insight and visualisation of the learning journey undertaken by students and trainees, from attendance, through engagement, (active) interaction with the learning content and training process, as well as collaborative learning with fellow students and trainees, toward step-wise and progressive achievement of learning milestones, learning objectives and outcomes."
- Poh Sun
(each and every highlighted word is hyperlinked to an online illustration/resource)

"I’ll like to share with the participants from the insider perspective of a commercial organisation, on how analytics are used to influence business operations and customer experience.
And what are some of the key success factors in adopting analytics."
-Wee Khee

Additional material presented at 15th APMEC pre-conference LA workshop, as well as presented and developed through interactive discussion with the audience and panelists on a dedicated Padlet wall.


Title
Using data (analytics) to inform eTeaching and eLearning 
Goh Poh Sun (GPS), Sergio Hernandez-Marin (SHM), Lim Wee Khee (LWK)

Brief description of session
The central thesis of this symposium will be to demonstrate and illustrate how data (analytics) can inform an educational practitioner (in their educational practice), and anchor scholarly teaching and educational scholarship.

The three panelists bring (to the audience) long standing practical frontline and academic experience in their roles as a clinical teacher and in medical education faculty development (GPS), in data analytics and strategic thinking (SHM), and in the real world application of social media analytics in business (LWK).

This interactive symposium and panel discussion (with the audience) will be anchored by a purpose built dedicated presentation online blog (for approximately 30 minutes to one hour of pre-session online review BEFORE the session), each symposium presenter giving a (maximum) 10 minute short overview presentation, followed by ONE HOUR of interactive live discussion (during the session) with the audience (who should bring a WiFi enabled mobile phone, Tablet or Laptop to the session to engage with the panelists - both synchronously and asynchronously, the day before, during, and day after the symposium; via a dedicated Padlet wall).


NOTE REGARDING PADLET:
We will use Padlet throughout the symposium.
* Padlet is an open access online writing surface. We will use it before and during the symposium.
* Anyone with a link can write on the Padlet wall. You do not need to sign in, but we ask that you please identify yourself because it makes for a more meaningful conversation.
* As long as you continue to use the same device, you can edit, add to, or delete your earlier postings. If you use a different device, you will not be able to edit your previous comments.
* The Padlet wall is organised so that the latest post is always on the top.


Made with Padlet

https://padlet.com/dnrgohps/analyticsMedEdsymposium

https://medicaleducationelearning.blogspot.sg/2017/09/see-one-do-one-teach-one-to-see-show-do.html

https://medicaleducationelearning.blogspot.sg/2017/09/learning-is-long-lasting-change-in.html


"We can manage better (to iteratively improve, share and build on) what is visible, what we can see – directly, through data, and data dashboards-visual data maps and illustrations. Data and observations, big data and small or rich data, quantitative and qualitative research (mixed methods research) gives us insights as educators into our teaching practice, its effectiveness and impact. Just as we blend the best features of traditional and online/eLearning/Technology enhanced learning in our teaching practice, we can “blend” and take advantage of “big data” or online data analytics, which when added to traditional classroom observations and measures-indicators of learning effectiveness and impact, can give us a more complete, comprehensive, and rounded picture of individual, and group learning."

above from
Goh, P.S. Learning Analytics in Medical Education. MedEdPublish. 2017 Apr; 6(2), Paper No:5. Epub 2017 Apr 4 https://doi.org/10.15694/mep.2017.000067
https://www.mededpublish.org/manuscripts/944/v1


"Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. The adoption of learning analytics too must be informed not only by what can be measured but also by what cannot. There will be limits in what learning analytics can do. In this vein, Siemens and Long have appropriately acknowledged that learning "is messy" and have warned that with learning analytics, "we must guard against drawing conclusions about learning processes based on questionable assumptions that misapply simple models to a complex challenge."5 The message here is important: not every aspect of learning can be captured by the powerful tool that analytics promises to be. Sometimes learning is ineffable! Therefore, multiple methods for assessing learning should be employed, including assessments that function as learning opportunities to support students' deep integration of knowledge, their personal development, and (hopefully!) their transformation over time."
and
"Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change. As this principle states, assessment alone changes very little; likewise, learning analytics cannot act alone in radically disrupting and transforming education. Assessment (when done well) is about the authentic and deep understanding and improvement of teaching and learning. Analytics is about using the power of information technology to see patterns of success (or failure) in learning. Combining the two might actually produce the seeds of transformation—a powerful inquiry into what supports authentic, deep, transformative learning for students."
above quotes from
https://er.educause.edu/articles/2012/7/learning-analytics-the-new-black


Another important developing trend ... is “online merging with offline” (OMO) – a trend that, along with AI, Sinovation Ventures is betting on. The physical world becomes digitized, with companies detecting a person’s location, movements, and identity, and then transmitting the data so that it can help shape online experiences ... (for example) OMO language learning will combine native teachers lecturing remotely, local assistants keeping the atmosphere fun, autonomous software correcting pronunciation, and autonomous hardware grading homework and tests.
above quotes from
https://www.project-syndicate.org/commentary/china-digital-age-innovation-by-kai-fu-lee-and-jonathan-woetzel-2017-12


https://datainformedelearning.blogspot.sg/2017/11/data-analytics-informed-eteaching-and.html









Seek-See-Do-Teach (Instagram)



(e)Learning in MedEd (Instagram)

e Learning, eLearning, (e)Learning or teL (Instagram)

teL with Learning as the objective (Instagram)

Why before How (Instagram)

Blending/Blurring Continuum (Instagram)

See(1), Do(1), Teach(1) (Instagram)

See, Do, Teach (elaborated)

Teacher as Producer, Editor, Curator, DJ, Mixer (Instagram)


3. Time on task

"It takes time to learn anything worthwhile. To accumulate knowledge and skills. To integrate this new learning, and be able to, and be confident applying this in the workplace, and real life settings.
This is the difference between undertaking a program of training, and formal courses, compared with short symposia and workshops, or an isolated lecture. Formal training programs gives students time, space, and a place to learn. On a regular basis. This promotes a cumulative increase in learning. Combining theoretical learning with practical case studies integrates basic principles with practice points, and promotes transfer of learning from the classroom to the real world. Online learning programs should include elements from traditional classroom practices which facilitate learning. This includes scheduled time to review the learning material, to work on applying what is learnt by working on assessments and assignments, individually and by learning collaboratively with peers; as well as provide opportunities for timely feedback from instructors."

above from
Goh, P.S. A series of reflections on eLearning, traditional and blended learning. MedEdPublish. 2016 Oct; 5(3), Paper No:19. Epub 2016 Oct 14.

Hence the questions which are often asked when inquiring about the background and training of a prospective employee/trainee:
-Educational and training background?
-Duration?
-Where?
-With whom?
-Doing what? (curriculum)
-Know, know how, show how, does
-Tell me about...
-What do you think about...
-What would you do when...
-Show me...
-Demonstrate how you would...
-Let's see how this candidate...


11. The motivation of your learners, their interest/attention/engagement is key
-Why before How

"Effective learning, and deep learning require interest in a topic, a commitment to learning by our students, regular and systematic participation in an educational process and program, with active engagement with the educational content and the training process. Effective teaching requires awareness of the educational background and prior experience of our students, efforts to maintain and increase motivation in our students, careful and knowledgeable selection of instructional content, and use of interactive and active learning exercises, sequenced and built into a systematic and programmatic educational and training process. Educational technology can anchor and facilitate both effective learning and teaching."

above from
Goh PS. eLearning or technology enhanced learning in medical education - Hope, not hype. Med Teach. 2016 Sep; 38(9): 957-958, Epub 2016 Mar 16
http://www.ncbi.nlm.nih.gov/pubmed/26982639

above from

https://medicaleducationelearning.blogspot.sg/2017/01/12-ideas-for-2017.html


Posted by at 1 comment:

How to use Data (analytics) to inform (e)Teaching and (e)Learning 15th APMEC pre-conference workshop

Pre-conference workshop (venue, MD6, SMART classroom, level 4)

"Dear workshop participants, 

Thank you for signing up for the workshop "How to use Data (analytics) to inform (e)Teaching and (e)Learning" at 15th APMEC.

We have prepared a dedicated workshop blog to complement the workshop. Please visit the workshop website below, and do preparation for the workshop (will take around 1 to 2 hours), BEFORE the workshop. This includes sharing your background, reason for attending workshop, and reaction to the pre-reading on the Padlet online space. This will not only allow you achieve the objectives of the workshop, to be able to THINK ABOUT (and have an informed discussion with colleagues on this topic), and to DO (this topic), by starting a personal, teaching blog with analytics; this will also form the foundation for within workshop activities and discussions, and allow the workshop faculty to further customize the content of the workshop to meet your specific needs. For workshop participants who post on the workshop Padlet space by Monday, 8 January 2017 BEFORE the workshop, the faculty will endeavour to specifically curate and customise material in the workshop for you.

https://datainformedelearning.blogspot.sg/2017/11/how-to-use-data-analytics-to-inform.html
(for pre-conference workshop)

With warm regards,
Poh-Sun
Sergio
Wee Khee"


PRE-WORKSHOP ACTIVITY

1. Read the following articles

Goh, P.S. Learning Analytics in Medical Education. MedEdPublish. 2017 Apr; 6(2), Paper No:5. Epub 2017 Apr 4. https://doi.org/10.15694/mep.2017.000067

https://er.educause.edu/articles/2012/7/learning-analytics-the-new-black

https://cloudplatform.googleblog.com/2013/11/kaplan-builds-online-education-platform-kapx-with-google-app-engine.html

https://edu.google.com/latest-news/stories/strayer-university-gcp/?modal_active=none

https://www.cio.com/article/3153389/it-industry/how-small-data-became-bigger-than-big-data.html

https://www.mckinsey.com/business-functions/operations/our-insights/the-ceo-guide-to-customer-experience


2. Respond to the following questions in the Padlet spaces below, BEFORE the workshop.
* Describe in 1 sentence who you are, and why you are coming to this workshop.
Please include your name.

[Padlet space #1]

* Describe in 1 sentence your reaction to the pre-reading assignment.
Please identify this posting with your name or initials.

[Padlet space #2]


NOTE REGARDING PADLET:
We will use Padlet throughout the workshop.
* Padlet is an open access online writing surface. We will use it before and during the workshop.
* Anyone with a link can write on the Padlet wall. You do not need to sign in, but we ask that you please identify yourself because it makes for a more meaningful conversation.
* As long as you continue to use the same device, you can edit, add to, or delete your earlier postings. If you use a different device, you will not be able to edit your previous comments.
* The Padlet wall is organised so that the latest post is always on the top.


3. Before coming to the workshop
* please set up a personal teaching Blogger website; and review basics of Google Analytics, and explore Google Analytics blog and YouTube channel. Please bring along a presentation you have given recently, or intend to give in the near future (in both Powerpoint, PDF and JPEG format with individual slides of the presentation, with each slide ready to be shared online, i.e content produced by you, or used with attribution or permission).


______________


Title
How to use data (analytics) to inform eTeaching and eLearning Goh Poh Sun (GPS), Sergio Hernandez-Marin (SHM), Lim Wee Khee (LWK)

Objectives
To illustrate and demonstrate the utility of off the shelf/free(ly available) data analytics to inform eTeaching and give visibility of eLearning (activities) by our students.

Workshop description
The workshop will be based on actual case studies from an experienced medical educator (GPS), who has been using Google Blogger (with in built data analytics), exclusively (rather than PowerPoint) for clinical teaching, and medical education faculty development over the last 6 years (in undergraduate, postgraduate and CME/CPD settings). Co-facilitators in the workshop (SHM and LWK) will share added insights from a technical-strategic (SHM) and market-engagement (LWK) perspective. Participants will have the opportunity to build their own prototype teaching blog (with use of Google Blogger as an illustrative freely available, and free to use platform), together with seeing how embedding additional online tools into a teaching blog (like Slideshare, SurveyMonkey, and Padlet) can give educators further data and visibility of student engagement, and actual learning within an eLearning process and platform. Participants will be expected to have engaged in one to two hours of pre-reading, and a pre-workshop exercise. Participants should bring a WiFi enabled laptop or tablet computer to the workshop.

Who should attend
Health professions educators (Medical, Nursing and Allied Health), and staff who have an administrative and leadership role in supporting and working with eLearning/Technology enhanced learning teams.

________________

Session Plan

1.30pm to 2pm         Introduction (GPS), including discussion of outcome(s) for session, see 0430pm to 5pm below, followed by short presentations (SG, WK, GPS)

2pm to 3pm              Activity 1 (Discuss in a group WHY use learning analytics, then present)

3pm to 3.30pm         Tea break

3.30pm to 4.30pm    Activity 2 (Discuss in group HOW to use learning analytics, in own teaching context)

4.30pm to 5pm         Individual reflection, and Padlet post (What did I learn from workshop-key ideas; and how will I use ideas presented and discussed in my own context; then post on Padlet)

5pm to 5.30pm         Closing remarks 

__________________

Take Home Points

Poh-Sun: for participants to be able to THINK ABOUT (and have an informed discussion with colleagues on this topic), and to DO (this topic), by starting a personal, teaching blog with analytics.
(each and every highlighted word is hyperlinked to an online illustration/resource)

Sergio: I would like participants to understand what's big data, why big data is important and when big data is not enough.

Wee Khee: At the end of my sharing, participants should be able to understand how an e-commerce business organisation use different types of analytics to drive business operations.

The faculty would like participants to reflect on how examples from their experience with the web, media consumption, travel planning, F and B, retail, and elaborated by Sergio and Wee Khee can translate and transfer to their teaching and learning practice. 


"The intent of bringing in examples and the perspective from Google, and business analytics is the much earlier adoption and experience of the use of data in these fields compared with (medical) education. We are clearly late adopters, and have much to learn." 
Poh-Sun


and lastly

Bite or Byte-size Ideas from the workshop facilitators

7 byte-size ideas from Poh-Sun
https://www.slideshare.net/dnrgohps/data-analytics-informed-eteaching-and-elearning-7-byte-size-ideas

and
5 (more) short takeaways from Poh-Sun

1 - Arguably the aim (outcome) of any learning session is to be able to THINK about (have a conversation), DO, and have a FEEL (opinion) about something, at the end of the educational and training process. Or gain Knowledge, Skills, and (form or change) Attitudes (KSA).
For this topic, when starting out, the material and terminology will feel unfamiliar at the beginning. With further reading, review of more material, repeatedly coming back to the topic, discussing and working through the ideas with other participants, colleagues and collaborators; starting and continuing to USE these ideas, you will progressively increase insight, understanding and skill in this area. Keep going. One step at a time. Iteratively. The more one thinks about, and uses ideas and strategies, the more one becomes familiar with, and confident in a topic, or area of professional practice.
2 - Start with the your goal or objective as a teacher or educator. What is your learning objective(s)? How will you know this is (being) achieved? What data, evidence, and observations inform you (that this has been accomplished)?
3 - Increasingly education, learning and training will be technology mediated, technology facilitated, and technology enhanced. Take advantage of your delivery and engagement platform and process to gather this data and evidence. In real time. To inform, and further customise your educational offering. Similar to what you are familiar with in a traditional classroom and training setting.
4 - Customisation and personalisation (personal learning) may simply require offering a "buffet" of (digital) content, and training scenarios, to suit individual preference and choice, for a specific purpose, at a specific time. Or use a "well stocked" digital repository for instructor facilitated, and specifically curated teaching and learning.
5 - Technology enhanced Learning (TeL) makes what we teach with, and assess on visible, accessible, and assessable. We can "show what we teach with, and assess on". For students, trainees, peers, colleagues, administrators. In fact all potential stakeholders. To make this (data) available as evidence of scholarly activity, and educational scholarship (the Scholarship of Teaching and Learning, or SOTL).
           

3 byte-size ideas from Sergio

1 - Big data is as good as the analysis we perform on it.
2 - Big data and small data are like partners in a dance. They both need to be considered to generate insights
3 - Not all projects require machine learning.


4 byte-size ideas from Wee Khee

1.    Begin with the End in Mind - Set goals before considering analytics and measurement
2.    Design with learners at heart – Use human-centred design principles in learning journey design.
3.    Test, Measure and Experiment – Use analytics to guide experimentation 
4.    Take actions with insights – analytics are only useful if you can make decisions and changes

__________________

(key slides Poh Sun will be using - with review of key areas in the workshop , including- 1) when and why to use learning analytics (LA) (vs academic analytics-see slide 12); focus on needs & objectives, not tools & tech, start with what you have, and what you need; 2) some basic principles underpinning thinking about, and using data analytics  -- Poh Sun will present exclusively from two online platforms, which can be used for Technology enhanced learning(TEL) / eLearning or mLearning / mobile learning (Blogger and Instagram), and will demonstrate the utility of both as presentation and curatorial tools; by blending online with classroom interaction); including sharing use in large group teaching (for undergraduate radiology and medical education faculty development), small group teaching / individual learning (for undergraduate radiology, postgraduate radiology, and medical education faculty development).


flow of ideas from presentation by Sergio will be as follows:
using very short case studies (including from pre-readings) of data analytics and machine intelligence for education, to illustrate what is big data, why big data is important, and when big data is not enough.


flow of ideas (sections) from presentation by Wee Khee will be as follows:
B2B Ecommerce business story – Business transformation from 80-year old traditional catalogue business to multi-channel operations.  Using & leveraging analytics for business operations. 

 Examples of Analytics used in business operations
Web analytics (web data, performance tracking, content, internal search)
Usability tests
Customer database (CRM) – sales force, business analytics
Customer feedback / NPS (net promoter score)
Social Media Analytics
Further development and illustration of these ideas will take place during the main conference LA symposium.

____________________


Made with Padlet

https://padlet.com/dnrgohps/dataMedEd

https://datainformedelearning.blogspot.sg/2017/11/data-analytics-informed-eteaching-and.html

https://medicaleducationelearning.blogspot.sg/2017/10/requirements-for-data-informed-personal.html

http://medicine.nus.edu.sg/cenmed/apmec15/programs.html






 above and below from
https://www.mededpublish.org/manuscripts/944/v1

above and below from
https://www.mededpublish.org/manuscripts/706/v1


https://www.theguardian.com/media-network/2013/mar/27/behind-paywall-us-newspaper-websites

https://www.theguardian.com/media/2014/aug/03/financial-times-paywall-online-advertisings-subscriptions

"The changes to the charging model will be accompanied by a new focus on audience engagement measured in terms of frequency of visits to the FT, return visits and time spent on page."
above quote from article below
https://www.theguardian.com/media/2015/feb/27/financial-times-to-change-way-it-charges-for-online-content


http://www.dailymail.co.uk/wires/afp/article-5241837/As-Globes-mark-cinemas-successes-audiences-shy-away.html

http://news.abs-cbn.com/entertainment/01/06/18/as-globes-mark-cinemas-successes-audiences-shy-away

http://www.businesstimes.com.sg/life-culture/tinseltown-in-danger-of-losing-its-glitter-as-audience-numbers-fall

https://www.nytimes.com/2018/01/05/business/media/487-original-programs-aired-in-2017.html

http://www.straitstimes.com/lifestyle/entertainment/record-487-original-shows-in-2017


above from
Customer data sharing comes out of the shadows (Financial Times on Twitter)

"Diverse entities that receive customer data from retailers and other businesses are in various ways involved in collecting and organising personal information to build up online profiles — when and where shoppers browse, what they look at and whether they buy something.
Such profiles — further developed by adding other information gleaned from social media sites, apps, search engine queries and other web behaviour — can be sold on to advertising businesses and then used to push targeted products and services at online users."
above quote from
Customer data sharing comes out of the shadows (Financial Times, from Twitter feed)

EU’s General Data Protection Regulation (from Wikipedia)


"...be an expert at something."
from
https://www.nytimes.com/2017/08/24/opinion/sunday/networking-connections-business.html?_r=0
linked to from
https://www.nytimes.com/guides/business/manage-your-career


http://www.bbc.com/future/story/20180104-is-social-media-bad-for-you-the-evidence-and-the-unknowns


"It’s easy to fall down the rabbit hole while reading blogs (except this one of course) or online forums. Stay focused by setting time limits for online activity."
above quote from
https://hbr.org/2009/10/why-the-internet-is-so-distrac

"Read actual books (they smell really good) or use devoted e-readers. If you’re anything like me, your dumb idiot brain often reads a great passage on a phone or tablet and immediately Googles more information on it. Awful move. Guaranteed rabbit hole. So read actual books, or read on devices that can’t Internet real good. Stay focused."
above quote from
https://medium.com/words-for-life/the-problem-with-every-how-to-be-a-genius-post-on-medium-496913a473da

https://collegeinfogeek.com/internet-time-wasting/


“Getting information off the Internet is like taking a drink from a fire hydrant” by Mitchell Kapor.









Goh, P.S. A series of reflections on eLearning, traditional and blended learning. MedEdPublish. 2016 Oct; 5(3), Paper No:19. Epub 2016 Oct 14.
http://dx.doi.org/10.15694/mep.2016.000105


above and below from
https://www.mededpublish.org/manuscripts/613/v1



https://www.moe.gov.sg/news/speeches/opening-address-by-guest-of-honour--minister-for-higher-education-and-skills--mr-ong-ye-kung--at-the-2nd-national-conference-on-technology-enhanced-learning-tel--at-nus--university-town--auditorium-2

http://www.nus.edu.sg/annualreport/pdf/nus-annualreport-2017.pdf
(see page 31, "the NUS Education Data Lake"

https://www.slideshare.net/ALSETNUS/nuss-digital-learning-journey-in-enhancing-teaching-and-learning
(see slide 18, NUS Education Data Lake; and slide 23, Research model: Faculty Involvement and External Partnerships)

https://www.slideshare.net/ALSETNUS/introducing-alset-soc-opportunities-at-nus-for-data-and-intervention-data-dictionary

https://www.slideshare.net/ALSETNUS/ravi-chandran-wesst-nus-technology-enhanced-learning-environment
(see slide 20, titled "Repositories for educational research, analytics and reusable content")

https://www.slideshare.net/ALSETNUS/presentations





























https://medicaleducationelearning.blogspot.sg/2017/03/learning-analytics-in-medical-education.html

https://analytics.google.com/




https://telatcenmed.blogspot.sg/2017/05/technology-enhanced-learning-elearning.html

above from
https://medicaleducationelearning.blogspot.sg/2017/05/cenmed-workshops-in-may-and-june-2017.html


Posted by at 4 comments:
Subscribe to: Posts (Atom)