SHEILA workshop at EC-TEL 2018

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Developing evidence-based
institutional learning analytics
policy
13th European Conference on Technology Enhanced Learning...

Schedule
• 9.00-10.45 SHEILA framework + Policy development
• 10.45-11.15 Break
• 11.15-12.45 Policy development
http://sh...

SHEILA project overview &
Senior managers’ views
Yi-Shan Tsai
University of Edinburgh
yi-shan.tsai@ed.ac.uk
@yi_shan_tsai
...

Supporting Higher Education to
Integrate Learning Analytics
http://sheilaproject.eu/

Objectives
• The state of the art
• Direct engagement with key stakeholders
• A comprehensive policy framework
http://shei...

Inclusive adoption process
Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative an...

Methodology
Literature
- Policy
- Adoption
Academic staff
- Survey
- Focus groups
Students
- Survey
- Focus groups
Senior ...

Methodology
Literature
- Policy
- Adoption
Academic staff
- Survey
- Focus groups
Students
- Survey
- Focus groups
Senior ...

Adoption challenge
Leadership for strategic
implementation & monitoring
Tsai, Y. S., & Gasevic, D. (2017). Learning analyt...

Adoption challenge
Equal engagement with
different stakeholders
Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in h...

Adoption challenge
Training to cultivate data literacy
among primary stakeholders
Tsai, Y. S., & Gasevic, D. (2017). Learn...

Adoption challenge
Policies for learning analytics practice
Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in highe...

Methodology
Literature
- Policy
- Adoption
Academic staff
- Survey
- Focus groups
Students
- Survey
- Focus groups
Senior ...

What is the state of the art?
What are the drivers?
What are the challenges?

Survey
• 22 countries, 46 institutions
• November 2016
NO P LA NS
IN P RE P A RA TION
IMP LE ME NT ED 2 13
15
16
The adopt...

Interviews
• 16 countries, 51 HEIs, 64 interviews, 78 participants
• August 2016 - January 2017
N O P L A N S
I N P R E P ...

Motivations to adopt learning analytics
• To improve student learning performance – 40 (87%)
• To improve student satisfac...

Motivations to adopt learning analytics
• To improve student learning performance – 40 (87%)
• To improve student satisfac...

Motivations to adopt learning analytics
• To improve student learning performance – 40 (87%)
• To improve student satisfac...

Why learning analytics?
LA
Learner
driver
Teaching
driver
Institutional
driver
Self-regulation
Learning support
Performance

“People are thinking about learning analytics as a way
to try and personalise education and enhance
education. And actuall...

“I think what we would be looking at is how do we
evolve the way we teach to provide better learning
outcomes for the stud...

“We’re trying to understand better the curriculum that
needs to be offered for the students in our region.
And…I think imp...

Barriers to the success of learning analytics
• Analytics expertise – 34 (76%)
• A data-driven culture at the institution ...

Ethical and privacy concerns
access transparency anonymity

Implications
• Interests were high but experiences were premature.
• There was strong motivation in increasing institution...

Methodology
Literature
- Policy
- Adoption
Academic staff
- Survey
- Focus groups
Students
- Survey
- Focus groups
Senior ...

Staff Survey

With regards to learning analytics …
… what do academic staff ideally expect to happen?
… what do academic staff predict t...

4 academic institutions
University of Edinburgh Carlos III Madrid
n = 81 n = 26
Open Universiteit University of Tallinn
n ...

16 items, some examples
The university will provide me with guidance on how to access LA
about my students
The LA service ...

University of Edinburgh:
• Ideal: LA will collect and present data that is accurate (M = 5.91) Q9
• Predicted: Providing g...

Highest expectation values
Open Universiteit Nederland:
• Ideal: LA will collect and present data that is accurate (M = 6....

Lowest expectation values
University of Edinburgh:
• Ideal: Teaching staff will have an obligation to act if students are ...

Lowest expectation values
Open Universiteit Nederland:
• Ideal: Teaching staff will have an obligation to act if students ...

Staff focus groups

Goal
To better understand the viewpoints of academic staff on:
• Learning analytics opportunities in the HEIs from the
per...

Study participants
• University of Edinburgh: 5 focus groups, 18 teaching staff
• Universidad Carlos III de Madrid: 4 focu...

Results: Expectations & LA opportunities
STUDENT
LEVEL
TEACHER
LEVEL
PROGRAM
LEVEL
Take responsibility for their
learning ...

Results: Meaningful data

Results: concerns –
student level
https://www.pinterest.com/pin/432486370448743887/

Results: concerns –
teacher level
http://create-learning.com
https://www.pinterest.com/pin/432486370448743887/
Http://meme...

Results: concerns – program level
• Interpretation of learning:
• Was the right data collected?
• Were the accurate algori...

What we should consider?
• LA should be just one component of many for collecting
feedback and enhancing decision-making
•...

What we should consider?
•Design of the tools that are:
•Easy to use
•Providing visualizations of data
•Not requiring math...

Student Views
Pedro Manuel Moreno Marcos
Department of Telematics Engineering
Universidad Carlos III de Madrid
pemoreno@it...

Methodology
Literature
- Policy
- Adoption
Academic staff
- Survey
- Focus groups
Students
- Survey
- Focus groups
Senior ...

Student survey results
http://sheilaproject.eu/

Background
• 12 Items Survey
• Two Subscales:
• Ethical and Privacy Expectations
• Service Expectations
• 6 Distributions:...

Ideal Expectation Scale Predicted Expectation Scale
Alternative Purpose Consent to Collect Identifiable Data Keep Data Sec...

Keep Data Secure – Predicted Expectation Scale
Blanchardstown
Edinburgh
Liverpool
Madrid
Open University of the Netherland...

Blanchardstown
Edinburgh
Liverpool
Madrid
Open University of the Netherlands
Tallinn
Strongly Disagree Disagree Somewhat D...

Ideal Expectation Scale Predicted Expectation Scale
ObligationtoAct
IntegrateintoFeedback
SkillDevelopment
RegularlyUpdate...

Blanchardstown
Edinburgh
Liverpool
Madrid
Open University of the Netherlands
Tallinn
Strongly Disagree Disagree Somewhat D...

Blanchardstown
Edinburgh
Liverpool
Madrid
Open University of the Netherlands
Tallinn
Strongly Disagree Disagree Somewhat D...

Summary
• Beliefs towards learning analytics are not consistent.
• Emphasis on data security and improving learning.
http:...

Student focus groups
http://sheilaproject.eu/

Background
• 18 focus groups
• 4 partners’ institutions
• 74 students
• Interviews: Around 1h
http://sheilaproject.eu/

Interests and expectations
• Improve the quality of teaching
• Better student-teacher feedback
• Better academic resources...

Awareness
• Students do not know what LA is, but they recognise its importance if
it can solve students’ problems
• Studen...

Concerns
http://sheilaproject.eu/
Surveillance Anonymization
Purpose of
data
Kind of data
Consent and
access
Security
Prov...

Methodology
Literature
- Policy
- Adoption
Academic staff
- Survey
- Focus groups
Students
- Survey
- Focus groups
Senior ...

Group Concept Mapping
Dr. Maren Scheffel
Open University Netherlands

• innovations in way network is delivered
• (investigate) corporate/structural alignment
• assist in the development of no...

Onderwerp via >Beeld >Koptekst en voettekst Pagina 68
27 March 2014@HDrachsler 68 / 31
An essential feature of a higher ed...

Online sorting
@HDrachsler 27 March 2014 69 / 31
Group Concept Mapping

Online rating
@HDrachsler 27 March 2014 70 / 31
Group Concept Mapping

Participants

Participants

Point Map
1 2
3
4 5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39 40
4...

Cluster Replay Map

Cluster Replay Map

Cluster Replay Map

Cluster Map
1. privacy & transparency
2. roles & responsibilities
(of all stakeholders)
3. objectives of LA
(learner and t...

Rating Map – Importance
1. privacy & transparency
2. roles & responsibilities
(of all stakeholders)
3. objectives of LA
(l...

Rating Map – Ease
1. privacy & transparency
2. roles & responsibilities
(of all stakeholders)
3. objectives of LA
(learner...

Rating Ladder Graph
importance ease
privacy & transparency
privacy & transparency
risks & challenges
risks & challenges
ro...

Go Zone – Roles & Responsibilities
5
38
62
11
19
22
33
39 48
70
91
25
28
37
40
55
61
66
27
47 49
6.08
4.72
3.12
ease
3.83 ...

Yi-Shan Tsai, Pedro Manuel
Moreno-Marcos, Ioana Jivet,
Maren Scheffel, Kairit Tammets,
Kaire Kollom, and Dragan
Gašević. (...

Methodology
Literature
- Policy
- Adoption
Academic staff
- Survey
- Focus groups
Students
- Survey
- Focus groups
Senior ...

SHEILA framework

SHEILA policy framework

Methodology
Literature
- Policy
- Adoption
Academic staff
- Survey
- Focus groups
Students
- Survey
- Focus groups
Senior ...

Share SlideShare

Facebook
Twitter
LinkedIn

Embed

Size (px)

Show related SlideShares at end

WordPress Shortcode

Link

Share
Email

ChatGPT and the Future of Work - Cl... by Clark Boyd 38120 views
Getting into the tech field. what n... by Tessa Mero 9541 views
Google's Just Not That Into You: Un... by Lily Ray 8467 views
How to have difficult conversations by Rajiv Jayarajah, ... 7433 views
Introduction to Data Science by Christy Abraham Joy 83577 views
Time Management & Productivity - B... by Vit Horky 169999 views

Top clipped slide

1 of 86 Ad

1 of 86 Ad