Learning Analytics Policy Development

June 25, 2018 in communication, data, featured, guestpost

—

The University of Edinburgh has just launched their Principles and Purposes for Learning Analytics.

In order to develop institutional policy on learning analytics, in 2016 we convened a task group reporting to our Senate Learning and Teaching Committee, and our Knowledge Strategy Committee. The task group was convened by Professor Dragan Gasevic, Chair of Learning Analytics and Informatics. The group included Professor Sian Bayne, Assistant Principal Digital Education; representatives from academic Colleges; the Edinburgh University’s Students Association; and representatives from Student Systems and Information Services.

Our Director of Academic Services produced an initial draft of a Learning Analytics policy for review by our institutional task group. It was a relatively detailed policy which covered the following sorts of topics:

Definitions
Sources of data for learning analytics
Sources of data for learning analytics
Initiating learning analytics activities
Transparency and consent
Privacy and access to data
Retention and disposal of data
Validity and interpretation of data
Supporting positive interventions
Enabling students to reflect on their learning
Supporting staff to make the most of learning analytics
Oversight of Learning Analytics activities
Other relevant policies

Ethical values, legal obligations and the reasons for engaging with learning analytics were all embedded in the policy, but as we worked on revisions, considered inputs from external sources, and planned how to consult on a draft it became clear that this detailed policy was likely to beg more questions than it answered without being more explicit about our values and our ethical position upfront. We also had to contend with periods of time where there was limited data protection resource available to the task group, and where the legal basis for processing under GDPR that would be available to us was still being debated in the House of Lords.

At the same time as we were developing local policy, colleagues at Edinburgh (Prof Dragan Gasevic and Dr Yi-Shan Tsai) were involved with the EU Sheila project, developing a learning analytics policy development framework for the EU. There were several key outputs from that project that we used in pre-print form to inform our work:

Executive summary of the literature on LA adoption and policies in other HEIs.
Slides presenting the results of the group concept mapping study from the SHEILA project.
Slides presenting the results of the interviews of senior leaders in over 50 institutions across Europe from the SHEILA project.

In particular, the group concept mapping activity carried out by the Sheila project (surveying various European Universities) identified that defining objectives for learning analytics was very important, but also very hard (http://sheilaproject.eu/wp-content/uploads/2017/04/The-state-of-learning-analytics-in-Europe.pdf). As part of our local policy development, myself and Dragan Gasevic met and discussed what we felt were the 6 main purposes for learning analytics in an Edinburgh context, and these were written up into the policy as a means of tackling this issue head-on for Edinburgh.

The literature review on learning analytics adoption that the Sheila project produced also identified various challenges to adoption, and on further consideration I drafted a separate Purposes and Principles document which extracted various of the principles embedded in the detailed policy and responded to many of the challenges and concerns identified in the literature review. Given some of the challenges we were experiencing around clarity on new data protection legislation for resolving areas the more detailed policy, this was the point at which our task group decided to separate the two pieces and start with a consultation on Purposes and Principles only.

The Purposes and Principles were outlined and discussed at Senate in early 2017 and then taken to each School for discussion as part of the consultation plan that Academic Services devised for us. To support this consultation we also developed a webpage that outlined existing research and operational activities in learning analytics at Edinburgh (https://www.ed.ac.uk/information-services/learning-technology/learning-analytics).

This high-level values-first route proved to be an effective way to start, as consultation with many Schools identified that the level of knowledge and understanding of learning analytics was highly variable across the institution, and that there were significant pockets of concern about ethics and about support for staff and students to make more use of data.

The Sheila project also ran a student survey at Edinburgh during this time period and we were also able to finesse the Principles and Purposes to respond to student concerns and expectations.

In considering how to achieve oversight and governance in the absence of the more detailed policy, and in a potentially quite complex and changing area, we also proposed the establishment of a Learning Analytics Review Group. As we pursue more data-driven operational activities this helps close out an ethical review gap in our operational activities. This governance model is now of interest to colleagues working on institutional data governance activities more generally.

Once the Principles and Purposes were approved, with support from our Data Protection Officer, and more clarity on GDPR we were then able to tidy up the more detailed policy which defines the ‘mechanics’ of how activities can be initiated, what roles and responsibilities exist, what sources of data might be implicated etc. This policy was approved by our Senate Learning and Teaching Committee in May 2018. Importantly, this policy has also been able to link in to other work around data governance within the institution, and formally recognises the role that our institutional ‘Data Stewards’ have to play in the approvals process for learning analytics projects.

Important inputs to the development of policy (as well as the Sheila project inputs) included:

The Open University learning analytics policy (http://www.open.ac.uk/students/charter/essential-documents/ethical-use-student-data-learning-analytics-policy)
Previous involvement in the development of the JISC Code of Practice for Learning Analytics (https://analytics.jiscinvolve.org/wp/2015/06/04/code-of-practice-for-learning-analytics-launched/)
Higher Education Commission “From Bricks to Clicks” report (http://www.policyconnect.org.uk/hec/research/report-bricks-clicks-potential-data-and-analytics-higher-education)
“Learning Analytics: Ethical Issues and Dilemmas” Sharon Slade and Paul Prinsloo (2013) (http://journals.sagepub.com/doi/10.1177/0002764213479366)
Discussions with peers and experts at sector groups like SHEEC
Conversations with Edinburgh Data Protection and Information Security professionals
JISC Effective Learning Analytics Co-Design project (https://www.jisc.ac.uk/rd/projects/effective-learning-analytics)
JISC Learning Analytics Network meetings
Outputs from the EU LACE project, in particular their DELICATE framework for ethics and privacy (http://www.laceproject.eu/ethics-privacy/)
Outputs from the Edinburgh PTAS-funded LARC project (http://www.de.ed.ac.uk/project/learning-analytics-report-card)
Experience from Edinburgh colleagues, notably Prof Gasevic, Dr Yi-Shan Tsai, colleagues in Digital Education, and colleagues in Medicine.
The Task group who helped develop the policy and debate the various drafts.

—

About the author

Anne-Marie Scott is Deputy Director of Learning, Teaching and Web Services, at the University of Edinburgh. Her background is in the design, management and support for academic IT services, particularly those used to support teaching and learning activities online. Amongst her interests are the use of new media and the open web in teaching and learning, scalable online learning platforms, and learning analytics.