Electronic Management of Assessment

 

What is it?

The term electronic management of assessment (EMA) is increasingly being used to describe the way that technology can be used to support the 'management of the whole assessment and feedback lifecycle, including the electronic submission of assignments, marking, feedback and the return of marks and feedback to students'. (Jisc) Electronic submission of assingments refers to the process by which a student submits work online for storage and retrieval by academic, administrative or other staff for marking, feedback or review. This can also include originality checking to help identify plagiarism using Turnitin.Electronic marking is the process by which staff provide marks and feedback to students online (through the Blackboard Grade Centre), which can include online annotations eg using Blackboard inline grading, annotated files eg using comments in Microsoft Word, audio, video, or the use of online forms. EMA at the University of Bristol
 

The Digital Education Office (DEO) have been supporting the use of tecnology to manage submission, marking and feedback for a number of years. This experience has enable both staff in DEO and staff in Schools to improve students and staff experience with EMA as well as developing a flexible support process that meets different requirements. As from the 2017-2018 academic year the Electronic Management of Assessment is part of a wider institution initiative supported by the DEO. The implementation will be phased to enable work with all Schools over three years. More information about the project with a list of Schools currently involved can be found on the EMA project page.
 

How do I start using Blackboard or Turnitin to manage submission of assingments and marking online? 

If your School is already involved in the EMA project please contact your School admin team who will be able to advise on the type of workflow that the School has adopted and what support is available. If your School is not currently involved please email Roger Gardner(Digital Education Development Manager) to discuss your needs. 

Also refer to the guidance on planning electronic submission and marking, and to the University Regulations and Code of Practice for taught Programmes (sections 9.22 to 9.27 of  Section 9: Conduct of Assessment).

 

EMA implementation process for Schools

The EMA implementation process for Schools starts with identifying key roles within the School who will lead on the implementation process. The School EMA implementation team should include member (s) of the academic team (academic lead), member (s) of the adminstrative team (admin lead), member (s) of local education support team if available and student rep if required.  

The EMA implementation process includes following stages:   

  • Scoping (overall rationale / aims, key people and timescales) and agreement of roles and responsibilities


At the scoping stage a member of the DEO team will meet with the School EMA implementation team to gather information behind the rationale for moving to EMA, introduce the process of implementing EMA at a School level and agree roles and responsibilities that will be undertaken by the the School and the DEO office. Schools wishing to work in partnership with the DEO on EMA will be required to agree to these roles and responsibilities in advance.Below is a list of responsibilities undertaken by the DEO and by the School for the implementation of EMA. 

DEO will:


1. Advise on setup and recommended workflows

2. Provide generic support materials for the School to customise

3. Train the admin team on setting up and managing the process

4. Provide up to 2 initial training sessions for markers (thereafter this training will need to becascaded within the School)

5. Where queries cannot be answered in the first instance by the School admin team, the DEO will respond to ongoing queries from staff via the DEO helpdesk

6. Respond to requests to access old archived courses e.g. for references


The School will:


1. Provide academic and administrative leads for the project - these roles will be responsible for ensuring stakeholder requirements in the School are captured, signing off the workflow, championing and driving forward the change to online marking, including communication strategies, physical resources and potential changes to existing processes. These would typically be the School Admin Manager and School Education Director or equivalent.

2. Provide dedicated administrative support - a minimum of 0.5 FTE for the transition period will be required where online marking is introduced. This role will liaise with the DEO on a day-to-day basis, provide project management e.g. arranging planning meetings and training, co-ordinate setup in Blackboard and preparation of support materials for staff and students. The 0.5 FTE could be made up from different roles, for example a day and a half a week for an UG admin and a day for a PG admin

3. Test the workflow and ensure it is set up correctly before submission and marking starts 4. Provide dedicated support to deal with admin and academic staff queries5. Provide first line support to students6. Clear out any existing submission points and associated information in Blackboard courses if appropriate

7. Setup Blackboard courses (parts of the setup can be copied across multiple courses through Blackboard packages)

8. Keep the DEO informed about (any changes to) submission dates

9. Write and update instructions for staff and students

10. Arrange training sessions

11. Cascade any training required after the initial sessions delivered by the TELED team

12. Support the online marking process at each stage as specified in the workflow document

13. Communicate with School staff and students regarding the process

14. Evaluate the student and staff experience of online marking

15. Archive courses when required

Please Note: More detailed information around DEO / School roles and responsibilities is available if a School wants to use Blackboard packages, whereby content is copied or imported across multiple courses.

 

  • Review of current practice to get a better understanding of the current practices in the Schools and the requirements for the implementation of EMA



This review focusses on management of coursework, and related activities. The Code of Practice defines coursework as “any summative assessment based on essays, assignments, creative writing or other tasks that is completed outside timetabled classes in the students' own time.” This review can also cover formative coursework.Please complete the questionnaire prior to the meeting with the DEO. 
  • Requirements gathering and workflow sign off


At this stage the DEO and the School implementation team will work together in the development of a standard workflow that can be applicable across all units. This is essential to ensure that staff and students use a standard process which has been agreed and can be used to scale up the EMA activities in the future. If the standard workflow doesn't fit your requirements contact the DEO to discuss your needs. A typical standard workflow involves the following stages:

 

 

  1. Setup - Instructor creates the submission point
  2. Submission - Students submit their assignments 
  3. Originality check - assignments are put through Turnitin and reports analysed 
  4. Marking - markers add marks and feedback in the Blackboard Grade Centre 
  5. Feedback - students access their own marks and feedback 

Depending on your role in the process you may be involved in some of these stages but not others.

A workflow example is provided in this document. This workflow is for 'individual assignment, fully marked anonymously by multiple markers' using the Blackboard EMA functionalities.

 

  • Training


The DEO can offer training for administrative staff and markers. This is highly recommended as it provides an overview of the chosen workflow and gives staff the opportunity to clarify any doubts about the functionalities or the process. Training requests should be forwarded by the administrative or academic lead to digital-education@bristol.ac.uk. Depending on the number and the requirements of the attendees training can be delivered face to face or online.It is recommended to provide students and staff with detailed instructions about the workflow steps. The DEO can provide standard guidance for admin staff, markers and students. Staff in Schools are welcome to use it for their own activities or adapt them to meet individual requirements.

 

 

(to go on the side bar)
Downloadable guides:
Blackboard administrative guide
Blackboard marker guide
Blackboard student guide
Turnitin administrative guide
Turnitin marker guide
Turnitin student guide



  • Setup

At this stage staff in School will be responsible for setting up the EMA activities on the Blackboard courses. The DEO can advise about options to automate some of the steps of the process such as populating courses with instructions for staff and students if these are exactly the same in all units.
  • Implementation

The implementation stage involves submission of assignment, marking and feedback, moderation, and release of marks. Staff in School will be the first point of contact for both staff and students while DEO can help with troubleshooting unexpected issue with the technology
  • Ongoing support

DEO will provide ongoing support for the EMA activities via the digital education helpdesk digital-education@bristol.ac.uk
  • Evaluation

This stage involves reviewing how the activities went, any feedback from staff and students and preparation for future iterations

 

Anonymity

Blackboard offers both full and trust-based anonymous marking options. Please contact the Digital Education office for further advice.  More guidance on anonymity is available in Section 14 of the Regulations and Code of Practice for Taught Programmes.

 

Examples of approaches to electronic marking

  • Use Blackboard's inline grading tool. This allows markers to annotate work directly in Blackboard, without having to download and upload files. See this inline grading tutorial (video, 2 minutes).
  • Annotate work, for example using comments in Word or Adobe Reader. Upload annotated assignments and/or feedback sheets back into Blackboard
  • Fill in an online feedback form (rubric) in Blackboard. See this online rubric tutorial (video, 3 minutes). Turnitin Grademark also offers rubrics: (screenshot of example used in the School of Modern Languages).
  • Download student work via Google Drive to a tablet e. ipad. Annotate using an App such as Notability or i-Annotate, possibly using a stylus for handwritten annotations. Upload back into Blackboard. See this case study: marking using an iPad.
  • Create audio or video/screencast feedback, and make this available to students via Blackboard. See this case study: audio and video feedback using Jing.
  • Use Turnitin Grademark, an onscreen marking tool available via Blackboard, to annotate work directly online without having to download and upload files. Also offers reusable comments and a mobile app for iPad. See these case studies: marking using Turnitin Grademark, marking using Turnitin Grademark iPad app.

 

Further information

Presentation: online approaches to marking and feedback

Roger Gardner delivered a presentation as part of the Education Excellence Seminar series. In it he explores ideas of effective feedback, and how technology can support this.  It shows a range of approaches, with benefits and considerations.

Useful links

Note: some of the documents on this page are in PDF format. In order to view a PDF you will need Adobe Acrobat Reader