Dissertation
Design and evaluation of video portfolios: reliability, generalizability, and validity of an authentic performance assessment for teachers
The research study presented in this dissertation focuses on issues pertaining to the reliability, generalizability, and validity of authentic performance assessments of teachers.
- Author
- Maria Elisabeth Jacomina Bakker
- Date
- 02 December 2008
- Links
- Fulltext in Leiden University Repository
Generally, these assessments include the use of complex and open-ended assessment tasks, often carried out in varying contexts. Compared with more 'traditional' forms of assessment, in which the assessment tasks used are more distant from the classroom reality, the context and tasks present in authentic performance assessments bring specific threats to the methodological quality of these assessments. Central to this dissertation are three studies addressing processes and factors that affect the reliability, generalizability, and validity of authentic performance assessments. The focus is particularly on issues related to the process of construction of performance assessments, assessors’ scoring, and the reliability, generalizability, and validity of performance scores.
Method
The three studies are based on data that was collected during and after assessors’ scoring of a structured video portfolio. Video portfolios were developed for assessing teachers’ coaching competence while coaching students in senior secondary vocational education. As part of this assessment procedure, trained assessors judged the video portfolios of four teachers. A structured video portfolio consists of a collection of evidence pertaining to the competence to be assessed, i.e., teachers’ coaching competence while coaching students in senior secondary vocational education. The main source of evidence consisted of video recordings that were systematically selected and showed teachers’ coaching performance in key situations in practice. Additional sources of information were included in the portfolio pertaining to the context in which the coaching took place, like information about how far students were in completing an assignment, information about students’ backgrounds, or an interview with the teacher about the decisions underlying his or her actions.