東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back to Search results for [ null ]

Switch To: Labeled | MARC Mode | ISBD

The impact of background characteris...

Riemer, Deborah K.

Linked to FindBook

Google Book

Amazon

博客來

The impact of background characteristics on rater severity in a cross-discipline composition assessment.

Record Type:	Electronic resources : Monograph/item
Title/Author:	The impact of background characteristics on rater severity in a cross-discipline composition assessment./
Author:	Riemer, Deborah K.
Published:	Ann Arbor : ProQuest Dissertations & Theses, : 2015,
Description:	171 p.
Notes:	Source: Dissertation Abstracts International, Volume: 76-10(E), Section: A.
Contained By:	Dissertation Abstracts International76-10A(E).
Subject:	Educational evaluation. -
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=3702767
ISBN:	9781321742381

The impact of background characteristics on rater severity in a cross-discipline composition assessment.
Riemer, Deborah K.

The impact of background characteristics on rater severity in a cross-discipline composition assessment. - Ann Arbor : ProQuest Dissertations & Theses, 2015 - 171 p.

Source: Dissertation Abstracts International, Volume: 76-10(E), Section: A.

Thesis (Ph.D.)--Capella University, 2015.

This study investigated the extent to which rater background variables influenced rater inclinations towards severity or leniency when evaluating students' performance on six composition criteria. The raters in this study were faculty members within the context of a university online classroom environment. Participants included 144 faculty members, representing multiple and diverse programs from four university colleges, who evaluated and scored the performance of 3,149 students on select integrated classroom assessments. The raters utilized an electronic rubric to score and record student performance. The subjective nature of performance evaluations, however, may undermine the validity of the resulting assessment data. A systematic source of measurement error can be attributed to rater bias, and more specifically the level of severity or leniency by which raters score student performance. Not so clear, however, is the source of such rater bias. This study sought to determine whether certain rater characteristics impacted a tendency towards severity or leniency when scoring performance assessments. The rater background variables studied were gender, nationality, educational attainment, teaching experience, academic discipline, and linguistic background. This study was guided by validity theory and situated within the context of the online classroom environment. The quantitative analysis utilized FACETS software and the multifaceted Rasch methodology to provide measurements of rater severity and leniency. The results indicated that approximately 33% of the total faculty-student variance could be explained by rater bias. Further analysis indicated a significant degree of bias in approximately 4.2% of all faculty-student interactions, with 79.5% of those on the more severe end of the spectrum. The gender, educational attainment, and teaching experience of participants indicated no significant influence on severity and leniency. Nationality and linguistic background could not be measured due to the homogeneity of the demographic information collected for these variables. Academic discipline, on the other hand, indicated a significant (p<0.05) influence on bias when considering faculty members from the business and management discipline, suggesting a tendency towards leniency. While further study is warranted, implications for discipline-specific bias could necessitate modifications to curriculum and assessment development, methods for student evaluations, and development opportunities for faculty.

ISBN: 9781321742381Subjects--Topical Terms:

526425
Educational evaluation.

The impact of background characteristics on rater severity in a cross-discipline composition assessment.
LDR:03521nmm a2200313 4500 001 2157574
005 20180605073454.5
008 190424s2015 ||||||||||||||||| ||eng d
020 $a 9781321742381
035 $a (MiAaPQ)AAI3702767
035 $a (MiAaPQ)capella:17125
035 $a AAI3702767
040 $a MiAaPQ $c MiAaPQ
100 1 $a Riemer, Deborah K. $3 3345382
245 1 4 $a The impact of background characteristics on rater severity in a cross-discipline composition assessment.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2015
300 $a 171 p.
500 $a Source: Dissertation Abstracts International, Volume: 76-10(E), Section: A.
500 $a Adviser: Rod Sims.
502 $a Thesis (Ph.D.)--Capella University, 2015.
520 $a This study investigated the extent to which rater background variables influenced rater inclinations towards severity or leniency when evaluating students' performance on six composition criteria. The raters in this study were faculty members within the context of a university online classroom environment. Participants included 144 faculty members, representing multiple and diverse programs from four university colleges, who evaluated and scored the performance of 3,149 students on select integrated classroom assessments. The raters utilized an electronic rubric to score and record student performance. The subjective nature of performance evaluations, however, may undermine the validity of the resulting assessment data. A systematic source of measurement error can be attributed to rater bias, and more specifically the level of severity or leniency by which raters score student performance. Not so clear, however, is the source of such rater bias. This study sought to determine whether certain rater characteristics impacted a tendency towards severity or leniency when scoring performance assessments. The rater background variables studied were gender, nationality, educational attainment, teaching experience, academic discipline, and linguistic background. This study was guided by validity theory and situated within the context of the online classroom environment. The quantitative analysis utilized FACETS software and the multifaceted Rasch methodology to provide measurements of rater severity and leniency. The results indicated that approximately 33% of the total faculty-student variance could be explained by rater bias. Further analysis indicated a significant degree of bias in approximately 4.2% of all faculty-student interactions, with 79.5% of those on the more severe end of the spectrum. The gender, educational attainment, and teaching experience of participants indicated no significant influence on severity and leniency. Nationality and linguistic background could not be measured due to the homogeneity of the demographic information collected for these variables. Academic discipline, on the other hand, indicated a significant (p<0.05) influence on bias when considering faculty members from the business and management discipline, suggesting a tendency towards leniency. While further study is warranted, implications for discipline-specific bias could necessitate modifications to curriculum and assessment development, methods for student evaluations, and development opportunities for faculty.
590 $a School code: 1351.
650 4 $a Educational evaluation. $3 526425
650 4 $a Educational tests & measurements. $3 3168483
650 4 $a Curriculum development. $3 684418
690 $a 0443
690 $a 0288
690 $a 0727
710 2 $a Capella University. $b School of Education. $3 1018430
773 0 $t Dissertation Abstracts International $g 76-10A(E).
790 $a 1351
791 $a Ph.D.
792 $a 2015
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=3702767