語系:
繁體中文
English
日文
簡体中文
說明(常見問題)
登入
回首頁
切換:
標籤
|
MARC模式
|
ISBD
Weighting Patterns and Rater Variabi...
~
Cai, Hongwen.
Weighting Patterns and Rater Variability in an English as a Foreign Language Speaking Test.
紀錄類型:
書目-語言資料,印刷品 : Monograph/item
書名/作者:
Weighting Patterns and Rater Variability in an English as a Foreign Language Speaking Test.
作者:
Cai, Hongwen.
面頁冊數:
212 p.
附註:
Source: Dissertation Abstracts International, Volume: 74-02(E), Section: A, page: .
Contained By:
Dissertation Abstracts International74-02(E)A.
標題:
Education, Tests and Measurements.
標題:
Language, Linguistics.
標題:
Education, English as a Second Language.
ISBN:
9781267652102
摘要、提要註:
This study is an attempt to measure the weighting patterns of the raters in a large-scale English as a Foreign Language (EFL) speaking test, classify these raters according to their weighting patterns, characterize the different types of raters in the rating process, and associate the rater types with different patterns of rater variability. The context was the Test for English Majors - Band 4, Oral Test (TEM4-Oral), a high-stakes certification test administered to college EFL majors in China toward the end of their sophomore year. To quantify the weighting patterns, 126 nonnative-speaking college teachers of English who served as TEM4-Oral raters in 2010 were requested to judge the EFL oral proficiency of 120 hypothetical test-takers with computer-generated score profiles featuring strengths and weaknesses in various criteria. Their relative weights on the criteria were derived from regression analyses, and then fed into cluster analyses to classify the raters into three types. To characterize different types of raters, a sample of 21 raters were involved in verbal protocols and requested to rate the performance of five real test-takers and justify their ratings. To associate the rater types with the different patterns of rater variability, the real ratings of 33 raters including all three types were analyzed through Many-Facet Rasch Measurement, Hierarchical Linear Modeling, Generalizability Theory, and Confirmatory Factor Analysis. The cluster analyses classified the raters into three types according to whether they gave the largest weights to form-related criteria or content-related criteria, or were balanced in the weighting patterns, and the three types were named form-oriented, content-oriented, and balanced respectively. In the verbal protocols, the form-oriented raters were found to be most severely subject to the anchoring and masking effects of pronunciation and intonation whereas the content-oriented raters displayed the strongest mitigation of such effects. The balanced raters came in between, but shared more similarity with the content-oriented raters. In association with rater variability, the form-oriented raters were found to be most severe among the three types and the content-oriented raters most lenient. On specific TEM4-Oral subscales, the form-oriented raters were unexpectedly severe in pronunciation and intonation, but unexpectedly lenient in grammar and vocabulary, whereas the content-oriented raters were unexpectedly lenient on the content-related subscale of discussion but unexpectedly severe on the subscale of grammar and vocabulary. However, no clear-cut relationship was found in reliability and restriction of range, and mixed results were reported in terms of halo effect.
電子資源:
http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=3540377
Weighting Patterns and Rater Variability in an English as a Foreign Language Speaking Test.
Cai, Hongwen.
Weighting Patterns and Rater Variability in an English as a Foreign Language Speaking Test.
- 212 p.
Source: Dissertation Abstracts International, Volume: 74-02(E), Section: A, page: .
Thesis (Ph.D.)--University of California, Los Angeles, 2012.
This study is an attempt to measure the weighting patterns of the raters in a large-scale English as a Foreign Language (EFL) speaking test, classify these raters according to their weighting patterns, characterize the different types of raters in the rating process, and associate the rater types with different patterns of rater variability. The context was the Test for English Majors - Band 4, Oral Test (TEM4-Oral), a high-stakes certification test administered to college EFL majors in China toward the end of their sophomore year. To quantify the weighting patterns, 126 nonnative-speaking college teachers of English who served as TEM4-Oral raters in 2010 were requested to judge the EFL oral proficiency of 120 hypothetical test-takers with computer-generated score profiles featuring strengths and weaknesses in various criteria. Their relative weights on the criteria were derived from regression analyses, and then fed into cluster analyses to classify the raters into three types. To characterize different types of raters, a sample of 21 raters were involved in verbal protocols and requested to rate the performance of five real test-takers and justify their ratings. To associate the rater types with the different patterns of rater variability, the real ratings of 33 raters including all three types were analyzed through Many-Facet Rasch Measurement, Hierarchical Linear Modeling, Generalizability Theory, and Confirmatory Factor Analysis. The cluster analyses classified the raters into three types according to whether they gave the largest weights to form-related criteria or content-related criteria, or were balanced in the weighting patterns, and the three types were named form-oriented, content-oriented, and balanced respectively. In the verbal protocols, the form-oriented raters were found to be most severely subject to the anchoring and masking effects of pronunciation and intonation whereas the content-oriented raters displayed the strongest mitigation of such effects. The balanced raters came in between, but shared more similarity with the content-oriented raters. In association with rater variability, the form-oriented raters were found to be most severe among the three types and the content-oriented raters most lenient. On specific TEM4-Oral subscales, the form-oriented raters were unexpectedly severe in pronunciation and intonation, but unexpectedly lenient in grammar and vocabulary, whereas the content-oriented raters were unexpectedly lenient on the content-related subscale of discussion but unexpectedly severe on the subscale of grammar and vocabulary. However, no clear-cut relationship was found in reliability and restriction of range, and mixed results were reported in terms of halo effect.
ISBN: 9781267652102Subjects--Topical Terms:
423104
Education, Tests and Measurements.
Weighting Patterns and Rater Variability in an English as a Foreign Language Speaking Test.
LDR
:03900nam 2200337 4500
001
377941
005
20130403093116.5
008
130522s2012 ||||||||||||||||| ||eng d
020
$a
9781267652102
035
$a
(UMI)AAI3540377
035
$a
AAI3540377
040
$a
UMI
$c
UMI
100
1
$a
Cai, Hongwen.
$3
506883
245
1 0
$a
Weighting Patterns and Rater Variability in an English as a Foreign Language Speaking Test.
300
$a
212 p.
500
$a
Source: Dissertation Abstracts International, Volume: 74-02(E), Section: A, page: .
500
$a
Adviser: Lyle F. Bachman.
502
$a
Thesis (Ph.D.)--University of California, Los Angeles, 2012.
520
$a
This study is an attempt to measure the weighting patterns of the raters in a large-scale English as a Foreign Language (EFL) speaking test, classify these raters according to their weighting patterns, characterize the different types of raters in the rating process, and associate the rater types with different patterns of rater variability. The context was the Test for English Majors - Band 4, Oral Test (TEM4-Oral), a high-stakes certification test administered to college EFL majors in China toward the end of their sophomore year. To quantify the weighting patterns, 126 nonnative-speaking college teachers of English who served as TEM4-Oral raters in 2010 were requested to judge the EFL oral proficiency of 120 hypothetical test-takers with computer-generated score profiles featuring strengths and weaknesses in various criteria. Their relative weights on the criteria were derived from regression analyses, and then fed into cluster analyses to classify the raters into three types. To characterize different types of raters, a sample of 21 raters were involved in verbal protocols and requested to rate the performance of five real test-takers and justify their ratings. To associate the rater types with the different patterns of rater variability, the real ratings of 33 raters including all three types were analyzed through Many-Facet Rasch Measurement, Hierarchical Linear Modeling, Generalizability Theory, and Confirmatory Factor Analysis. The cluster analyses classified the raters into three types according to whether they gave the largest weights to form-related criteria or content-related criteria, or were balanced in the weighting patterns, and the three types were named form-oriented, content-oriented, and balanced respectively. In the verbal protocols, the form-oriented raters were found to be most severely subject to the anchoring and masking effects of pronunciation and intonation whereas the content-oriented raters displayed the strongest mitigation of such effects. The balanced raters came in between, but shared more similarity with the content-oriented raters. In association with rater variability, the form-oriented raters were found to be most severe among the three types and the content-oriented raters most lenient. On specific TEM4-Oral subscales, the form-oriented raters were unexpectedly severe in pronunciation and intonation, but unexpectedly lenient in grammar and vocabulary, whereas the content-oriented raters were unexpectedly lenient on the content-related subscale of discussion but unexpectedly severe on the subscale of grammar and vocabulary. However, no clear-cut relationship was found in reliability and restriction of range, and mixed results were reported in terms of halo effect.
590
$a
School code: 0031.
650
4
$a
Education, Tests and Measurements.
$3
423104
650
4
$a
Language, Linguistics.
$3
423211
650
4
$a
Education, English as a Second Language.
$3
423323
690
$a
0288
690
$a
0290
690
$a
0441
710
2
$a
University of California, Los Angeles.
$b
Applied Linguistics 0074.
$3
506803
773
0
$t
Dissertation Abstracts International
$g
74-02(E)A.
790
1 0
$a
Bachman, Lyle F.,
$e
advisor
790
1 0
$a
Bachman, Lyle F.
$e
committee member
790
1 0
$a
Bentler, Peter M.
$e
committee member
790
1 0
$a
Erickson, Frederick D.
$e
committee member
790
1 0
$a
Webb, Noreen M.
$e
committee member
790
$a
0031
791
$a
Ph.D.
792
$a
2012
856
4 0
$u
http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=3540377
筆 0 讀者評論
多媒體
多媒體檔案
http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=3540377
評論
新增評論
分享你的心得
Export
取書館別
處理中
...
變更密碼
登入