|
|
|
ProblemsWriting Placement | Online Placement › Problems with entry writing placement are legion and partly account for the variety of systems and the frequency with which college switch from one system to another. Some problems are general since they inhere in the difficulties of assessing writing skill at all—difficulties in getting raters to agree, in obtaining a reliable sample of the student’s writing skills, in providing a fair opportunity for the student to demonstrate those skills. There also may be systemic or logistical problems peculiar to the testing site and time. • Students are little in the mood for writing if they are testing during summer orientation sessions or just before classes begin in the fall.
• Teachers are little in the mood for reading if they are placing students in the same circumstances.
• Topics, modes, criteria, and readers used by testing firms may be far removed from the local writing curriculum.
• Decisions to test or to use a particular testing system often are not made by the writing faculty, who may not even be consulted.
Other problems adhere to the particular placement system used. • The method of reading—holistic or general impression—used by commercial testing firms to score essays is simplistic and designed to be cost efficient (e.g., quick and with high rater reliability) rather than to be course appropriate.
• Automatic computer scoring (D) just duplicates that simplistic kind of reading.
• Portfolios (E) are time consuming to read, have low rater agreement, and seem biased in favor of women writers.
• Indirect tests of writing (F) may be biased against marginalized or minority groups.
• Self-placement systems (G and H) depend much on a doubtful factor, the students’ sense of self-efficacy, how accurately they can judge their own abilities, and upon knowledge of a course about which they may know little.
• Re-enrollment after the beginning of classes (I) can be a logistical nightmare.
• The judgment of non-composition teachers (J) may be spotty, eccentric, or unattuned to the actual writing instruction or objectives.
Many of these problems are entangled with the one most crucial problem, the low predictive validity of writing placement. When research has compared placement decisions with the subsequent student performance in the courses, good match is rarely if ever found. It does not matter when the performance used as target criterion is final grade in the course, passing the course, student judgment of course fit or learning, teacher judgment of fit or learning, or academic success in future courses. The correlation between placement test score and performance ranges between .2 and .4. This is an embarrassingly weak correlation. It explains only about a tenth of the variability of student performance; the rest, nine tenths of it, must be accounted for by other factors. Placement by diect testing, preferred by writing teachers, is as poor a predictor as indirect testing. |