Bernard Spolsky, Professor emeritus, Bar-Ilan University
                              Jerusalem, Israel

Modernizing Language Assessment

       The historical development of language assessment, starting with the Shibboleth incident in the Bible, and continuing through the Chinese Imperial Examinations, the Medieval Treviso test, the Jesuit 17 th century schools, the French Revolution, the Indian Civil Service test in 19 th century Britain, and the British primary school standards, all formed the basis of what is sometimes called Traditional Testing, though each was defined for a different purpose. Just over 100 years ago, statisticians raised fundamental questions about the reliability of these methods of measurement, showing their "inevitable uncertainty". Modern psychometric testing (the multiple choice test especially) attempted to correct this. These tests, favored by governments and large institutions and industrially available, are certainly fairer, but more recently, serious issues concerning what is being tested, the misuse of test results and the ethics of testing have led to new developments, aiming to produce assessment that is fair to all, relevant to the use and purpose of the instrument, cautious in interpretation, and efficient.

James E. Purpura, Teachers College, Columbia
                                  University

A Learning-Oriented Approach to Assessing Grammar in Classroom Contexts

      Large-scale and classroom approaches to grammar assessment rooted in structural linguistics and discrete-point measurement of the 1960s (Lado, 1960; Carroll, 1961) have served a number of purposes quite well over the years. However, approaches to grammar assessment based uniquely on the accuracy of linguistic forms may no longer support more dynamic and complex understandings of how students represent grammatical knowledge at different proficiency levels or of how learners are able to use grammatical knowledge to convey a range of literal and pragmatic meanings (Purpura, 2004). Furthermore, grammar assessments in classroom contexts have generally been conceptualized with reference to some explicit or implicit model of communicative language ability but have unfortunately not been linked to a model of second or foreign learning--even though the most common goal of classroom-based assessment of grammar is to provide information to support learning.
       In this talk, I will question the assumptions that underlie traditional grammar assessment and assessment practices. I will first argue for a learning-oriented model of language assessment-one that situates language assessment within both a model of language learning and language proficiency-and I will show how grammar assessment can be designed with reference to grammar processing. After that, I will argue for a model of grammatical ability that defines grammatical knowledge not only in terms of linguistic forms, but also in terms of how these forms encode both literal and pragmatic meanings in situated language use. In the last part of talk, I will discuss a range of methods for eliciting grammatical and pragmatic knowledge and for scoring grammatical performance.

Jared Bernstein, Linguistics Department, Stanford
                               University & Knowledge Technologies, Pearson

Automatic Evaluation of Speaking in Historical Perspective

    Speaking a second language nicely is both the core competence and the crowning jewel of second language acquisition. Functional speaking and listening in a language classroom is a reasonable indicator that something is right, as reading and writing can surely be taught to people who speak the language. If we accept the idea, in principle, that frequent appropriate assessment can be a key element in instructional design, then frequent appropriate assessment of speaking and listening should be central to second language pedagogy. But it isn't; and it isn't because the traditional methods are time consuming and often unreliable. This situation calls out for efficient, accurate methods to evaluate speaking skills.
       A speaking test for instruction is different from a spoken language test for selection. In selection, a spoken language test should measure expected performance over a whole domain of language use. For instructional use, a speaking test may be focused more exactly on the skills and levels of performance defined in a curriculum. The history of automatic methods for evaluating speaking will be reviewed in this context. First, I illustrate and explain the basic acoustic pattern-matching methods that are used to evaluate style-of-speaking and return scores of pronunciation quality and phonological fluency. These methods are extensions of speech recognition technology, which have clear applications in providing formative feedback for instruction - they make fine speaking tests. However, the application of these style-of-speaking methods in estimating spoken language proficiency runs into limits that can be analyzed with reference to existing concepts of spoken language proficiency. Finally, more complete current automatic methods for spoken language testing will be presented. A brief analysis of the strengths and limitations these methods suggests the likely future directions of research and applications with automated spoken language tests.