Invited speakers

Zohreh R. Eslami, Department of Teaching, Learning, and Culture, Texas A&M University

Assessing Second Language Pragmatics

With a gradual increase in efforts at assessing pragmatic ability in recent years, there have emerged various paradigms for how to carry out the assessment. Pragmatic ability has been referred to as the ability to deal with meaning as communicated by a speaker/writer and interpreted by a listener/reader. It involves the ability to interpret people's intended meanings, their assumptions, their purposes or goals, and the kinds of actions (for example, requests) that they are performing when they speak (Yule, 1996: 3-4). Because of its highly contextualized nature, assessment of pragmatics involves significant tension between the construction of authentic assessment tasks and practicality. Social context must be established and learner responses are often productive, so simulations of real world situations and scoring by human raters would be ideal, but they are also very costly.

In this presentation, I will first give a brief description of L2 pragmatics, followed by a review of some of the assessment instruments for L2 pragmatics. I will then discuss some of the persistent issues of testing pragmatics, especially the use and misuse of discourse completion tests (DCTs), and highlight issues in testing sociopragmatics. I will also briefly explore the role of agency, subjectivity, and identity preservation in L2 pragmatic assessment. Some practical classroom-based assessment strategies will be proposed which promote pluralistic norms of behavior in the target community through the discussion of various community interpretations, as well as through the use of research-based information about how speech acts are actually realized. The presentation will conclude with some suggestions for future research and practice in the area.

Annie Brown, Ministry of Higher Education and Scientific Research , United Arab Emirates

Interactional competence can be defined in terms of how speakers structure and sequence their speech, and how they apply turn-taking rules. It can also be defined in terms of how speakers collaborate and support their interactional partner to co-construct the spoken performance. This concept of interactional competence has led to critiques of more traditional interview formats of speaking tests, and support for paired or group candidate tasks, both in large-scale testing contexts in classroom-based assessment.

While early research into paired or group interactions tended to focus on the impact of candidate features, such as personality, proficiency, or gender, on the interactional partners' performance and assessment, more recent research has begun to explore what it is that makes paired interaction truly different from interview performance. Studies have focused not only on the nature of the discourse produced, but also on the nature of the criteria that can be used to judge interactional competence.

This paper reviews some of the recent literature in this area, focusing particularly on explorations of the nature of interactional competence displayed in paired tasks, on how raters orient to assessments of interactional competence, and on methods of scale development for such assessments          

Gholam Reza Kiany, Trabiat Modares University  

Detecting Differential Item Functioning through Two DIF Procedures: IRT and Logistic Regression

Differential Item Functioning (DIF) occurs when two groups of test takers with equal ability show a differential probability of a correct response. The presence of DIF shows that factors related to group membership (e.g., gender, ethnicity, language background, academic background, socioeconomic status, etc.) influence the probability of a correct response, consequently, threatening fair assessment. This is an important issue because for test scores to be comparable across groups, the items must work in the same way. Academic field differences (mathematics, science and humanities) are one of the problematic issues in high-stake tests. DIF is among the factors which might influence test performance in favour of one or another particular group. Although this influence is well known and recognized, a preliminary DIF analysis is not a common practice in academic field studies. In order to estimate DIF for the three field groups in this study, two methods were utilized i.e. Item-Response Theory, and logistic regression procedure.  

S. Susan Marandi, Al-Zahra University

Bridging the gap among language testers: Introducing EFL teachers to online assessment

While many language assessment experts are ambitiously aiming at high-stakes computer-adaptive language tests, it is a fact that the majority of language teachers lack the know-how and confidence to create or even administer a simple computer-based test. However, as online learning becomes increasingly common and whole academic degrees are being offered through the Internet, learning how to create and utilize online assessment tasks is imperative. For this reason, the present paper intends to serve as an introduction to the basics of e-assessment in TEFL. Guidelines will be provided for e-assessment practices, and several user-friendly tools for developing and administering computer-based tests for teachers with limited computer knowledge will be introduced. These tools include but are not limited to Easy Test Maker, RubiStar, and Hot Potatoes. 

Kourosh Lachini, Qatar University, Doha, Qatar

Evaluation Methods in Research: Foundation Program at Qatar University Case

This presentation is intended to provide both background on the ideas and theories, which underpin educational evaluation, and practical guidance on how to plan and undertake an evaluation. It will also argue the case for evaluation being most effective when a multi-method approach is adopted: in other words, an approach that draws on a variety of perspectives on evaluation and also employs a range of research strategies and techniques to gather different forms of data. The presenter will attempt to briefly discuss ways of defining and explaining evaluation, models and approaches to evaluation, models of change which have emerged from evaluation studies, research strategies and techniques which are employed in evaluation, and practicalities of planning and evaluation. Any discussion on evaluation needs to address the issue of how far details of general aspects of research methods should be included. In this instance, information will be included about key research strategies and techniques, as these are central to the planning and design of an evaluation, but to exclude material on aspects of research methods. Thus, matters to do with the detail of designing research instruments (questionnaire, interview schedule, observation schedules) and data analysis are not included in this presentation. This presentation will conclude with the evaluation of the Foundation Program at Qatar University, which is currently in process by the presenter as a representative case of program evaluation. The ultimate purpose of this evaluation is to provide legitimate answers to the following questions: What works? How does it work? How can it be made to work better?