Yves BESTGEN (Université Catholique de Louvain)Evaluation of texts in English as a foreign language and phraseological series: comparison of two freely accessible automatic procedures2019, Vol. XXIV-1, pp. 81-94
When automatically evaluating the quality of a text written in a foreign language, phraseological series are frequently neglected while their mastery is a major component of learning. Recently, two automatic systems capable of taking them into account in English have been made freely available: the COCA Parser and TAALES. The study evaluates the effectiveness and utility of these systems by applying them to two sets of learner texts. While both systems proved to be relatively effective, only the COCA Parser was able to learn a predictive model on one dataset and successfully apply it to another. It is also the only one of the two systems that produces a file allowing a qualitative analysis of the evaluated texts.
Galina BOUBNOVA (Moscou, Russie)The Federal Olympiad for French in Russia: a field for pedagogical reflection and experiment2010, Vol. XV-1, pp. 75-88
An overview of FLE (French as a foreign language) student assessment in Russia reveals the following state of affairs. On the one hand, diplomas DELF-DALF (Diploma in French Language Studies - Diploma in Advanced French Language Studies) and TCF (Test of Knowledge of French) are not officially recognized in Russia. On the other hand, the Ministry of Education and Science of the Russian Federation proposes two types of certification: the Unified State Examination and the All-Russian Olympiad. The tasks of the latter were conceived in a way to become the interlink between the French language curricula and performance assessment criteria used in Russia and those devised for French and European tests and examinations. In other words, the goal is to create mark schemes and assessment criteria which would correspond to those of DALF-DELF and would conform to the recommendations of the CEFR (Common European Framework of Reference for Languages). The present article deals with the All-Russian Olympiad in French (the annual contest organized by the Ministry of Education and Science of the Russian Federation) from the methodological point of view as inviting pedagogical reflection and experiment, the purpose being to introduce assessment procedures which would fit into the common assessment system already adopted in Europe.
Jean-Marie DE KETELE (Louvain-la-Neuve, Belgique)Choosing the right evaluation method2010, Vol. XV-1, pp. 25-37
This article starts from the curious observation that, in a world where complex trades become increasingly professionalized, assessments in schools come generally down to awarding marks. If done professionally, the evaluator should before all other things answer questions about the types of functions that the evaluation has to fulfill and the procedures to follow. More specifically, the article addresses two important aspects of professionalization : evaluation of language competencies and the reinforcement of the functions of orientation and regulation.
Jean-Marie DE KETELE (Louvain-la-Neuve, Belgique)The evaluation of writing2013, Vol. XVIII-1, pp. 59-74
The term “evaluating written language” hides underlying polysemic concepts that this article aims at clarifying from the start. Therefore both the certifying and the formative evaluation of written production will be addressed. The former is doubly institutional: its aim is to give an orientation and a planification of learning writing skills, and also constitutes the basis of the final decision of success or failure. The latter, widely claimed but less widely applied, aims to improve the learning of the skill during the learning, and is based on diagnostic concepts, and, in coherence with those, on the setting up of permanent actions allowing learners to progress in their learning. These aspects will be illustrated by concrete examples.
Jean ECALLE (Lyon 2)Assessment of reading and related skills2010, Vol. XV-1, pp. 105-120
Reading assessment should be completed with cognitive skills (memory and rapid automatized naming) and linguistic skills (particularly phonological domain) related to reading and learning to read. This review aims to present the importance of assessing in general and the two aspects of reading assessment in terms of screening and diagnosis. For screening and more precisely to detect children at-risk of failure in learning to read, tests of language development and phonological skills are presented. For a diagnosis in reading, we distinguish pencil-paper tests and computer-based tests. For a complete diagnosis, tests in written word recognition and in reading comprehension are presented with their limits. We conclude that assessment should be closely related to interventions as prevention or remediation.
François-Marie GERARD (BIEF, Louvain-la-Neuve, Belgique)Evaluation as a tool for regulating learning : challenges, necessities and difficulties2013, Vol. XVIII-1, pp. 75-92
School assessment is a systematic procedure which allows enhancing the value of acquired learning as well as giving it meaning.The first challenge is to distinguish between certification evaluation, which aims at probing success in learning, and formative evaluation. This latter is more oriented to improving the acquired learning in itself. This “assessment for learning” can take different forms, depending on the standpoints of the various teachers. A second challenge concerns grasping the evaluation objects: on the one side the knowledge and know-how, and on the other side, the competences needed for their mobilization in order to solve a complex situation.
Antony John KUNNAN (Los Angeles, Etats-Unis)Statistical analyses for test fairness2010, Vol. XV-1, pp. 39-48
Test fairness in language assessment has now been discussed by researchers for over a decade but statistical methods that can be used to analyze test fairness qualities have not been clearly articulated. In this article, I describe briefly my test fairness framework, the principles behind it and the test qualities that encompass test fairness. I then describe statistical analyses that can be used to analyze tests and testing practice such as factor analysis, structural equation modeling and differential item functioning.
Yann MATHET (GREYC)Annotation, evaluation and agreement measures in corpus linguistics2019, Vol. XXIV-1, pp. 111-129
The establishment and evaluation of annotated resources as well as automatic annotation systems have growing interest in linguistics and NLP. However, evaluation methods are not always well mastered and used, particularly in linguistics, where the sequential nature of textual data requires specific methods. In this article, we first analyze the notions of corpus and annotation in their multiple forms, then focus on the results of the evaluation questions. In particular, we emphasize the difference in nature between the evaluation of an annotation system and the evaluation of manually multi-annotated resources. We provide an overview of existing evaluation tools and their possible use for each of these two cases.
Tim MCNAMARA (Melbourne, Australie)The use of language tests in the service of policy: issues of validity2010, Vol. XV-1, pp. 7-23
This paper considers the role of language tests as instruments of policy and contemporary society and the issues for validity that this raises. It contrasts approaches which focus solely on the fairness of tests with a broader view of validity which addresses issues of justice in the uses of tests. It illustrates its argument with reference to the Common European Framework of Reference for Languages and tests used in workplace accreditation such as language tests for pilots and air traffic controllers and for health professionals.
Jean-Philippe RIVIÈRE (IUFM de Bretagne)Reading skills of French youngsters: new ways of large-scale assessing of cognitive performance2010, Vol. XV-1, pp. 121-144
Every year, reading performance of about 700.000 young people is assessed in the framework of a day (JAPD, Journée d'Appel de Préparation à la Défense) organised by the French Ministry of Defense. These tests changed recently due to some constraints of test administration and the taking into account of the results of the previous version of the test. Based on the results collected over the years 2004-2008, this paper presents the methodological design of this large span evaluation of reading. This study is an occasion to moderate the binary oppositions between literate and illiterate, and to show that various more or less efficient ways of reading, characterize this activity. For some young people, the switch from schooling to active professional life implies a new relation with written language. The school, which is the main location of reading and writing solicitation, will be replaced by economical activities. For some young people, the relation with writing will be definitively distended.
Aude SEURRAT (Université Paris 13)Evaluating effectiveness or measuring satisfaction in short vocational training?2019, Vol. XXIV-1, pp. 53-66
This article questions the forms, modalities and issues in short vocational training. The evaluation is seen as both the outcome of training and the upstream direction of its implementation: it is both the purpose and the genesis of the action. Presented as a modality of rationalization for performance optimization, evaluation is particularly grappling with industrialization issues. As an addition to training professionals, evaluation guides the way actors act and think about their action in organizations. Equipped and instrumented, it finally involves a fine-tuned approach to mediations and, first and foremost, to the computerized tools that give it form.
Jacques TARDIF (Faculté d’éducation, Université de Sherbrooke)On the nature of transversal competencies until their evaluation : a race with sometimes impossible obstacles2013, Vol. XVIII-1, pp. 29-45
Transversal competences define and target knowledge that every human being should develop in school in order to understand the complexity of situations in normal life and act intelligently. However, without anchoring them in concrete situations, the course of development of these competences and their assessment seem to stay rather vague. After defining the concept of competence and sketching the guidelines of competence assessment, the article illustrates the difficulty of influencing critical learning inherent in transversal competence and the necessity of evaluating it. The last part of the article makes suggestions for professionalisation of this activity.
Sylviane VALDOIS (Grenoble 2)Assessing difficulties in the acquisition of reading2010, Vol. XV-1, pp. 89-103
Improvement of knowledge about the cognitive mechanisms involved in skilled reading and reading acquisition contributed to the development of new tools for the assessment of reading disability. A first step of the assessment focuses on the efficacy of the global and analytic reading procedures. Evidence for disharmonious development of these procedures then requires investigating the cognitive mechanisms involved in their establishment. Phonological processing skills are then typically assessed because of their well-known impact on reading acquisition. Recent findings have shown that the integrity of another cognitive mechanism, the visual attention span, was also required for the normal development of the reading system.