Home Elements for a Diagnosis of the Spanish Educational System
1 - Previous studies of diagnosis and evaluation (end)


Previous Table of Contents Next


Research aimed at diagnosis of the Educational System

N.B.: The INCE has taken advantage of two specific studies on this important aspect: the first of these (Aportaciones de la investigación al diagnóstico del sistema educativo espańol - Contributions by research to diagnosis of the Spanish educational system) is due to the generosity of the Research Service at the Centre for Educational Research and Documentation, coordinated by Mercedes Muńoz Repiso and Javier Murillo Torrecilla, a very ample study that provides considerable detail of much of the research carried out by this institution and by the institutions which, under another name, preceded it; the second is a summary of the former, which the INCE commissioned to Ángel Lázaro, Associate Teacher of Educational Orientation and Diagnosis at the Complutense University of Madrid, under the title of Contributions by research to diagnostic study of the Spanish educational system. The lines included here are based on these studies, which may be requested from the INCE by those who are especially interested.
 

Considering that, as said at the beginning, the limits to this chapter prevent detailed treatment of such a large matter, we shall restrict this in the following lines to demonstrate a certain fact; the attention paid in the last decades in Spain to educational research and, within it, to matters related to the situation and possibilities of improvement of more or less large sectors of the academic system. A fair part of this work has been possible thanks to direct support from the educational authorities of the country, especially since the National Centre for Educational Research and Development (CENIDE) began its operations in 1970. It began a policy of stimulation and aid for research themes since then, more related to educational theory and practice, that has not ceased till present, with phases of greater or lesser activity. The institutional forerunners of the first CENIDE, that is to say, the National Educational Sciences Institute (INCIE) and, later, the Centre for Educational Research and Development (CIDE) were those commissioned to materialise this effort that, one must say, they have not carried out alone. Other institutions, such as the former Institute for Pedagogy at the Higher Council for Scientific Research, or some such as the Institute of Educational Sciences at the Universities, also carried out important research. Particular mention must be given to the Schools of Education or Educational Sciences, which have intensified their research tasks as the years go by and, through works by the Professors, doctoral theses, etc., have increasingly influenced matters related to diagnosis and improvement of the educational system and, perhaps especially, in deeper study and diffusion of the appropriate methodologies.

From all this wide range of items, we must be content here to show only those financed directly by the educational authorities through the institutional group CENIDE-INCIE-CIDE. We hope that in future studies, at the instances of the INCE itself, or other institutions, we may also emphasise the important effort made in other settings.



DIAGNOSIS AND EVALUATION OF THE OVERALL EDUCATIONAL SYSTEM

First of all, one must point out that the mainstream research commissioned by said institutional group was aimed at research themes applied to classroom action and educational processes and techniques, there being few studies aimed directly at diagnosis and evaluation research.

The study document prepared by the Research Service at the CIDE was based on an ample sample of the total research carried out by the institutional group. The sample covers 346 studies out of a total 870, which is almost 40% of the total production. In their choice, they took into account criteria set at the specific request of the INCE, such as referring only to pre-university levels, the possibility of generalising diverse contexts, the nature of applied and not basic research, etc. In order for the reader to be able to understand the themes covered by the sample as well as the specific volume of the research dedicated to each theme, we reproduce one of the graphs from the CIDE document (graph 1).


Graph 1: Number of researches comprising the sample by specific theme

If we had to predict where research into diagnosis and/or evaluation of the educational system were to fit into this thematic distribution, in general, or in any of its levels or modes, it would be logical to look first at the group labelled "Evaluation and inspection" and perhaps also, in second place, that on "General structure and organisation of the system".

As to the first group, the thematic classification is that shown on graph 2, which shows the scarce interest dedicated to it.


Graph 2: Number of research studies on evaluation (and inspection) by specific theme

As we may see, the greatest part of the evaluation effort has concentrated on the part that is surely furthest from all aimed diagnosis: programme evaluation. The studies affecting this sector are substantially dedicated to analyze reform programmes or experimental programmes in general; and in other cases, specific programmes (integration, drug prevention, press-school, etc.) and syllabus programmes.

As reads the summary prepared by Ángel Lázaro, "the evaluation of schools and evaluation of teachers are matters that are scarcely covered, as evaluation of these scopes gives rise to rejection and, even more, is identified with matters of control, and even more so if an external evaluation".

Throughout the period, there is only one study dedicated to evaluation of the educational system which, in Lázaro’s opinion, is not significant in number or results.

There may be greater interest for our purposes in the set of research studies that the main reference document brings together under the heading "General structure and organisation of the educational system", the content of which is shown in graph 3. As may be seen from this, a certain number of research studies have been dedicated to studying the situation of the levels of schooling, and one must emphasise the fact that the level that attracts the greatest attention of the researchers is secondary education. However, none of the studies carried out seems to cover the situation of secondary education overall or in any of its two cycles (lower and upper), but specific aspects. As to the research into implementation of the educational reforms, an appreciable number of those follows the line of comparing experimental groups with control groups taken from the previous educational reform. There is a particular interest in those included in the analysis of attitudes of diverse groups to the draft reform that led to the LOGSE; and as to these concerning "moments of transition", most are dedicated to the always controversial matter of university entrance.


Graph 3: Number of research studies on structure and organisation of the educational system
in each thematic scope

After this brief approximation, the conclusion that must be reached is that the attention paid to diagnosis and evaluation actions for the education system by educational researchers has been quite scarce, hardly comparable in quantitative and qualitative terms to that spent on other items. However, this does not mean that the contributions made due to the research efforts are scarcely significant in the scope concerned. On the contrary, they form a source of reflection and even of data to which it will be indispensable to resort for more in-depth study, one by one, into the different aspects.



CONTRIBUTIONS TO THE DIFFERENT ASPECTS OF DIAGNOSIS

STUDENT PERFORMANCE

Although these are not as numerous as one could perhaps expect, the research into student performance forms a body of special importance within the objective concerned here. The document taken as reference orders them in three fundamental chapters (graph 4).


Graph 4: Number of research studies into student performance by specific theme

As to performance analysis, to which the greatest attention is paid, the studies provide interesting data and considerations on the features of academic performance and on the variables affecting this, of a personal nature and of the contextual type, the latter subdividable into "school" and "social-family". Although a general diagnosis action, such as that now performed in Spain and forming the object of this work must make special emphasis on the really existing situation and in principle is not aimed at detecting the main variables that affect this situation, there is no doubt that study of these may not be postponed till later steps. It will then be the moment to look back over the research performed up to present.

In spite of there being many less, the studies on academic performance diagnosis are also very important. As one might expect, they concentrate above all on classroom actions and one may obtain suggestions of undoubtable methodological usefulness from these. There are also few - above all if one takes their great social influence into account - studies concerning academic failure which, in any case, show the high rates of unsatisfactory results obtained in diverse terminal stages, particularly at the end of Primary Education (EGB).


SYLLABUSES AND TEACHING METHODS

There is a particularly considerable number of financed research studies concentrating on the curricula or syllabuses. A considerable percentage of the sample presented in the study of the CIDE (25%) was dedicated to secondary or intermediate education. If one also bears in mind that among those dedicated to EGB, there is also an appreciable number that concentrate on its last phase (which in international terms is also part of lower secondary education), we may conclude that the secondary level in general has been the preferred object of research for decades and, thus, an evaluation of present syllabuses at that level and the immediately preceding ones would require contact with at least some of the research studies referred to.

As to the specific themes the curriculum related research concerns, see graph 5.


Graph 5: Number of research studies on the syllabus by specific theme

As it may be seen, the teaching objectives and contents form a very important zone, where one may note lacunae or deficiencies in certain approaches; specifically, there are interesting studies into vocabulary, language, foreign language, mathematics, etc. There are even more studies on the methodology of teaching-learning processes. However, there are very few on learning resources (printed material, computer resources, etc.), and some more dedicated to evaluation of learning (evaluation systems, diagnosis instruments, etc.). Cross-curricular have also been the subject matter of a few studies.

After those dedicated to syllabus, the following ones in number (graph 1) is research organised under the heading of inequalities in education, that perhaps may also be included in the general section dedicated to the syllabus, as they affect some of its modes. As may be seen on graph 6, they mainly concern two matters: special education and compensatory education. As Ángel Lázaro says, they are generally "a posteriori" studies, that is to say, aimed to check the result of implementation of innovations and to monitor these; definitively, far from any aim of diagnosis. However, some of these concerning compensatory education may provide interesting data as to quantitative and qualitative schooling deficiencies among some fringe populations (Gypsies, immigrants, refugees). For similar reasons, one must bear in mind studies on intercultural education.


Graph 6: Number of research studies on inequality in education by specific theme


SCHOOL FUNCTIONING

The sample taken in the reference document includes 19 research studies that may fall under this heading. The majority are related to matters of management and participation and almost always consist of studies of a descriptive nature. Ángel Lázaro considers that these are studies which "have hardly influenced the administrative and legal decisions related to organisational and pedagogical functioning of schools", adding that "it is more probable that diverse social groups (associations of parents, Trade Union organisations) have had more influence on the management tendencies, embodied in legal regulations, than the actual results of research into the management function of schools".

This heading may also include some studies on school financing, that undoubtably provide most interesting data on the cost of each place at school.


THE TEACHING PROFESSION

Graph 7 shows the research efforts dedicated to the teaching staff, in which one may see which aspects were preferred.


Graph 7: Number of research studies on teaching staff by specific theme

We see the most outstanding is the one on training, with particular reference to initial training. The studies clearly show how many of the training problems still detected by teachers have been dragged down over the years such as, for example, that concerning the scarce amount of practical training, the deficient link between this and theory, etc. The following ones in number of studies are those related to action by the teaching staff, undoubtably of great interest for decisions as to possible modification. The greater or lesser extent of professional satisfaction among the teaching staff has also been taken the subject for interesting studies, showing diverse manifestations of what is sometimes known as "teacher’s uneasiness". Particularly in recent years, research has frequently been aimed at detecting teacher’s attitudes, either in general (emphasising, for example, the frequent resistance to innovations), or such matters as sexism, integration of students with difficulties, etc.


EDUCATION AND SOCIETY

The set of research studies shown on graph 1 under the heading education and society does not initially seem as large as one might have expected. The studies mainly (two out of three) deal with the relations between the educational system and labour market, while there are few dedicated to family and education, that, moreover, concentrate almost exclusively on kindergarten level.



PRECEDING EVALUATION STUDIES CARRIED OUT BY THE INCE

Since it began its activities in 1994, the National Quality and Evaluation Institute started a programme of actions aimed at providing Spanish society solvent, periodic evaluations of the educational system overall and the different levels that comprise it. This is not the place to describe that programme which, on the other hand, is easily accessible to anyone who is interested in it, but rather to make a concise reference to the actions that may be a precedent, to a certain extent, to the effort of diagnosis now being carried out. We shall make specific reference to three precise lines of action: Evaluation of Primary Education, collaboration in preparing the International Indicators set by the OECD and, lastly, The International Study on Teaching English as a Second Language. In the three cases, the only aim is to emphasise the data that may be significant for better understanding of the new diagnosis.


EVALUATION OF PRIMARY EDUCATION

The aim of the study was to ascertain the results obtained by students aged 12 (6th of EGB) in tests set for this purpose in the fields of Language, Mathematics, Social Sciences and Natural Sciences. These tests were administered to 10,500 students from a sample 438 schools distributed according to three variables: ownership, size and Autonomous Region. The study also used questionnaires for parents, teachers and principals and tests for the students in the first cycle of Primary Education, to ascertain the effects of implementation of the new system in the educational processes and school operation.

Among the very varied conclusions of the study, the final Report of which was published by the INCE, one must emphasise the following:

  • The average percentage of correct answers per student in the Mathematics test, considered globally, 50.1%; 64% in Language; 60.6% in Social Science; 63.6% in Natural Sciences. The traditional difficulty the students have in Mathematics and also in other studies was confirmed.
  • The average performance by girls was higher than that for boys in Language and Mathematics. the students who began schooling at the age of three have a higher average performance in all the subjects. Students whose families have a large number of books obtain better results.
  • Student performance drops in the following cases: bad relation with parents, less interest of parents in their children’s studies, untrusting relationship between parents and their offspring.
  • Students who never or almost never do homework have a lower performance than those who frequently do.
  • The levels of teacher satisfaction as to their relations at school coincide with higher student results.
  • The teaching staff calls for training in attention to diversity and syllabus adaptation.
  • Families have a special appraisal of providing specialist music, physical education and foreign language teachers as novelties in the system.
  • 70% of the parents state they are satisfied with the relation they have with their children concerning their education. Moreover, mothers are more involved than fathers in supporting their children’s education.
  • The teaching staff is satisfied to practice their profession, but less so with the support they say they receive from the relevant educational authorities. On the other hand, they mainly agree that it is necessary to remain at a school for more than two years to achieve sufficient levels of cohesion.
  • Most of the principals support formulas to professionalise the management function; most of the teachers do not. Neither one nor the other support creation of a specific body (in the case of state schools).


THE INES PROJECT BY THE OECD

In spite of the long tradition in collecting statistical information on the different aspects of educational systems in OECD countries, when the matter of the quality of education and the efficiency of educational systems began to become a key aspect in most countries, the dilemma of how to provide those responsible for educational decision making a type of useful information to plan the educational offer, so efficient steps are taken on that road to quality. In this sense, since the beginning of the eighties, the demand for data to make international comparisons between educational systems has increased as, moreover, such comparisons also provide very valuable national information.

This demand for information gave rise to many technical problems, so it led the authorities of the OECD countries to consider new ways of comparing their educational systems. Due to this, an agreement was reached to prepare an international set of indicators that would show the main aspects of the educational systems in the member countries, on the basis of high quality statistical data, to allow comparative monitoring of their evolution.

The Centre for Educational Research and Innovation (CERI) at the OECD responded to this demand for compared information by launching the International Indicators of Educational Systems (INES Project), which was planned in three phases: exploratory phase (between 1988-1991); a start-up phase of the project to set up the operational networks (1991-96), in which it aimed to prepare and propose a system of educational indicators; and the definitive phase, in which it aims to produce an annual publication with the constantly updated data from the different countries with the definitive indicators.

The indicators will provide information that is regularly updated on organisation and functioning of the educational systems and provide information on the way the systems react to changes in policy priorities and the present development in society. That information is published in a work called Education at a Glance, that has been produced annually since 1992.

To carry out this work, the CERI established an international consulting mechanism - the Technical Group and the Operational Networks - to exchange points of view and reach a common conception of matters related to definition, measurement and organisation of the indicators. These bodies cooperate with the OECD Secretariat to develop indicators of common interest, according to the political priorities of the member countries and the conceptual framework prepared by the INES Project. Since it was founded, the INCE has been the Spanish institution in charge of collaboration with the OECD in this task.

The last edition of Education at a Glance published indicators of the following educational fields:

  • Demography, social and economic context of education
  • Human and financial resources invested in education
  • Access to education, participation and progress
  • The setting of learning and school organisation
  • Social and labour results of education
  • Student performance
  • Number of graduates at the different educational levels

This set of indicators aims to provide a panoramic view of the comparative functioning of the educational systems (through reflection on the context and resources, as well as on the processes and results), to allow each educational system to find its weaknesses, as well as to identify the aspects that work best and may go unnoticed in internal debates.

Finally, it is necessary to bear in mind that the future of the INES will not consist only on the preparation of a limited number of indicators, but rather its framework will be extended to also include a set of related activities, among which links will be established to provide a unity to the project: the International Adult Literacy Study (on the level of literacy of the adult population), investment in education, the transition from school to active life, etc.


INTERNATIONAL STUDY ON ENGLISH LANGUAGE TEACHING AND LEARNING IN SECONDARY EDUCATION

The Directorate for Evaluation and Prospective (DEP) at the National Education Ministry of France, the National Education Agency (SKOLVERKET) from Sweden and the National Institute for Quality and Evaluation (INCE) from the Ministry of Education and Culture of Spain participated in in this study.

The first objective of this international evaluation study was to evaluate the English language knowledge and skills among students aged 15 and 16 in those three countries in the European Union, in addition to obtaining information from each of the three countries on the efficiency of English language teaching in their educational systems, show the convergences and divergencies of the pedagogical option adopted by each of these countries as to the content of the programmes, pedagogical practices of the teachers and the students’ process of learning English, and provide data that may be useful as the basis, in each of the three countries, for the necessary decision making to develop efficient English teaching.

There were two key moments in organisation of the study. First of all, the DEP proposed that the INCE participate in a bilateral study to allow, on one hand, checking of the results of students at the end of the obligatory stage using a linguistic proficiency test that France had prepared for use in general and periodic evaluation of its educational system, which took place in June 1995; and on the other hand, obtain data on the factors that would influence English language learning, so two questionnaires were prepared in Spain, one for the students and the other for the teaching staff, which were administered with the test in April 1996.

Sweden later decided to participate in the study, so in 1996 it administered a specific linguistic skill test with France for this evaluation, featuring several test questions previously used in Spain and France, the same as several questions in the student and teacher questionnaires designed by those responsible for the study at the INCE. Thus, some of the data Spain obtained from the international study not only could be compared with that from France, but also with that from Sweden, as the three countries had a common core in the performance test as well as in the student and teacher questionnaires.

The results obtained as to performance by the students shows a great similarity between French and Spanish students. The average percentage of correct answers overall in the performance test was 60% for Spanish students and 59% for the French ones. As to the different blocks of content forming the test, the French students obtained an average percentage of correct answers in linguistic knowledge of 65% and the Spanish students 64%. In written comprehension, the Spanish students obtained 65% of right answers and the French ones 61%. In oral comprehension the Spanish students obtained 49% of right answers and the French students 47%. In written expression, the French students obtained 41% of right answers and the Spanish students 39% of right answers.

As to the results obtained from the common core of the performance test for France, Spain and Sweden, the average mark of right answers was 61% for Swedish students, 48% for French students and 49% for Spanish students. In the two blocks of contents where equal items were included for the three countries, the average percentages of correct answers were, as to linguistic knowledge, 61% of right answers for the Swedish students, 52% for Spanish students and 50% for the French. In the written expression block, the Swedish students obtained 60% of right answers, the Spanish 35% and the French 37%.


Previous Table of Contents Next


© Instituto Nacional de Calidad y Evaluación (INCE)
calle San Fernando del Jarama, 14
E28002 MADRID
Tel: +34 91.745.9200
Fax: +34 91.745.9249
email: info@ince.mec.es