In early December, the results of the international study on the quality of education PISA were published, which in 2018 was joined by almost 80 countries, including Ukraine for the first time. According to the rating, Ukraine is in the middle of the list of participating countries with results of 466 points in reading, 453 points in mathematics and 469 points in natural sciences (the best indicator in each industry is defined as 1000 points). But what are these results talking about and with what sign should they be taken - let's figure it out together
What is PISA?
The Program for International Student Assessment (PISA) is an international assessment that measures 15-year-old students' reading, mathematics, and science literacy every three years. First conducted in 2000, the major domain of study rotates between reading, mathematics, and science in each cycle. PISA also includes measures of general or cross-curricular competencies, such as collaborative problem-solving. By design, PISA emphasizes functional skills that students have acquired as they near the end of compulsory schooling.
PISA is coordinated by the Organization for Economic Cooperation and Development (OECD), an intergovernmental organization of industrialized countries, and is conducted in the United States by NCES.
Emerging research suggests that international standardized assessments are having an impact on national assessment policy and practice. PISA is being integrated into national policies and practices on assessment, evaluation, curriculum standards, and performance targets; its assessment frameworks and instruments are being used as best-practice models for improving national assessments; many countries have explicitly incorporated and emphasized PISA-like competencies in revised national standards and curricula; others use PISA data to complement national data and validate national results against an international benchmark.
Each student takes a two-hour handwritten test. Part of the test is multiple-choice and part involves fuller answers. There are six and a half hours of assessment material, but each student is not tested on all the parts. Following the cognitive test, participating students spend nearly one more hour answering a questionnaire on their background including learning habits, motivation, and family.
The National Report on the results of the international study of the quality of education PISA-2018 notes that the main goal of the Ministry of Education and Science of Ukraine is to create opportunities for the development of talent of each person, which should not depend on social status, place of birth and origin. Everyone should have access to professional teachers, a modern educational environment and the current content of education throughout life (which is also regulated by the Law of Ukraine "On Education" from 09/05/2017).
According to Minister of Education Anna Novosad, the achievement of these goals is possible only if the policy of the Ministry is based on real measured data, and not on assumptions. Therefore, full participation along with other countries in PISA-2018 should be an important step in understanding the weak links in the education system for the introduction of qualitative changes.
Good or bad
If we look at the table of results of each of the countries in comparison with Ukraine, even without being an expert, we can note that we are significantly behind the average indicators in OECD countries:
So, the average reading score for the OECD countries is 488.89 points (555 for the leader country), 492.03 points for mathematics (591 highest score), 490.78 points for natural sciences (leader country 590). The difference in the success of Ukrainian pupils/students compared with these indicators is 23, 39 and 22 points, respectively.
In addition, the study presents a rating that identifies the positions of countries (the range using extreme midpoints, given the possible statistical error) in each of the industries. For example, out of 79 participating countries, Ukraine ranks 36–41 in reading, 41–46 in mathematics, and 36–42 in natural sciences.
Commenting on these indicators during the presentation of the National Report on the results of the study, Anna Novosad noted: “I read the report only once, but this is enough to get the so-called PISA shock and see the problems that exist in our system. Although in general, our results are not as bad as some expected.”
The minister also published a summing up message on her Facebook page, where she highlighted the main problems that require an urgent solution:
- The most problematic area is mathematics;
- Since, according to the OECD, 30 points correspond to one year of study in a general secondary education institution, it can be concluded that in mathematics, Ukrainian students are behind their peers from other countries by more than a year.
- The significant difference in student/student achievement depends on the area;
- In large cities, the average score is 499.4, which differs significantly from the average score of students living in towns - 479.9. The average scores of students who live in villages and rural areas are significantly different from all. The average score for such students is 420.6. This makes a difference with major cities in more than 2.5 years of study. 2.5 years.
- According to the adviser to the Minister of Education and Science of Ukraine Ivanna Kobernyk (in a commentary for DW), the problem of inequality in access to better education depending on the place of residence is also demonstrated annually by the Ukrainian External Independent Assessment (UPE): “City students show better results than students from rural areas.”
- A significant percentage of pupils/students with reading, math, and science literacy are below the basic level (the minimum level of competency necessary in life) and a small percentage of pupils / students who have achieved a high level of literacy (according to the 6-point scale of student success rating 5-6- d - the level of participants in the olympiads) in at least one of the PISA industries;
- Boys read much less than girls;
The average reading performance among children is lower (450.1 points) than among girls (483.6 points). In Ukraine, the gap (33.5 points) is larger than the OECD average of 30.1 points. This is a gap of more than one year of study.
Participation in the PISA-2018 International Study of the Quality of Education has become a point of reference. Indeed, now the reform has affected only 2 classes of elementary school, but already in the spring of next year, the Ministry of Education and Science promises to present the basic standard of school education. He will take into account all the errors that were identified in the study. Moreover, the solution to some of them lay on the surface, as well as the answer to the question: "What is the reason for such results relative to other countries?"
Due to the fact that the study makes it possible to see the results of students not only regarding their peers in other countries, but also among compatriots, it was discovered that the most important reason for such results, which requires an urgent solution, is that in Ukraine the achievements of pupils/students depend not only from the locality (big cities, small towns, and villages) but also from the institution in which the pupil or student receives education (there is a significant gap (up to 2 years of study) between the average student outcomes secondary specialized schools and the results of elite special education institutions).
Another example - the same unpleasant results of Ukrainian teenagers in mathematics could be different if the Ukrainian textbooks set before students more realistic, life-like tasks. Indeed, if we compare the average Ukrainian task from the textbook with the analog of the PISA task of the past years, we can conclude that the domestic approach is not really progressive - problems of this type do not occur in real life. Only 38% of Ukrainian 15-year-old students succeeded in solving PISA problems.
Based on the results, the Ministry of Education and Science promised to finalize the state standard for basic education (grades 5–9). But this will only affect those who went to school since 2018.
Moreover, many of the parents insist on changes for grades 1-4, noting that sometimes there are tasks that are too confusing and completely not close to real-life situations.
In order to understand that the situation is not really critical, and Ukraine, after working on the mistakes, will have every chance of becoming one of the five leaders in the rating in all sectors, it is worth giving the example of Estonia, which in the PISA-2018 ranking is the first in Europe in all three directions of research. Moreover, it spends 30% less on education than OECD countries (high-income countries).
“Estonia has very few students who do not reach the basic (2nd) level in PISA, but there are a significant number of children who reach the 5-6th level (in our opinion, these are national Olympiads participants). I quote this fact for those who scary - if the country focuses on weaker children, it will reduce the level of strong ones. PISA proves that this is not so if the state has a reasonable educational policy,” Ivanna Kobernyk notes.
The next PISA study will take place in 2021. 88 countries should become participants in the new cycle, and the organizers promise to expand the list of measured competencies, adding creative thinking to it (the main areas are mathematical literacy and creative thinking).
Compared to other countries, Ukraine shows good results in this aspect - the satisfaction index for reading Ukrainian teenagers was 0.28 (higher than the average in OECD countries). Based on this index, it was determined that in Ukraine, 62.7% of pupils/students read for pleasure and 37.3% - only when they need to get information.