Det första nationella provet i samhällskunskap - en studie i bedömarsamstämmighet
DOI:
https://doi.org/10.5617/adno.6283Keywords:
Samhällskunskap, nationella prov, interbedömarreliabilitet, intraclass correlation.Abstract
Skolämnet samhällskunskap som eget ämne existerar i princip enbart i de nordiska länderna. I många andra länder delar flera skolämnen på ämnesinnehållet, till exempel geografi och civics. Ämnesinnehållet är stort och genomgår ständig förändring. År 2013 genomfördes de första nationella proven i samhällskunskap i Sverige för årskurs 9. Med tanke på ämnets karaktär kan det vara speciellt viktigt att undersöka om dessa prov är ”rättvisa.” Avsikten med denna studie är att undersöka en aspekt av denna ”rättvisa”, nämligen interbedömarstabilitet, dvs om samma elevsvar ger upphov till samma bedömning oavsett bedömare. Skolverket i Sverige genomförde 2009 en större studie av de ämnen som då genomförde nationella prov och föreliggande studie försöker dels efterlikna och dels bygga ut upplägget från Skolverket. Studien genomfördes på de första nationella proven i samhällskunskap 2013. Genom att pröva olika reliabilitetsmått inom kategorierna ”consensus estimates”, och ”consistency estimates” analyseras resultaten, bland annat diskuteras måttet intraclass correlation. Syftet är också, då detta var de första proven, att skapa en ram för återkommande studier av Interbedömarreliabilitet. Upplägget med en större mängd lärare som genomför totalt tre bedömningar av de utvalda hela proven försöker också efterlikna bedömningssituationen ute på skolorna såtillvida att det var relativt många lärare med i studien, och de kom från olika skolor spridda över Sverige. Genom detta testas också bedömningsanvisningarnas stabilitet. Själva genomförandet var omfattande och tog två hela dagar. Resultaten pekar på en god överensstämmelse för provbetyget, det sammanfattande omdöme eleverna får. Studien avses att återupprepas under kommande år.
Nyckelord: Samhällskunskap, nationella prov, interbedömarreliabilitet, intraclass correlation
The first national test in samhällskunskap – a study of interrater reliability
Abstract
The Swedish school subject Samhällskunskap (Societal knowledge) exists basically only in the Nordic countries. In other countries a number of different subjects, such as geography and civics, share the content. The content of the subject is constantly changing, depending on how society is changing. The first national tests in Samhällskunskap for all Swedish ninth graders took place in 2013. A large part of the test contains constructed responses. Given the characteristics of the subject we consider it especially important to investigate whether these tests are “fair” or not. The intent of this study is to investigate one aspect of “fairness”, interrater reliability, meaning the degree to which the same student responses are scored equally by different raters. In 2009, the National Agency of Education in Sweden conducted a large study of the subjects Swedish, English and Mathematics. Our study aims to mimic and further develop the design of the study from 2009. Our study was carried out on the first national tests in 2013. The results were analyzed by exploring different reliability measures within the categories consensus estimates, and consistency estimates.
As the 2013 tests were the first tests of its kind in Sweden the purpose was also to create a framework for regular studies of interrater reliability. The rater design with a relatively large number of teachers from all over the country, each assessing a total of three complete student test responses aimed at mimicking the way the tests are assessed in schools. This also allowed us to study the stability of our assessment rubrics. The study itself was extensive and took two days to perform. The results indicate a large compliance when it comes to the final grade of the test. The study is meant to be repeated in the coming years.
Keywords: Social science, civics, national testing, interrater reliability, intraclass correlation
Downloads
Published
How to Cite
Issue
Section
License
Content published in Acta Didactica is - unless otherwise is stated - licensed through Creative Commons License BY-NC-ND 4.0. Content can be copied, distributed and disseminated in any medium or format under the following terms:
Attribution: You must give appropriate credit and provide a link to the license
Non-Commercial: You may not use the material for commercial purposes.
No derivatives: If you remix, transform, or build upon the material, you may not distribute the modified material.
No additional restrictions: You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
Notice: No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material.
Authors who publish in Acta Didactica accept the following conditions:
Author(s) retains copyright to the article and give Acta Didactica rights to first publication while the article is licensed under the Creative Commons CC BY-NC-ND 4.0. This license allows sharing the article for non-commercial purposes, as long as the author and first publishing place Acta Didactica are credited.
The author is free to publish and distribute the work/article after publication in Acta Didactica, as long as the journal is referred to as the first place of publication. Submissions that are under consideration for publication or accepted for publication in Acta Didactica cannot simultaneously be under consideration for publication in other journals, anthologies, monographs or the like. By submitting contributions, the author accepts that the contribution is published online in Acta Didactica.