Skip to main content
SpringerLink
Log in

Change in test-taking motivation and its relationship to test performance in low-stakes assessments

  • Published:
Educational Assessment, Evaluation and Accountability Aims and scope Submit manuscript

An Erratum to this article was published on 19 November 2016

Abstract

Since the turn of the century, an increasing number of low-stakes assessments (i.e., assessments without direct consequences for the test-takers) are being used to evaluate the quality of educational systems. Internationally, research has shown that low-stakes test results can be biased due to students’ low test-taking motivation and that students’ effort levels can vary throughout a testing session involving both cognitive and noncognitive tests. Thus, it is possible that students’ motivation varies throughout a single cognitive test and in turn affects test performance. This study examines the change in test-taking motivation within a 2-h cognitive low-stakes test and its association with test performance. Based on expectancy-value theory, we assessed three components of test-taking motivation (expectancy for success, value, and effort) and investigated its change. Using data from a large-scale student achievement study of German ninth-graders, we employed second-order latent growth modeling and structural equation modeling to predict test performance in mathematics. On average, students’ effort and perceived value of the test decreased, whereas expectancy for success remained stable. Overall, initial test-taking motivation was a better predictor of test performance than change in motivation. Only the variability of change in the expectancy component was positively related to test performance. The theoretical and practical implications for test practitioners are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Due to the nonsignificant, slightly negative residual variances of some first-order factors in the second-order latent growth models, we had to fix some of the residual variances of the first-order factors to zero: for effort and importance for the first and third time point, and for probability of success for the third time point. An investigation of the residual variances using latent growth modeling with a composite of the manifest indicators per time point (instead of a latent variable) showed that these residual variances were close to zero. This supported our decision to fix the corresponding residual variances to zero.

  2. Beta refers to the stdyx standardization in the Mplus output using full standardization with respect to both latent and observed variables.

References

  • Asseburg, R. (2011). Motivation zur Testbearbeitung in adaptiven und nicht-adaptiven Leistungstests [Test-taking motivation in adaptive and sequential achievement testing] (Doctoral dissertation). Christian-Albrechts-Universität zu Kiel. Retrieved from the website http://d-nb.info/1013153863/34.

  • Barry, C. L., & Finney, S. J. (2016). Modeling change in effort across a low-stakes testing session: a latent growth curve modeling approach. Applied Measurement in Education, 29(1), 46–64. doi:10.1080/08957347.2015.1102914.

    Article  Google Scholar 

  • Barry, C. L., Horst, S. J., Finney, S. J., Brown, A. R., & Kopp, J. P. (2010). Do examinees have similar test-taking effort? A high-stakes question for low-stakes testing. International Journal of Testing, 10(4), 342–363. doi:10.1080/15305058.2010.508569.

    Article  Google Scholar 

  • Baumert, J., & Demmrich, A. (2001). Test motivation in the assessment of student skills: the effects of incentives on motivation and performance. European Journal of Psychology of Education, 16(3), 441–462.

    Article  Google Scholar 

  • Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.

    Book  Google Scholar 

  • Browne, M. W., & Du Toit, S. H. C. D. (1992). Automated fitting of nonstandard models. Multivariate Behavioral Research, 27(2), 269–300. doi:10.1207/s15327906mbr2702_13.

    Article  Google Scholar 

  • Cao, J., & Stokes, S. L. (2008). Bayesian IRT guessing models for partial guessing behaviors. Psychometrika, 73(2), 209–230. doi:10.1007/s11336-007-9045-9.

    Article  Google Scholar 

  • Chen, S.-K., Yeh, Y.-C., Hwang, F.-M., & Lin, S. S. J. (2013). The relationship between academic self-concept and achievement: a multicohort–multioccasion study. Learning and Individual Differences, 23, 172–178. doi:10.1016/j.lindif.2012.07.021.

    Article  Google Scholar 

  • Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. doi:10.1207/S15328007SEM0902_5.

    Article  Google Scholar 

  • Cole, J. S., Bergin, D. A., & Whittaker, T. A. (2008). Predicting student achievement for low stakes tests with effort and task value. Contemporary Educational Psychology, 33(4), 609–624.

    Article  Google Scholar 

  • DeMars, C. E., Bashkov, B. M., & Socha, A. B. (2013). The role of gender in test-taking motivation under low-stakes conditions. Research & Practice in Assessment, 8, 69–82.

    Google Scholar 

  • Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53(1), 109–132. doi:10.1146/annurev.psych.53.100901.135153.

    Article  Google Scholar 

  • Eklöf, H. (2006). Development and validation of scores from an instrument measuring student test-taking motivation. Educational and Psychological Measurement, 66(4), 643–656. doi:10.1177/0013164405278574.

    Article  Google Scholar 

  • Eklöf, H. (2007). Test-taking motivation and mathematics performance in TIMSS 2003. International Journal of Testing, 7(3), 311–326.

    Article  Google Scholar 

  • Eklöf, H. (2008). Test-taking motivation on low-stakes tests: A Swedish TIMSS 2003 example. In Issues and methodologies in large-scale assessments, IERI Monograph Series (Vol. 1, pp. 9–21). Hamburg: IEA-ETS Research Institute.

  • Eklöf, H. (2010a). Skill and will: test‐taking motivation and assessment quality. Assessment in Education: Principles, Policy & Practice, 17(4), 345–356. doi:10.1080/0969594X.2010.516569.

    Article  Google Scholar 

  • Eklöf, H. (2010b). Student motivation and effort in the Swedish TIMSS Advanced field study. Presented at the meeting of the 4th IEA International Research Conference, Gothenburg.

  • Eklöf, H., & Nyroos, M. (2013). Pupil perceptions of national tests in science: perceived importance, invested effort, and test anxiety. European Journal of Psychology of Education, 28(2), 497–510. doi:10.1007/s10212-012-0125-6.

    Article  Google Scholar 

  • Eklöf, H., Pavešič, B. J., & Grønmo, L. S. (2013). A cross-national comparison of reported effort and mathematics performance in TIMSS Advanced. Applied Measurement in Education, 131127082739006. doi:10.1080/08957347.2013.853070.

  • Ferrer, E., Balluerka, N., & Widaman, K. F. (2008). Factorial invariance and the specification of second-order latent growth models. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 4(1), 22–36. doi:10.1027/1614-2241.4.1.22.

    Article  Google Scholar 

  • Freund, P. A., & Holling, H. (2011). Who wants to take an intelligence test? Personality and achievement motivation in the context of ability testing. Personality and Individual Differences, 50(5), 723–728. doi:10.1016/j.paid.2010.12.025.

    Article  Google Scholar 

  • Freund, P. A., Kuhn, J. T., & Holling, H. (2011). Measuring current achievement motivation with the QCM: short form development and investigation of measurement invariance. Personality and Individual Differences, 51(5), 629–634. doi:10.1016/j.paid.2011.05.033.

    Article  Google Scholar 

  • Frey, A., Hartig, J., & Rupp, A. A. (2009). An NCME instructional module on booklet designs in large-scale assessments of student achievement: theory and practice. Educational Measurement: Issues and Practice, 28(3), 39–53. doi:10.1111/j.1745-3992.2009.00154.x.

    Article  Google Scholar 

  • Ganzeboom, H. B. G., De Graaf, P. M., & Treiman, D. J. (1992). A standard international socio-economic index of occupational status. Social Science Research, 21(1), 1–56. doi:10.1016/0049-089X(92)90017-B.

    Article  Google Scholar 

  • Geiser, C., Keller, B. T., & Lockhart, G. (2013). First- versus second-order latent growth curve models: some insights from latent state-trait theory. Structural Equation Modeling: A Multidisciplinary Journal, 20(3), 479–503. doi:10.1080/10705511.2013.797832.

    Article  Google Scholar 

  • Hancock, G. R., Kuo, W.-L., & Lawrence, F. R. (2001). An illustration of second-order latent growth models. Structural Equation Modeling: A Multidisciplinary Journal, 8(3), 470–489. doi:10.1207/S15328007SEM0803_7.

    Article  Google Scholar 

  • Horst, S. J. (2010). A mixture-modeling approach to exploring test-taking motivation in large-scale low-stakes contexts (Unpublished doctoral dissertation). Harrisonburg: James Madison University.

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.

    Article  Google Scholar 

  • Jerrim, J., & Micklewright, J. (2014). Socio-economic gradients in children’s cognitive skills: are cross-country comparisons robust to who reports family background? European Sociological Review, 30(6), 766–781. doi:10.1093/esr/jcu072.

    Article  Google Scholar 

  • Kong, X. J., Wise, S. L., Harmes, J. C., & Yang, S. (2006). Motivational effects of praise in response-time-based feedback: A follow-up study of the effort-monitoring CBT. Presented at the annual meeting of the National Council on Measurement in Education, San Francisco.

  • Lau, A. R., Swerdzewski, P. J., Jones, A. T., Anderson, R. D., & Markle, R. E. (2009). Proctors matter: strategies for increasing examinee effort on general education program assessments. The Journal of General Education, 58(3), 196–217. doi:10.1353/jge.0.0045.

    Article  Google Scholar 

  • Moneta, G. B., & Csikszentmihalyi, M. (1996). The effect of perceived challenges and skills on the quality of subjective experience. Journal of Personality, 64(2), 275–310. doi:10.1111/j.1467-6494.1996.tb00512.x.

    Article  Google Scholar 

  • Muthén, L. K., & Muthén, B. O. (1998). Mplus user’s guide (7th ed.). Los Angeles: Muthén & Muthén.

    Google Scholar 

  • Pant, H. A., Stanat, P., Schroeders, U., Roppelt, A., Siegle, T., & Pöhlmann, C. (2013). The IQB National Assessment Study 2012. Competencies in mathematics and the sciences at the end of secondary level I. Summary. Münster: Waxmann. Retrieved from http://www.iqb.hu-berlin.de/laendervergleich/laendervergleich/lv2012/Bericht/IQB_NationalAsse.pdf.

  • Pekrun, R., Elliot, A. J., & Maier, M. A. (2009). Achievement goals and achievement emotions: testing a model of their joint relations with academic performance. Journal of Educational Psychology, 101(1), 115–135. doi:10.1037/a0013383.

    Article  Google Scholar 

  • Penk, C., & Schipolowski, S. (2015). Is it all about value? Bringing back the expectancy component to the assessment of test-taking motivation. Learning and Individual Differences, 42, 27–35. doi:10.1016/j.lindif.2015.08.002.

    Article  Google Scholar 

  • Penk, C., Pöhlmann, C., & Roppelt, A. (2014). The role of test-taking motivation for students’ performance in low-stakes assessments: an investigation of school-track-specific differences. Large-Scale Assessments in Education, 2(1). doi:10.1186/s40536-014-0005-4

  • Preacher, K. J., Wichmann, A. L., MacCallum, R. C., & Briggs, N. E. (2008). Latent growth curve modeling. Los Angeles: Sage.

    Book  Google Scholar 

  • Ramm, G., Prenzel, M., Baumert, J., Blum, W., Lehmann, R., Leutner, D., … Schiefele, U. (2006). PISA 2003: Dokumentation der Erhebungsinstrumente [Documentation of the assessment instruments]. Münster: Waxmann.

  • Rheinberg, F., Vollmeyer, R., & Burns, B. D. (2001). FAM: Ein Fragebogen zur Erfassung aktueller Motivation in Lern-und Leistungssituationen [A questionnaire for the measurement of current achievement motivation in learning and achievement situations]. Diagnostica, 47(2), 57–66.

    Article  Google Scholar 

  • Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys. Educational and Psychological Measurement, 74(1), 31–57. doi:10.1177/0013164413498257.

    Article  Google Scholar 

  • Sayer, A. G., & Cumsille, P. E. (2001). Second-order latent growth models. In L. M. Collins & A. G. Sayer (Eds.), New methods for the analysis of change (1st ed., pp. 179–200). Washington: American Psychological Association.

    Chapter  Google Scholar 

  • Schunk, D. H., Pintrich, P. R., & Meece, J. L. (2008). Motivation in education: theory, research, and applications (3rd ed.). Upper Saddle River: Pearson Education.

    Google Scholar 

  • Spinath, B. (2012). Academic achievement. In V. S. Ramachandran (Ed.), Encyclopedia of human behavior (pp. 1–8). London: Elsevier/Academic.

    Chapter  Google Scholar 

  • Stanat, P., & Christensen, G. (2006). Where immigrant students succeed: a comparative review of performance and engagement in PISA 2003. Paris: Organisation for Economic Co-operation and Development.

    Google Scholar 

  • Stanat, P., & Lüdtke, O. (2013). International large-scale assessment studies of student achievement. In J. Hattie & E. M. Anderman (Eds.), International guide to student achievement (pp. 481–483). New York: Routledge.

    Google Scholar 

  • Sundre, D. L. (2007). The Student Opinion Scale: a measure of examinee motivation: test manual. Retrieved from the Center for Assessment and Research Studies website: http://www.jmu.edu/assessment/resources/resource_files/sos_manual.pdf.

  • Sundre, D. L., & Kitsantas, A. (2004). An exploration of the psychology of the examinee: can examinee self-regulation and test-taking motivation predict consequential and non-consequential test performance? Contemporary Educational Psychology, 29(1), 6–26. doi:10.1016/S0361-476X(02)00063-2.

    Article  Google Scholar 

  • Swerdzewski, P. J., Harmes, J. C., & Finney, S. J. (2011). Two approaches for identifying low-motivated students in a low-stakes assessment context. Applied Measurement in Education, 24(2), 162–188. doi:10.1080/08957347.2011.555217.

    Article  Google Scholar 

  • Thelk, A. D., Sundre, D. L., Horst, S. J., & Finney, S. J. (2009). Motivation matters: using the Student Opinion Scale to make valid inferences about student performance. The Journal of General Education, 58(3), 129–151. doi:10.1353/jge.0.0047.

    Article  Google Scholar 

  • Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54(3), 427–450.

    Article  Google Scholar 

  • Wigfield, A. (1994). Expectancy-value theory of achievement motivation: a developmental perspective. Educational Psychology Review, 6(1), 49–78. doi:10.1007/BF02209024.

    Article  Google Scholar 

  • Wigfield, A., & Eccles, J. S. (2000). Expectancy–value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81. doi:10.1006/ceps.1999.1015.

    Article  Google Scholar 

  • Wise, S. L. (2006). An investigation of the differential effort received by items on a low-stakes computer-based test. Applied Measurement in Education, 19(2), 95–114. doi:10.1207/s15324818ame1902_2.

    Article  Google Scholar 

  • Wise, S. L., & DeMars, C. E. (2005). Low examinee effort in low-stakes assessment: problems and potential solutions. Educational Assessment, 10(1), 1–17. doi:10.1207/s15326977ea1001_1.

    Article  Google Scholar 

  • Wise, S. L., & Kong, X. J. (2005). Response time effort: a new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183. doi:10.1207/s15324818ame1802_2.

    Article  Google Scholar 

  • Wise, S. L., & Smith, L. F. (2011). A model of examinee test-taking effort. In J. A. Bovaird, K. F. Geisinger, & C. W. Buckendahl (Eds.), High-stakes testing in education: science and practice in K-12 settings (1st ed., pp. 139–153). Washington: American Psychological Association.

    Chapter  Google Scholar 

  • Wise, S. L., Bhola, D. S., & Yang, S. (2006). Taking the time to improve the validity of low-stakes tests: the effort-monitoring CBT. Presented at the annual meeting of the National Council on Measurement in Education, San Francisco.

  • Wise, S. L., Pastor, D. A., & Kong, X. J. (2009). Correlates of rapid-guessing behavior in low-stakes testing: implications for test development and measurement practice. Applied Measurement in Education, 22(2), 185–205. doi:10.1080/08957340902754650.

    Article  Google Scholar 

  • Wolf, L. F., & Smith, J. K. (1995). The consequence of consequence: motivation, anxiety, and test performance. Applied Measurement in Education, 8(3), 227–242. doi:10.1207/s15324818ame0803_3.

    Article  Google Scholar 

  • Zilberberg, A., Finney, S. J., Marsh, K. R., & Anderson, R. D. (2014). The role of students’ attitudes and test-taking motivation on the validity of college institutional accountability tests: a path analytic model. International Journal of Testing, 14(4), 360–384. doi:10.1080/15305058.2014.928301.

    Article  Google Scholar 

Download references

Acknowledgments

We thank Sara J. Finney for her enriching comments and methodological support as well as Bo Bashkov for proofreading the manuscript.

Author information

Authors and Affiliations

  1. German Institute for International Educational Research (DIPF), Warschauer Str. 34-38, 10243, Berlin, Germany

    Christiane Penk

  2. University of Potsdam, Potsdam, Germany & Institute for Educational Quality Improvement (IQB), Berlin, Germany

    Dirk Richter

Authors
  1. Christiane Penk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dirk Richter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christiane Penk.

Additional information

The original version of this article was revised: There is a typographical error in the acknowledgements section. The name should be "Sara J. Finney" and not "Sara J. Fin ney". Also, in the Appendix section, the columns in Table 5 were switched.

An erratum to this article is available at http://dx.doi.org/10.1007/s11092-016-9249-6.

Appendices

Appendix 1

Table 4 Test of the strong measurement invariance test of test-taking effort and probability of success, with autocorrelated errors
Full size table

Appendix 2

Table 5 Correlations of the growth parameters for effort, importance, probability of success, and self-concept in mathematics: the indirect effects and nonsignificant effects for model 2
Full size table
Table 6 Correlations of the growth parameters for effort, importance, probability of success, and self-concept in mathematics with the background variables for model 2
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Penk, C., Richter, D. Change in test-taking motivation and its relationship to test performance in low-stakes assessments. Educ Asse Eval Acc 29, 55–79 (2017). https://doi.org/10.1007/s11092-016-9248-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11092-016-9248-7

Keywords

Search

Navigation