Skip to main content
SpringerLink
Log in

Does syntax highlighting help programming novices?

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Program comprehension is an important skill for programmers – extending and debugging existing source code is part of the daily routine. Syntax highlighting is one of the most common tools used to support developers in understanding algorithms. However, most research in this area originates from a time when programmers used a completely different tool chain. We examined the influence of syntax highlighting on novices’ ability to comprehend source code. Additional analyses cover the influence of task type and programming experience on the code comprehension ability itself and its relation to syntax highlighting. We conducted a controlled experiment with 390 undergraduate students in an introductory Java programming course. We measured the correctness with which they solved small coding tasks. Each test subject received some tasks with syntax highlighting and some without. The data provided no evidence that syntax highlighting improves novices’ ability to comprehend source code. There are very few similar experiments and it is unclear as of yet which factors impact the effectiveness of syntax highlighting. One major limitation may be the types of tasks chosen for this experiment. The results suggest that syntax highlighting squanders a feedback channel from the IDE to the programmer that can be used more effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Microsoft Visual Studio and Eclipse still can print out source code, though. Interestingly, Eclipse retains syntax highlighting in the print-out and Visual Studio removes it.

  2. The *.cxx files in lexer and lexlib contain 38,482 and 908 (LOC), respectively, whereas the whole project comprises 75,182 (LOC) in version 225.

References

  • Allen E, Cartwright R, Stoler B (2002) Drjava: a lightweight pedagogic environment for java. ACM SIGCSE Bullet. 34(1):137. https://doi.org/10.1145/563517.563395

    Article  Google Scholar 

  • Baecker RM, Marcus A (1989) Human factors and typography for more readable programs. ACM, New York

    Google Scholar 

  • Beelders TR, du Plessis JPL (2015) Syntax highlighting as an influencing factor when reading and comprehending source code. J Eye Mov Res 9(1). https://doi.org/10.16910/jemr.9.1.1

  • Bergin S, Reilly R (2005) Programming: Factors that influence success. SIGCSE Bull 37(1):411–415. https://doi.org/10.1145/1047124.1047480

    Article  Google Scholar 

  • Bird C, Nagappan N, Murphy B, Gall H, Devanbu P (2011) Don’t touch my code!: Examining the effects of ownership on software quality. In: Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ACM, New York, ESEC/FSE ’11. https://doi.org/10.1145/2025113.2025119, pp 4–14

  • Chaffin WW, Rhiel SG (1993) The effect of skewness and kurtosis on the one-sample t test and the impact of knowledge of the population standard deviation. J Stat Comput Simul 46(1-2):79–90. https://doi.org/10.1080/00949659308811494

    Article  Google Scholar 

  • Crosby M, Stelovsky J (1990) How do we read algorithms? a case study. Computer 23(1):25–35. https://doi.org/10.1109/2.48797

    Article  Google Scholar 

  • Dieste O, Aranda AM, Uyaguari F, Turhan B, Tosun A, Fucci D, Oivo M, Juristo N (2017) Empirical evaluation of the effects of experience on code quality and programmer productivity: an exploratory study. Empir Softw Eng 22:1–86. ISSN:1573-7616. https://doi.org/10.1007/s10664-016-9471-3

  • Dimitri GM (2015) The impact of syntax highlighting in sonic pi. In: Psychology of Programming Interest Group Annual Conference 2015. Proceedings of, pp 59–70

  • Dragicevic P (2016) Fair Statistical Communication in HCI. Springer International Publishing, Cham, pp 291–330. https://doi.org/10.1007/978-3-319-26633-6_13

    Google Scholar 

  • Erguler K (2013) Barnard: Barnard’s Unconditional Test. R package version 1.3. http://cran.r-project.org/package=Barnard, [accessed 2014-07-22]

  • Feigenspan J, Kästner C, Apel S, Liebig J, Schulze M, Dachselt R, Papendieck M, Leich T, Saake G (2013) Do background colors improve program comprehension in the #ifdef hell?. Empir Softw Eng 18(4):699–745. https://doi.org/10.1007/s10664-012-9208-x

    Article  Google Scholar 

  • Gellenbeck EM, Cook CR (1991) Does signaling help professional programmers read and understand computer programs?. In: Fourth Workshop on Empirical Studies of Programmers, Ablex Publishing Corporation, Norwood, pp 82–98

  • Goldstein EB (1995) Sensation and Perception. Wadsworth Publishing, Belmont

    Google Scholar 

  • Grant EE, Sackman H (1967) An exploratory investigation of programmer performance under on-line and off-line conditions. IEEE Trans Hum Fact Electron HFE-8(1):33–48. https://doi.org/10.1109/THFE.1967.233303

    Article  Google Scholar 

  • Gray KE, Flatt M (2003) ProfessorJ: A gradual introduction to java through language levels. In: Companion of the 18th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, ACM Press, New York, OOPSLA ’03. https://doi.org/10.1145/949344.949394, pp 170–177

  • Gruhn V, Hannebauer C (2012) Using wikis as software development environments. In: Proceedings of the the 11th SoMeT_12, IOS Press, Amsterdam, Frontiers in Artificial Intelligence and Applications. https://doi.org/10.3233/978-1-61499-125-0-3, vol 246, pp 3–20

  • Hakala T, Nykyri P, Sajaniemi J (2006) An experiment on the effects of program code highlighting on visual search for local patterns. In: 18th Workshop of the Psychology of Programming Interest Group, Proceedings of, pp 38–52

  • Hannebauer C, Hesenius M, Gruhn V (2017) Lab package for the syntax highlighting experiment. https://www.uni-due.de/~hw0433/sh/

  • Holden E, Weeden E (2003) The impact of prior experience in an information technology programming course sequence. In: Proceedings of the 4th Conference on Information Technology Curriculum, ACM, New York, CITC4 ’03. https://doi.org/10.1145/947121.947131, pp 41–46

  • Jedlitschka A, Ciolkowski M, Pfahl D (2008) Reporting experiments in software engineering. Springer, Berlin, pp 201–228

    Google Scholar 

  • Kitchenham B, Pfleeger S, Pickard L, Jones P, Hoaglin D, El Emam K, Rosenberg J (2002) Preliminary guidelines for empirical research in software engineering. IEEE Trans Softw Eng 28(8):721–734. https://doi.org/10.1109/TSE.2002.1027796

    Article  Google Scholar 

  • Kleinschmager S, Hanenberg S (2011) How to rate programming skills in programming experiments?: A preliminary, exploratory, study based on university marks, pretests, and self-estimation. In: Proceedings of the 3rd ACM SIGPLAN Workshop on Evaluation and Usability of Programming Languages and Tools, ACM, New York, PLATEAU ’11. https://doi.org/10.1145/2089155.2089161, pp 15–24

  • Knuth DE (1984) Literate programming. Comput J 27(2):97–111. https://doi.org/10.1093/comjnl/27.2.97

    Article  MATH  Google Scholar 

  • van Laar D (1989) Evaluating a colour coding programming support tool. In: Sutcliffe A, Macaulay L (eds) Proceedings of the Fifth Conference of the British Computer Society Human-Computer Interaction Specialist Group, Cambridge University Press. British Computer Society Workshop Series, Cambridge, pp 217–230

  • Lientz BP, Swanson EB, Tompkins GE (1978) Characteristics of application software maintenance. Commun ACM 21(6):466–471. https://doi.org/10.1145/359511.359522

    Article  Google Scholar 

  • LPLUS GmbH (2014) LPLUS-System on-campus. http://lplus.de/lplus.de/index_9_19_2__.html, [accessed 2014-07-16]

  • Ludbrook J (2008) Analysis of 2 x 2 tables of frequencies: matching test to experimental design. Int J Epidemiol 37(6):1430–1435. https://doi.org/10.1093/ije/dyn162

    Article  Google Scholar 

  • Mehta R, Zhu RJ (2009) Blue or red? exploring the effect of color on cognitive task performances. Science 323(5918):1226–1229. https://doi.org/10.1126/science.1169144

    Article  Google Scholar 

  • Microsoft Developer Network (2015) C# coding conventions (c# programming guide). http://msdn.microsoft.com/en-us/library/ff926074.aspx, [accessed 2017-03-08]

  • Oman PW, Cook CR (1990) Typographic style is more than cosmetic, vol 33. https://doi.org/10.1145/78607.78611

  • Oracle Technology Network (1999) Code conventions for the java programming language. http://www.oracle.com/technetwork/java/javase/documentation/codeconvtoc-136057.html, [accessed 2017-03-08]

  • Prechelt L (1999) The 28:1 Grant/Sackman legend is misleading, or: How large is interpersonal variation really? Interner Bericht, Fakultät für Informatik. Universität Karlsruhe, Karlsruhe

    Google Scholar 

  • Rambally GK (1986) The influence of color on program readability and comprehensibility. In: Proceedings of the Seventeenth SIGCSE Technical Symposium on Computer Science Education, ACM, New York, SIGCSE ’86. https://doi.org/10.1145/5600.5702, pp 173–181

  • Reijers H, Freytag T, Mendling J, Eckleder A (2011) Syntax highlighting in business process models. Decis Support Syst 51(3):339 – 349. https://doi.org/10.1016/j.dss.2010.12.013

    Article  Google Scholar 

  • Sackman H, Erikson WJ, Grant EE (1968) Exploratory experimental studies comparing online and offline programming performance. Commun ACM 11(1):3–11. https://doi.org/10.1145/362851.362858

    Article  Google Scholar 

  • Sarkar A (2015) The impact of syntax colouring on program comprehension. In: Psychology of Programming Interest Group Annual Conference 2015, Proceedings of, pp 49–58

  • Scintilla Project (2014) Scintilla – a free source code editing component for win32, gtk+, and os x. http://www.scintilla.org, [accessed 2014-07-22]

  • Seo H, Sadowski C, Elbaum S, Aftandilian E, Bowdidge R (2014) Programmers’ build errors: A case study (at google). In: Proceedings of the 36th International Conference on Software Engineering, ACM, New York, ICSE 2014. https://doi.org/10.1145/2568225.2568255, pp 724–734

  • Siegmund J, Kästner C, Liebig J, Apel S, Hanenberg S (2014) Measuring and modeling programming experience. Empir Softw Eng 19(5):1299–1334. https://doi.org/10.1007/s10664-013-9286-4

    Article  Google Scholar 

  • Tapp R, Kazman R (1994) Determining the usefulness of colour and fonts in a programming task. In: Proceedings of the Third IEEE Workshop on Program Comprehension. https://doi.org/10.1109/WPC.1994.341265. IEEE Computer Society Press, Los Alamitos, pp 154–161

  • Tiarks R (2011) What programmers really do: an observational study. Softwaretechnik-Trends 31(2):36–37

    Google Scholar 

  • Wallis WA (1942) Compounding probabilities from independent significance tests. Econometrica, J Econ Soc 10(3/4):229–248

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

We thank all students enrolled in the course Programming in Java for their participation in our experiment. We also thank Matthias Book, Tobias Brückmann, and Tobias Griebe for useful comments on earlier draughts of this paper. We further thank Florian Stefan and again Matthias Book for their help setting up and supervising the experiment. We thank Stefan Hanenberg for his feedback on the paper and for the discussions about statistics. We furthermore thank the anonymous reviewers for their valuable feedback.

Author information

Authors and Affiliations

  1. paluno – The Ruhr Institute for Software Technology, University of Duisburg-Essen, Essen, Germany

    Christoph Hannebauer, Marc Hesenius & Volker Gruhn

Authors
  1. Christoph Hannebauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marc Hesenius
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Volker Gruhn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christoph Hannebauer.

Additional information

Communicated by: Sven Apel

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hannebauer, C., Hesenius, M. & Gruhn, V. Does syntax highlighting help programming novices?. Empir Software Eng 23, 2795–2828 (2018). https://doi.org/10.1007/s10664-017-9579-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-017-9579-0

Keywords

Search

Navigation