Sensitive Questions in Surveys : A Comprehensive Meta-Analysis of Experimental Survey Studies on the Performance of the Item Count Technique

Ehler, Ingmar; Wolter, Felix; Junkermann, Justus

Sensitive Questions in Surveys : A Comprehensive Meta-Analysis of Experimental Survey Studies on the Performance of the Item Count Technique

Dateien

Ehler_2-817x4qalcptg5.pdfGröße: 303.07 KBDownloads: 444

Datum

2021

Autor:innen

Ehler, Ingmar

Wolter, Felix

Junkermann, Justus

Open Access-Veröffentlichung

Open Access Green

Sammlungen

Exzellenzcluster "Die politische Dimension von Ungleichheit"
Soziologie

Publikationstyp

Zeitschriftenartikel

Publikationsstatus

Published

Erschienen in

Public Opinion Quarterly. Oxford University Press (OUP). 2021, 85(1), pp. 6-27. ISSN 0033-362X. eISSN 1537-5331. Available under: doi: 10.1093/poq/nfab002

Zusammenfassung

In research on sensitive questions in surveys, the item count technique (ICT) has gained increased attention in recent years as a means of counteracting the problem of misreporting, that is, the under- and over-reporting of socially undesirable and socially desirable behaviors or attitudes. The performance of ICT compared with conventional direct questioning (DQ) has been investigated in numerous experimental studies, yielding mixed evidence. This calls for a systematic review.

For this purpose, the present article reports results from a comprehensive meta-analysis of experimental studies comparing ICT estimates of sensitive items to those obtained via DQ. In total, 89 research articles with 124 distinct samples and 303 effect estimates are analyzed. All studies rely on the “more (less) is better” assumption, meaning that higher (lower) estimates of negatively (positively) connoted traits or behaviors are considered more valid.

The results show (1) a significantly positive pooled effect of ICT on the validity of survey responses compared with DQ; (2) a pronounced heterogeneity in study results, indicating uncertainty that ICT would work as intended in future studies; and (3) as meta-regression models indicate, the design and characteristics of studies, items, and ICT procedures affect the success of ICT. There is no evidence for an overestimation of the effect due to publication bias.

Our conclusions are that ICT is generally a viable method for measuring sensitive topics in survey studies, but its reliability has to be improved to ensure a more stable performance.

Fachgebiet (DDC)

300 Sozialwissenschaften, Soziologie

Schlagwörter

Sensitive questions, misreporting, response validity, item count technique, meta-analysis

Zitieren

ISO 690

EHLER, Ingmar, Felix WOLTER, Justus JUNKERMANN, 2021. Sensitive Questions in Surveys : A Comprehensive Meta-Analysis of Experimental Survey Studies on the Performance of the Item Count Technique. In: Public Opinion Quarterly. Oxford University Press (OUP). 2021, 85(1), pp. 6-27. ISSN 0033-362X. eISSN 1537-5331. Available under: doi: 10.1093/poq/nfab002

BibTex

@article{Ehler2021Sensi-56033,
  year={2021},
  doi={10.1093/poq/nfab002},
  title={Sensitive Questions in Surveys : A Comprehensive Meta-Analysis of Experimental Survey Studies on the Performance of the Item Count Technique},
  number={1},
  volume={85},
  issn={0033-362X},
  journal={Public Opinion Quarterly},
  pages={6--27},
  author={Ehler, Ingmar and Wolter, Felix and Junkermann, Justus}
}

RDF

<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/56033">
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dc:contributor>Ehler, Ingmar</dc:contributor>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-12-23T10:08:36Z</dcterms:available>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/56033"/>
    <dc:rights>terms-of-use</dc:rights>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/43613"/>
    <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:contributor>Junkermann, Justus</dc:contributor>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/34"/>
    <dc:creator>Ehler, Ingmar</dc:creator>
    <dcterms:abstract xml:lang="eng">In research on sensitive questions in surveys, the item count technique (ICT) has gained increased attention in recent years as a means of counteracting the problem of misreporting, that is, the under- and over-reporting of socially undesirable and socially desirable behaviors or attitudes. The performance of ICT compared with conventional direct questioning (DQ) has been investigated in numerous experimental studies, yielding mixed evidence. This calls for a systematic review.&lt;br /&gt;&lt;br /&gt;For this purpose, the present article reports results from a comprehensive meta-analysis of experimental studies comparing ICT estimates of sensitive items to those obtained via DQ. In total, 89 research articles with 124 distinct samples and 303 effect estimates are analyzed. All studies rely on the “more (less) is better” assumption, meaning that higher (lower) estimates of negatively (positively) connoted traits or behaviors are considered more valid.&lt;br /&gt;&lt;br /&gt;The results show (1) a significantly positive pooled effect of ICT on the validity of survey responses compared with DQ; (2) a pronounced heterogeneity in study results, indicating uncertainty that ICT would work as intended in future studies; and (3) as meta-regression models indicate, the design and characteristics of studies, items, and ICT procedures affect the success of ICT. There is no evidence for an overestimation of the effect due to publication bias.&lt;br /&gt;&lt;br /&gt;Our conclusions are that ICT is generally a viable method for measuring sensitive topics in survey studies, but its reliability has to be improved to ensure a more stable performance.</dcterms:abstract>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/34"/>
    <dc:creator>Junkermann, Justus</dc:creator>
    <dcterms:issued>2021</dcterms:issued>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/56033/3/Ehler_2-817x4qalcptg5.pdf"/>
    <dc:language>eng</dc:language>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/43613"/>
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/56033/3/Ehler_2-817x4qalcptg5.pdf"/>
    <dc:creator>Wolter, Felix</dc:creator>
    <dc:contributor>Wolter, Felix</dc:contributor>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-12-23T10:08:36Z</dc:date>
    <dcterms:title>Sensitive Questions in Surveys : A Comprehensive Meta-Analysis of Experimental Survey Studies on the Performance of the Item Count Technique</dcterms:title>
  </rdf:Description>
</rdf:RDF>

Universitätsbibliographie

Ja

Begutachtet

Ja