An Empirical Comparison of Flat and Hierarchical Performance Measures for Multi-Label Classification with Hierarchy Extraction

Brucker, Florian; Benites, Fernando; Sapozhnikova, Elena

An Empirical Comparison of Flat and Hierarchical Performance Measures for Multi-Label Classification with Hierarchy Extraction

Dateien

Zu diesem Dokument gibt es keine Dateien.

Datum

2011

Autor:innen

Brucker, Florian

Benites, Fernando

Sapozhnikova, Elena

URI (zitierfähiger Link)

http://nbn-resolving.de/urn:nbn:de:bsz:352-257571

DOI (zitierfähiger Link)

https://doi.org/10.1007/978-3-642-23851-2_59

Projekt

DAMIART

Sammlungen

Informatik und Informationswissenschaft

Publikationstyp

Beitrag zu einem Konferenzband

Publikationsstatus

Published

Erschienen in

KÖNIG, Andreas, ed., Andreas DENGEL, ed., Knut HINKELMANN, ed., Koichi KISE, ed., Robert J. HOWLETT, ed., Lakhmi C. JAIN, ed.. Knowledge-Based and Intelligent Information and Engineering Systems. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 579-589. Lecture Notes in Computer Science. 6881. ISBN 978-3-642-23850-5. Available under: doi: 10.1007/978-3-642-23851-2_59

Zusammenfassung

Multi-label Classification (MC) often deals with hierarchically organized class taxonomies. In contrast to Hierarchical Multi-label Classification (HMC), where the class hierarchy is assumed to be known a priori, we are interested in the opposite case where it is unknown and should be extracted from multi-label data automatically. In this case the predictive performance of a classifier can be assessed by well-known Performance Measures (PMs) used in flat MC such as precision and recall. The fact that these PMs treat all class labels as independent labels, in contrast to hierarchically structured taxonomies, is a problem. As an alternative, special hierarchical PMs can be used that utilize hierarchy knowledge and apply this knowledge to the extracted hierarchy. This type of hierarchical PM has only recently been mentioned in literature. The aim of this study is first to verify whether HMC measures do significantly improve quality assessment in this setting. In addition, we seek to find a proper measure that reflects the potential quality of extracted hierarchies in the best possible way. We empirically compare ten hierarchical and four traditional flat PMs in order to investigate relations between them. The performance measurements obtained for predictions of four multi-label classifiers ML-ARAM, ML-kNN, BoosTexter and SVM on four datasets from the text mining domain are analyzed by means of hierarchical clustering and by calculating pairwise statistical consistency and discriminancy.

Fachgebiet (DDC)

004 Informatik

Zitieren

ISO 690

BRUCKER, Florian, Fernando BENITES, Elena SAPOZHNIKOVA, 2011. An Empirical Comparison of Flat and Hierarchical Performance Measures for Multi-Label Classification with Hierarchy Extraction. In: KÖNIG, Andreas, ed., Andreas DENGEL, ed., Knut HINKELMANN, ed., Koichi KISE, ed., Robert J. HOWLETT, ed., Lakhmi C. JAIN, ed.. Knowledge-Based and Intelligent Information and Engineering Systems. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 579-589. Lecture Notes in Computer Science. 6881. ISBN 978-3-642-23850-5. Available under: doi: 10.1007/978-3-642-23851-2_59

BibTex

@inproceedings{Brucker2011Empir-25757,
  year={2011},
  doi={10.1007/978-3-642-23851-2_59},
  title={An Empirical Comparison of Flat and Hierarchical Performance Measures for Multi-Label Classification with Hierarchy Extraction},
  number={6881},
  isbn={978-3-642-23850-5},
  publisher={Springer Berlin Heidelberg},
  address={Berlin, Heidelberg},
  series={Lecture Notes in Computer Science},
  booktitle={Knowledge-Based and Intelligent Information and Engineering Systems},
  pages={579--589},
  editor={König, Andreas and Dengel, Andreas and Hinkelmann, Knut and Kise, Koichi and Howlett, Robert J. and Jain, Lakhmi C.},
  author={Brucker, Florian and Benites, Fernando and Sapozhnikova, Elena}
}

RDF

<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/25757">
    <dc:contributor>Benites, Fernando</dc:contributor>
    <dc:rights>terms-of-use</dc:rights>
    <dc:contributor>Brucker, Florian</dc:contributor>
    <dc:contributor>Sapozhnikova, Elena</dc:contributor>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:creator>Benites, Fernando</dc:creator>
    <dcterms:title>An Empirical Comparison of Flat and Hierarchical Performance Measures for Multi-Label Classification with Hierarchy Extraction</dcterms:title>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2014-01-16T10:56:14Z</dcterms:available>
    <dc:creator>Sapozhnikova, Elena</dc:creator>
    <dc:creator>Brucker, Florian</dc:creator>
    <dcterms:abstract xml:lang="eng">Multi-label Classification (MC) often deals with hierarchically organized class taxonomies. In contrast to Hierarchical Multi-label Classification (HMC), where the class hierarchy is assumed to be known a priori, we are interested in the opposite case where it is unknown and should be extracted from multi-label data automatically. In this case the predictive performance of a classifier can be assessed by well-known Performance Measures (PMs) used in flat MC such as precision and recall. The fact that these PMs treat all class labels as independent labels, in contrast to hierarchically structured taxonomies, is a problem. As an alternative, special hierarchical PMs can be used that utilize hierarchy knowledge and apply this knowledge to the extracted hierarchy. This type of hierarchical PM has only recently been mentioned in literature. The aim of this study is first to verify whether HMC measures do significantly improve quality assessment in this setting. In addition, we seek to find a proper measure that reflects the potential quality of extracted hierarchies in the best possible way. We empirically compare ten hierarchical and four traditional flat PMs in order to investigate relations between them. The performance measurements obtained for predictions of four multi-label classifiers ML-ARAM, ML-kNN, BoosTexter and SVM on four datasets from the text mining domain are analyzed by means of hierarchical clustering and by calculating pairwise statistical consistency and discriminancy.</dcterms:abstract>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2014-01-16T10:56:14Z</dc:date>
    <dcterms:issued>2011</dcterms:issued>
    <dcterms:bibliographicCitation>Knowledge-Based and Intelligent Information and Engineering Systems : 15th International Conference, KES 2011, Kaiserslautern, Germany, September 12-14, 2011, Proceedings, Part I / ed. by Andreas König ... - Berlin [u.a.] : Springer, 2011. - S. 579-589. - (Lecture Notes in Computer Science ; 6881). - ISBN 978-3-642-23850-5</dcterms:bibliographicCitation>
    <dc:language>eng</dc:language>
    <bibo:uri rdf:resource="http://kops.uni-konstanz.de/handle/123456789/25757"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
  </rdf:Description>
</rdf:RDF>

Universitätsbibliographie

Ja