Outside the Box : Abstraction-Based Monitoring of Neural Networks

Henzinger, Thomas A.; Lukina, Anna; Schilling, Christian

Outside the Box : Abstraction-Based Monitoring of Neural Networks

Dateien

Henzinger_2-1gbobv4s6kgjt1.pdfGröße: 1.56 MBDownloads: 107

Datum

2020

Autor:innen

Henzinger, Thomas A.

Lukina, Anna

Schilling, Christian

Open Access-Veröffentlichung

Open Access Bookpart

Sammlungen

Informatik und Informationswissenschaft

Publikationstyp

Beitrag zu einem Konferenzband

Publikationsstatus

Published

Erschienen in

DE GIACOMO, Giuseppe, ed., Alejandro CATALA, ed., Bistra DILKINA, ed. and others. ECAI 2020 : 24th European Conference on Artificial Intelligence. Amsterdam: IOS Press, 2020, pp. 2433-2440. Frontiers in Artificial Intelligence and Applications. 325. ISSN 0922-6389. eISSN 1879-8314. ISBN 978-1-64368-100-9. Available under: doi: 10.3233/FAIA200375

Zusammenfassung

Neural networks have demonstrated unmatched performance in a range of classification tasks. Despite numerous efforts of the research community, novelty detection remains one of the significant limitations of neural networks. The ability to identify previously unseen inputs as novel is crucial for our understanding of the decisions made by neural networks. At runtime, inputs not falling into any of the categories learned during training cannot be classified correctly by the neural network. Existing approaches treat the neural network as a black box and try to detect novel inputs based on the confidence of the output predictions. However, neural networks are not trained to reduce their confidence for novel inputs, which limits the effectiveness of these approaches. We propose a framework to monitor a neural network by observing the hidden layers. We employ a common abstraction from program analysis - boxes - to identify novel behaviors in the monitored layers, i.e., inputs that cause behaviors outside the box. For each neuron, the boxes range over the values seen in training. The framework is efficient and flexible to achieve a desired trade-off between raising false warnings and detecting novel inputs. We illustrate the performance and the robustness to variability in the unknown classes on popular image-classification benchmarks.

Fachgebiet (DDC)

004 Informatik

Konferenz

24th European Conference on Artificial Intelligence - ECAI 2020, 29. Aug. 2020 - 8. Sep. 2020, Santiago de Compostela, Spain

Zitieren

ISO 690

HENZINGER, Thomas A., Anna LUKINA, Christian SCHILLING, 2020. Outside the Box : Abstraction-Based Monitoring of Neural Networks. 24th European Conference on Artificial Intelligence - ECAI 2020. Santiago de Compostela, Spain, 29. Aug. 2020 - 8. Sep. 2020. In: DE GIACOMO, Giuseppe, ed., Alejandro CATALA, ed., Bistra DILKINA, ed. and others. ECAI 2020 : 24th European Conference on Artificial Intelligence. Amsterdam: IOS Press, 2020, pp. 2433-2440. Frontiers in Artificial Intelligence and Applications. 325. ISSN 0922-6389. eISSN 1879-8314. ISBN 978-1-64368-100-9. Available under: doi: 10.3233/FAIA200375

BibTex

@inproceedings{Henzinger2020Outsi-53573,
  year={2020},
  doi={10.3233/FAIA200375},
  title={Outside the Box : Abstraction-Based Monitoring of Neural Networks},
  number={325},
  isbn={978-1-64368-100-9},
  issn={0922-6389},
  publisher={IOS Press},
  address={Amsterdam},
  series={Frontiers in Artificial Intelligence and Applications},
  booktitle={ECAI 2020 : 24th European Conference on Artificial Intelligence},
  pages={2433--2440},
  editor={De Giacomo, Giuseppe and Catala, Alejandro and Dilkina, Bistra},
  author={Henzinger, Thomas A. and Lukina, Anna and Schilling, Christian}
}

RDF

<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/53573">
    <dc:creator>Henzinger, Thomas A.</dc:creator>
    <dcterms:title>Outside the Box : Abstraction-Based Monitoring of Neural Networks</dcterms:title>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-04-30T13:35:07Z</dc:date>
    <dc:contributor>Henzinger, Thomas A.</dc:contributor>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/53573/1/Henzinger_2-1gbobv4s6kgjt1.pdf"/>
    <dc:creator>Schilling, Christian</dc:creator>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dcterms:issued>2020</dcterms:issued>
    <dc:creator>Lukina, Anna</dc:creator>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dcterms:abstract xml:lang="eng">Neural networks have demonstrated unmatched performance in a range of classification tasks. Despite numerous efforts of the research community, novelty detection remains one of the significant limitations of neural networks. The ability to identify previously unseen inputs as novel is crucial for our understanding of the decisions made by neural networks. At runtime, inputs not falling into any of the categories learned during training cannot be classified correctly by the neural network. Existing approaches treat the neural network as a black box and try to detect novel inputs based on the confidence of the output predictions. However, neural networks are not trained to reduce their confidence for novel inputs, which limits the effectiveness of these approaches. We propose a framework to monitor a neural network by observing the hidden layers. We employ a common abstraction from program analysis - boxes - to identify novel behaviors in the monitored layers, i.e., inputs that cause behaviors outside the box. For each neuron, the boxes range over the values seen in training. The framework is efficient and flexible to achieve a desired trade-off between raising false warnings and detecting novel inputs. We illustrate the performance and the robustness to variability in the unknown classes on popular image-classification benchmarks.</dcterms:abstract>
    <dc:language>eng</dc:language>
    <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:contributor>Schilling, Christian</dc:contributor>
    <dc:rights>terms-of-use</dc:rights>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/53573/1/Henzinger_2-1gbobv4s6kgjt1.pdf"/>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/53573"/>
    <dc:contributor>Lukina, Anna</dc:contributor>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-04-30T13:35:07Z</dcterms:available>
  </rdf:Description>
</rdf:RDF>

Universitätsbibliographie

Ja