A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation

Beel, Joeran; Genzmehr, Marcel; Langer, Stefan; Nürnberger, Andreas; Gipp, Bela

A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation

Dateien

Beel_0-285617.pdfGröße: 328.95 KBDownloads: 2219

Datum

2013

Autor:innen

Beel, Joeran

Genzmehr, Marcel

Langer, Stefan

Nürnberger, Andreas

Gipp, Bela

Open Access-Veröffentlichung

Open Access Green

Sammlungen

Informatik und Informationswissenschaft

Publikationstyp

Beitrag zu einem Konferenzband

Publikationsstatus

Published

Erschienen in

ALEJANDRO BELLOGIN, , ed.. Proceedings of the International Workshop on Reproducibility and Replication in Recommender Systems Evaluation. New York: ACM, 2013, pp. 7-14. ISBN 978-1-4503-2465-6. Available under: doi: 10.1145/2532508.2532511

Zusammenfassung

Offline evaluations are the most common evaluation method for research paper recommender systems. However, no thorough discussion on the appropriateness of offline evaluations has taken place, despite some voiced criticism. We conducted a study in which we evaluated various recommendation approaches with both offline and online evaluations. We found that results of offline and online evaluations often contradict each other. We discuss this finding in detail and conclude that offline evaluations may be inappropriate for evaluating research paper recommender systems, in many settings.

Fachgebiet (DDC)

004 Informatik

Schlagwörter

Research paper recommender systems, evaluation, offline evaluation, click-through rate, online evaluation, comparative study

Konferenz

RepSys '13, 12. Okt. 2013, Hong Kong

Zitieren

ISO 690

BEEL, Joeran, Marcel GENZMEHR, Stefan LANGER, Andreas NÜRNBERGER, Bela GIPP, 2013. A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation. RepSys '13. Hong Kong, 12. Okt. 2013. In: ALEJANDRO BELLOGIN, , ed.. Proceedings of the International Workshop on Reproducibility and Replication in Recommender Systems Evaluation. New York: ACM, 2013, pp. 7-14. ISBN 978-1-4503-2465-6. Available under: doi: 10.1145/2532508.2532511

BibTex

@inproceedings{Beel2013compa-31055,
  year={2013},
  doi={10.1145/2532508.2532511},
  title={A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation},
  isbn={978-1-4503-2465-6},
  publisher={ACM},
  address={New York},
  booktitle={Proceedings of the International Workshop on Reproducibility and Replication in Recommender Systems Evaluation},
  pages={7--14},
  editor={Alejandro Bellogin},
  author={Beel, Joeran and Genzmehr, Marcel and Langer, Stefan and Nürnberger, Andreas and Gipp, Bela}
}

RDF

<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/31055">
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/31055/1/Beel_0-285617.pdf"/>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:creator>Beel, Joeran</dc:creator>
    <dc:contributor>Genzmehr, Marcel</dc:contributor>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2015-05-28T12:43:41Z</dcterms:available>
    <dcterms:issued>2013</dcterms:issued>
    <dc:language>eng</dc:language>
    <dc:contributor>Langer, Stefan</dc:contributor>
    <dc:creator>Gipp, Bela</dc:creator>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:creator>Genzmehr, Marcel</dc:creator>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/31055/1/Beel_0-285617.pdf"/>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2015-05-28T12:43:41Z</dc:date>
    <dc:rights>terms-of-use</dc:rights>
    <dc:contributor>Nürnberger, Andreas</dc:contributor>
    <dcterms:abstract xml:lang="eng">Offline evaluations are the most common evaluation method for research paper recommender systems. However, no thorough discussion on the appropriateness of offline evaluations has taken place, despite some voiced criticism. We conducted a study in which we evaluated various recommendation approaches with both offline and online evaluations. We found that results of offline and online evaluations often contradict each other. We discuss this finding in detail and conclude that offline evaluations may be inappropriate for evaluating research paper recommender systems, in many settings.</dcterms:abstract>
    <dc:creator>Langer, Stefan</dc:creator>
    <dcterms:title>A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation</dcterms:title>
    <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
    <bibo:uri rdf:resource="http://kops.uni-konstanz.de/handle/123456789/31055"/>
    <dc:contributor>Gipp, Bela</dc:contributor>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:contributor>Beel, Joeran</dc:contributor>
    <dc:creator>Nürnberger, Andreas</dc:creator>
  </rdf:Description>
</rdf:RDF>

Universitätsbibliographie

Nein