Evaluating Recommender Algorithms for Learning using CrowdSourcing
Key: Erd14-1
Author: Mojisola Erdt, Christoph Rensing
Date: July 2014
Kind: In proceedings
Publisher: CPS
Book title: Proceedings of the 14th IEEE International Conference on Advanced Learning Technologies (ICALT 2014)
Abstract: Keeping focused on a certain goal or topic when learning with resources found on the Web is a challenge. Creating a hierarchical learning goal structure with activities and subactivities can help the learner to keep on track. Moreover, providing useful recommendations to such activities can further support the learner. However, recommendations need to be relevant to the specific goal or activity the learner is currently working on, as well as being novel and diverse to the learner. Such user-centric metrics like novelty and diversity are best measured by asking the users themselves. Nonetheless, conducting user experiments are notoriously time-consuming and access to an adequate amount of users is often very limited. Crowdsourcing offers a means to evaluate TEL recommender algorithms by reaching out to sufficient participants in a shorter time-frame and with less effort. In this paper, a concept for evaluating TEL recommender algorithms using crowdsourcing is presented as well as a repeated proof-of-concept evaluation experiment of a TEL graphbased recommender algorithm AScore that exploits hierarchical activity structures. Results from both experiments support the postulated hypotheses, thereby showing that crowdsourcing can be successfully applied to evaluate TEL recommender algorithms.
View Full paper (PDF) | Download Full paper (PDF)

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, not withstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.