Collaborative Benchmarking Rule-Reasoners with B-Runner
Résumé
Conducting experimental analysis on rule reasoners is a mainstream task for validating novel algorithms and systems. Nevertheless, providing robust, veriable, and reproducible experiments can still raise a sensible challenge. We propose to demonstrate B-Runner, an open library for collaborative benchmarking focusing on the deployment of articulate tests for knowledge and rule-based systems with low cost and high robustness. B-Runner reduces the benchmarking setup time while guaranteeing experiment repeatability. At the same time, it improves the scrutability of experimental protocols thereby enhancing their robustness as well as fairness of system comparisons. This demonstration proposes to show- case the use of the tool for systematically testing a number of systems as well as to introduce its architecture and its extensibility to novel tools and experimental protocols.
Domaines
Informatique| Origine | Fichiers produits par l'(les) auteur(s) |
|---|