Cultural heritage collections are essential knowledge infrastructures that reflect the different cultures and civilizations revealed in certain temporal and spatial frames. They provide a solid representation of the historical background of human communities, the development of their linguistic idioms, the blooming or withering of their artistic expression, while in cases of international aggregations, like the European Union, they constitute important instruments for the social and ethnic cohesion. There are significant previous efforts that focused on the development of cultural heritage collections, which contributed on the fertilization of the successful creation, management and coordination of cultural heritage management activities. Despite many of these efforts pursued issues of quality and its assessment, there are still plenty of open topics in regard to the evaluation of these collections. PROMISE Network of Excellence addresses many of these topics based on three pillars, namely the adoption of regular and thorough experimental evaluation activities, the collaboration and re-use over the acquired knowledge-base and the automation of the experimental evaluation process. In this speech, we introduce the Digital Library Evaluation Ontology (DiLEO) that integrates the dominant concepts of the digital library evaluation domain and their relations. DiLEO formulates a concrete framework aiming to homogenize under a common language the tower of Babel of evaluation approaches, methods, techniques and practices. The presented ontology can contribute to the scope and aims of PROMISE NoE in two large areas: (a) in the creation of knowledge bases around cultural heritage evaluation, and (b) in the planning of forthcoming evaluation activities.