Evaluation  
     
  What is AquaLog

AquaLog is a portable question-answering system which takes queries expressed in natural language and an ontology as input and returns answers drawn from one or more knowledge bases. AquaLog present an elegant solution in which different strategies are combined together. It makes use of the GATE NLP platform as part of the linguistic process , string metrics algorithms , a learning mechanism as a solution to manage domain-dependent lexicon, WordNet and a novel ontology-based relation similarity service to make sense of user queries with respect to the target knowledge base.

The work on the AquaLog query answering system is based on the premise that the semantic web will benefit from the availability of natural language query interfaces, which allow users to query semantic markup viewed as a knowledge base

As a conclusion, AquaLog provides an interface through which the users, which an specific issue or question in mind, can search for the information they are interested on, in an specific domain, as for example explore the KMi resource, with the facility of formulating the query in natural language. AquaLog is interactive. The ontology helps us to reformulate and understand the query in terms of concepts, instances, values and relations between them

Domain

AquaLog is currently using the KMi ontology and is coupled with the platform WebOnto, a web server which contains the knowledge models or ontologies in an OCML format.

The domain covered by KMi ontology and knowledge base is related to the academic life in our department KMi (Knowledge Media institute), with information like personnel, projects, technologies, planet news, events, etc., which is automatically extracted from departmental databases and unstructured web pages.

As pointed out above, AquaLog is an interface to query this semantic markup directly. For instance, we can ask a query such as "which are the projects in KMi related to the semantic web area" and, thanks to an inference engine able to reason about the semantic markup and draw inferences from axioms in the ontology, we can then get the correct answer.

Evaluation

Before hand, read this documentation, in particular, have a look into the AquaLog linguistic coverage and limitations. Think ten questions you would like to ask into the AquaLog system, about the academic KMi domain. You don't need to have a look into the rest of the website or to play with the system before the evaluation. Somehow, it is even better if you have never used AquaLog before our evaluation. In our meeting, you will be asked to put your questions directly into AquaLog and play with the system. Your interaction with the system will be recorded.

AquaLog Linguistic Coverage

So far, AquaLog is able to handle different categories of queries based on pure linguistic criteria. Brief explanation/examples of the main categories are presented in this section.
Note: AquaLog is able to deal with some spelling mistakes
Note: Wh-queries are queries formed by: "who", "which", "what", "does anybody", "are there any", "how many", "Give/List/Show/Tell (me) (all)"
  • Wh-queries formed by a relation between two terms: "What are the research areas related to buddyspace?"
  • Wh-queries formed by a relation between two terms, in which one of terms is implicit in the question: "Which is the job title of Motta?"
  • Wh-queries formed by an implicit relation between two terms: "is there any project about natural language?"
  • Description: "what is an ontology", "who is Vanessa?"
  • Affirmative-Negative: Is John Domingue working with knowledge management?; Is Yuangui a Phd student?
  • Wh-queries formed by a relation and three terms: how many projects are headed by researchers in the open university?; Are there any planet news in knowledge media institute related to semantic web services?
  • Wh-queries formed by three terms and an implicit relation: is there any research area about Semantic Web in akt?
  • Wh-queries formed by a relation and three terms, in which one of the terms is implicit: "Show me the contact details from the researchers in compendium".
  • Combination of queries with "and/or": "who is interested in ontologies or in the semantic web?"; "which projects are funded by eprsc and are about semantic web?"
  • Combination of queries (two relations): "which researchers wrote publications related to semantic portals?"
  • Combination of a wh-query with a wh-clause in the middle: "what researchers, who works for akt, has publications about magpie?"
  • Combination of two wh-queries: "what are the contact details of the researchers who works in hypermedia?"; "which research areas are related to the academic staff members who are interested in ontologies?", "who works with peter who has interest in semantic web?" ; which are the planet news of Liliana which are related to semantic services?
AquaLog Limitations

  • AquaLog doesn't handle temporal reasoning. Avoid questions formed with: yesterday, last year, in 1991 ...
  • AquaLog doesn't handle yet queries which require similarity or ranking reasoning. For instance: "what are the top/most successful researchers ..?, "what are the key/main publications ..?", "who share same interest .."?
  • AquaLog doesn't understand the meaning of the special words: most, more, main, top, same, different, average
  • Type of queries non-implemented: How long, when, why, how, where, affirmative-negative with three terms, complex queries which more than two relations or many terms (five or more), or combination of terms non separated by a preposition, like "French researchers" "kmi publications"
  • Doesn't handle genitive: What's, Enrico's projects, ...
Questioner - after the evaluation session -

  • What is your level of expertise with the domain or underlying ontology [1(low) - 5(high)]? Have you interact with AquaLog before doing this evaluation?
  • What new types of questions would you have like to ask? What are you missing in AquaLog? What are the biggest limitations for you?
  • What's your opinion about the usability/performance of the system? Do you think the interface is clear? What would you like to improve about the interface? Are the AquaLog answers or explanations enough clear for you?
  • It is the natural language subset covered by AquaLog clear to you or you find you have to rephrase your questions very often for AquaLog to understand them? If not, why?


Note: After using the system - Evaluate level of expertise of the user and correctness of the natural language expressions

THANKS A LOT! :-D
 
 

 
 
Dot.Kom
AKT
KMi
GATE