DIY (Do-It-Yourself) requires extensive knowledge such as the usage of tools, properties of materials, and the procedure of activities. Most DIYers use online search to find information but it is usually time-consuming and challenging for novice DIYers to understand the retrieved results and later apply them to their individual DIY tasks. In the work, we present a Question Answering (QA) system which can address the DIYers' specific needs. The core component is a knowledge base (KB) which contains a vast amount of domain knowledge encoded in a knowledge graph. The system is able to explain how the answers are derived with reasoning process. Our user study shows that the QA system addresses DIYers' needs more effectively than the web search.
Question Type Sub-question type Example required entities / properties WWhhaatt ipsotwheerletonogltshdoof tIhneederdililnbtihtenpeerodjeedcti?n the project? alternative entities Can I use a circular saw instead of a jigsaw in step 2? Project Question teixmpela/ndaitfifioncuolftyac/tcioosnts CHaonwylooungexdpoleasinthteheprsoajwecintgtasktee?p in more detail? specific location of actions Where should it be screwed? alternative actions Are there other options instead of pre-drilling? definition of tool / accessory / material What is jigsaw? related action What can I do with a jigsaw? Domain Question tsitprusctural info IWs hthaetrdeoaensyasjaigfestaywtilpofookrluiksein?g a jigsaw? comparison How does a jigsaw differ from a circular saw?
KR Taxonomy of Concept
The KB is then used by the reasoner to derive an answer. For most question types, our system converts the question into a SPARQL query using in-house NLP solutions and execute it against the knowledge graph. For some complex question types (e.g., ’alternative entities’ in Table. 1), we use advanced AI reasoning techniques 3. 3
In the pilot study, we compared our system against online search, a common method for information seeking. Specifically, we conducted a user study where 20 users were given a sample DIY project along with 5 questions and were asked to find an answer using online search and our QA system in separate sessions. With online search, the average time of finding an answer was 3.8 min. while our QA system can instantly provide an answer. The users’ satisfaction rate with our system was also found to be significantly better than that with online search. In the talk, we also want to share the lessons we learned from this project: pipelined architecture vs. end-to-end architecture, importance of explainability, and the knowledge acquisition bottleneck.