Facilitating the Online Search of Experts at NASA using Expert Seeker People-Finder Irma Becerra-Fernandez, Ph.D. Assistant Professor Decision Sciences and Information Systems Florida International University, Miami, FL 33199 Tel: 305-348-3476, E-mail: becferi@fiu.edu Abstract 1.Educational KMS: To elicit and catalog tacit knowledge, and at the same time serve as an educational People-Finder Systems are knowledge repositories tool. that attempt to manage knowledge by holding 2.Problem-solving KMS: Organizations with significant pointers to experts who possess specific knowledge intellectual capital require eliciting and capturing within an organization. This paper presents insights knowledge for reuse in solving new problems as well as from the development of Expert Seeker, an recurring old problems. organizational People-Finder KMS that will be used 3.Knowledge repositories: Under the auspices of KM, to locate experts at the National Aeronautics and tools historically used for singular, unrelated purposes Space Administration (NASA). This paper are integrated to address the corporate memory problem. discusses insights and lessons learned from the One type of knowledge repository is People-Finder development of this system, and the role of Systems, also known as Knowledge Yellow Pages. technology in automating the maintenance of the People-Finder Systems are knowledge repositories that expert's profiles. Expert Seeker represents an attempt to manage knowledge by holding pointers to important first step towards achieving our objective experts who possess specific knowledge within an of automatically and intuitively discovering and organization. Several organizations in different business identifying intellectual capital within the categories have identified the need to develop systems to organization. While several systems in place today help locate intellectual capital, or People-Finder KMS. rely on self-assessment, we look at the potential of The intent in developing these systems is to catalog artificial intelligence (AI) technologies, in particular, knowledge competencies, including information not data mining and clustering techniques, to uncover typically captured by Human Resources systems, in a and map organizational expertise. way that could later be queried across the organization. A literary review and a table comparing the characteristics of hallmark People-Finder KMS in use in 1 Introduction to Knowledge Management organizations today appears in [Bec00]. Systems The paper presents insights from the development of Expert Seeker, an organizational People-Finder KMS Knowledge Management Systems (KMS) have been that will be used to locate experts at the National defined as “an emerging line of systems [which] target Aeronautics and Space Administration (NASA). This professional and managerial activities by focusing on paper discusses insights and lessons learned from the creating, gathering, organizing, and disseminating an development of this system, and the role of technology in organization’s ‘knowledge’ as opposed to ‘information’ automating the maintenance of the expert's profiles. or ‘data’” [Ala99]. KMS currently in use at most organizations, fall into three categories [Bec99A]: 2 Motivation: Developing an Organizational Knowledge Management The copyright of this paper belongs to Irma Becerra-Fernandez PhD. Permission to copy without fee all or part of this material is granted provided that the copies Strategy are not made or distributed for direct commercial advantage. Proc. of the Third Int. Conf. on Practical Aspects of In order to assess the areas of Intellectual Capital for Knowledge Management (PAKM2000) Kennedy Space Center, a Knowledge Management Basel, Switzerland, 30-31 Oct. 2000, (U. Reimer, ed.) Assessment (KMA) was designed and implemented http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-34/ between the months of February and April 1998. The goals of this effort were: Irma Becerra Fernandez Ph.D. 3-1 1. To analyze the current types, sources and uses of Following from the recommendations presented at the knowledge in the organization; conclusion of this study, the KSC Executive team 2. To develop a detailed set of system specifications decided to fund the implementation of the KSC Expert and implementation plan for future related activities; Seeker People-Finder. 3. To create a detailed plan for dealing with future needs; and 3 Summary of previous research in 4. To gather data for the implementation of a prototype that will address some of KSC's Knowledge People-Finder KMS: The Searchable Management needs. Answer Generating Environment For this purpose, a series of assessment interviews were (SAGE) designed and implemented, with the cooperation of representatives of the majority of the functional groups at The NASA/Florida Minority Institution Entrepreneurial KSC. The goals of the interviews were to assist KSC in Partnership (FMIEP) Grant is funding the development identifying key competencies and analyze the current of the Searchable Answer Generated Environment knowledge architecture for the center. This step would (SAGE) which is in the category of People-Finder KMS ensure that the appropriate methodology is recommended [Bec99B]. The purpose of this KM System is to create a at the end of this phase. repository of experts in the State of Florida (FL) State Following is a summary of the findings from the University System (SUS). Currently, each State Knowledge Management Assessment of KSC’s University in Florida keeps a database of funded Knowledge Management (KM) needs and possible research, but these databases are disparate and dissimilar. enhancements to the current KM Environment: The SAGE KM System creates a single repository by 1. Expert Knowledge Elicitation and Virtual Mentoring incorporating a distributed database scheme, which can Tools: Six of the eight interviewed groups identified the be searched by a variety of fields, including research need for the implementation of tools to elicit, capture, topic, investigator name, funding agency or university. and transfer the knowledge of experts acquired through As NASA-KSC looks to develop new technologies years of experience at NASA. necessary for the continuation of their space exploration 2. An “Expert Seeker” Knowledge Management missions, their need to partner with Florida SUS experts System: Six of the eight interviewed groups expressed becomes evident. the need for an application that holds pointers to experts The SAGE system combines the unified database by with a particular background. This application would masking multiple databases as if they were one. One help locate Intellectual Capital within the center at all advantage of this method is that there is no need to levels, from technicians to Ph.D.’s. The Expert Seeker reconfigure the data to fit it into one template. This application would store competencies available within methodology provides flexibility to the users and the the organization, including for all KSC employees database administrator, regardless of the type of program completed past projects, patents, and their relevant used to collect the information at the source. Although expertise. An added benefit would be to include the project SAGE is specific in nature, what was desired competencies outside NASA-KSC, for example those of was to develop tools and techniques that would make subcontractors. managing these independent databases as seamless as 3. Collaborative Tools: Of the 8 technical groups possible. One of SAGE's advantages is that there is only interviewed, 6 expressed a need for Internet/Intranet one user point of entry at the web-enabled interface, based collaborative tools that capture knowledge as allowing multiple occurrences of the interface and giving teams create it, integrated with an electronic document the end user deployment flexibility. The main interfaces storage. developed on the query engine use text fields to search 4. Decision Support and Expert Systems: The need for the processed data for key words, fields of expertise, the implementation of KMS that would enhance decision names, or other applicable search fields. The application making and would facilitate the decision process by processes the end user's query and returns the pertinent incorporating knowledge factors from past projects that information. might prove useful and help make better, more educated SAGE has been online since August 16, 1999 at decisions in the future. http://sage.fiu.edu. Future developments for SAGE 5. Center-wide Lessons Learned Repository: Five of include such projects as the development of algorithms the eight interviewed groups expressed the need for a that will facilitate the maintenance of SAGE in a more center-wide Lessons Learned Repository. automatic fashion. This inter-organizational system will require coding developments at both the SAGE server Irma Becerra Fernandez Ph.D. 3-2 and at each of the university's servers. A complete People-Finder KMS will help to identify a researcher's description of SAGE, including implementation details expertise, within a discipline, and to facilitate and results, appears in [Bec00]. communication with a point of contact. 4 The Expert Seeker People-Finder 5 Expert Seeker at Goddard Space Flight System at Kennedy Space Center Center The NASA Faculty Awards for Research (FAR) is To further create synergies between the efforts to funding the development of Expert Seeker, which is in develop Expert Seeker at Kennedy Space Flight Center the category of People-Finder KMS. Previous (KSC), a similar effort to prototype Expert Seeker at Knowledge Management studies at KSC affirm the need Goddard Space Flight Center (GSFC) was funded by the for a center wide repository, which will provide KSC Center of Excellence in Space Data and Information with Intranet-based access to experts with specific Sciences. Efforts related to this proposal will attempt to backgrounds. Currently KSC is reorganizing from an mirror, as funds allow, some of the efforts currently operations center into a research and development center. underway at KSC, including: Expert Seeker aims to help locate intellectual capital 1. System specification and selection of the within NASA-KSC, and is this particular characteristic organizational groups to prototype the GSFC Expert that differentiates Expert Seeker from SAGE (the latter a Seeker People-Finder. KMS to find experts within the Florida universities). 2. Development of the GSFC knowledge taxonomy. Expert Seeker will be used to search for experts located 3. Design and development of the GSFC-Expert at KSC, although its use is expected to expand to other Seeker. NASA Centers. The Expert Seeker KMS will be 4. Implementation of the system prototype. accessed via KSC's Intranet. In contrast, the SAGE 5. Testing of the system prototype. KMS, which is on the world-wide-web, is accessible 6. Rollout. through the Internet. Another important difference It is expected that implementing the GSFC version of between SAGE and Expert Seeker is that the latter will Expert Seeker will be to a large extent a replication of the enable the user to search for much more detailed ongoing efforts at KSC, in order to minimize duplication information regarding the experts' achievements, of efforts and maximize the return-on-investment for including information such as intellectual property, skills NASA. The resources that will be provided by this grant and competencies, as well as the proeficiency level for will serve to ensure generic features for this innovative each of the skills and competencies. The Expert Seeker system. Furthermore, implementation of Expert Seeker KMS will provide access to competencies available at GSFC will further validate the effectiveness of this within the organization, including items that are not KMS and ensure the development of a system that could typically captured by the typical Human Resource potentially be of value to all of NASA. On the other applications, such as completed past projects, patents, hand, it is expected that the Knowledge Taxonomy for hobbies, and other relevant knowledge. This People- GSFC will differ from the one for KSC. But this Finder KMS will be especially useful when organizing requirement does not pose a concern, as Expert Seeker cross-functional teams. could be developed so the software could be The main interfaces on the query engine in Expert Seeker "configured" with customizable knowledge taxonomy. will use text fields to search the proposed data for keywords, fields of expertise, names or other applicable search fields. The application will process the end user's 6 The Technologies to Implement Expert query and returns the pertinent information. The Seeker information will be collected from a conglomeration of multimedia databases, and the presented as queried. The The development of Expert Seeker is being accomplish purpose of the Expert Seeker KMS is to unify myriad with the use of the following technologies: data collections into web-enabled repository that could 1. Cold Fusion 4.0, Java Script, Active Server Pages easily searched for relevant data. Prior to this project, ASP (Coding and Programming) there was no single point of entry into a unified 2. SQL Server 7.0 (Databases) repository that allowed identification of employees based 3. Verity (Search capabilities) on specific skills. Expert Seeker will allow KSC experts 4. Adobe Phototshop 5.0 (GUI) more visibility, and at the same time allow interested 5. HTML and other web development tools parties to identify available expertise within KSC. This Irma Becerra Fernandez Ph.D. 3-3 The development of Expert Seeker requires the appears frequently in fewer documents will generate a utilization of existing data as much as possible. Expert higher TFIDF score for wij than a term that appears with Seeker will use the data in existing Human Resources comparatively high frequency but appears in many databases for information such as employee's formal documents. Thus the TFIDF score is a measure of how educational background, the X.500 Directory for the relevant or unique a document is for a keyword in employee point-of-contact information, a Skills Database relation to a collection of documents. The resulting which profiles each employee’s competency areas, internal representation vector of the documents can then GPES, an employee performance evaluation system, and be searched by keyword. The TFIDF algorithm will be PRMS a project resource management system. Figure 1 used within the Expert Seeker system to locate experts depicts the architecture of Expert Seeker. Furthermore, within the NASA Goddard Space Flight Center and other related information deemed important in the NASA Kennedy Space Center by mining published generation of an expert profile which is not currently documents within the Intranet of these organizations. stored in an in-house database system can be user- This can be done periodically to keep the internal supplied, such as employee’s picture, project document representations up to date and to index new participation data, hobbies, and volunteer or civic documents. The resulting TFIDF vector will be used for activities Information regarding skills and competencies, search queries. Documents that are returned as a query as well as proficiency levels for the skills and result will then be indexed by author name. The final competencies needs to be collected, to a large extent, result will rank authors according to those with the through self-assessment. Recognizing that there are highest-ranking documents for that keyword and display significant shortcomings of self-assessment, we propose these to the user as a subject-matter expert. to use an increased reliance in technology to update employees' profiles, and thus place less reliance on self- 7 Challenges in the Implementation of assessed data. For example, we are proposing the use of People-Finder KMS Global Performance Evaluation System (GPES), an in- house performance evaluation tool, to mine employees' Previous research [Bec00] conducted to establish the accomplishments and automatically update their profiles. parameters to design Expert Seeker application has Typically, employees find it difficult to make time to demonstrated that one of the challenges in developing keep their resumes updated. Performance evaluations, People-Finder KMS is related to the inherent on the other hand, are without a doubt, part of shortcoming of self-assessment. Most of the People- everybody's job. We therefore seek to use this tool, Finder KMS in place today, except for example SAGE augmented with appropriate queries, to inconspicuously People-Finder or Mitre's Expert Finder [Kot98], rely on keep the employees profiles up-to-date. each employee to complete a self-assessment of Finally, a data mining effort of the document repository competency, which is later used when searching for will also contribute to update employees' profiles. Based specific knowledge areas. The disadvantage of self- on the assumption that authors of documents in the assessment is that the results of self-assessment are repository are subject matter experts, therefore, mining subjective, based on each person's self-perception, the the electronic document repository will contribute to results could be hard to normalize, and employees’ keeping employees' profiles up-to-date in an unobtrusive speculation about its possible use could 'skew' the results. way. For this purpose, we are currently experimenting For example, one particular organization conducted a with the use of the Term Frequency Inverse Document skills self-assessment study during a period of Frequency (TFIDF) algorithm. The TFIDF algorithm is downsizing. This resulted in employees' exaggeration of used as a measure of the uniqueness or relevance of a their competencies, for fear they might have been laid- document within a collection of documents with respect off. On the other hand, another organization made it to a specific keyword. TFIDF is calculated by the clear the self-assessment would be used to contact people following formula: with specific competencies to answer related questions. This self-assessment caused employees to be overly wij = tfij * log(N/n) modest about their skill profiles, for they would be required to put to test their specific knowledge. where w is the TFIDF score for term i in document j, tf is Furthermore, one People-Finder in place at Microsoft the frequency of term i in document j, and the inverse [Bec00] required supervisors to ratify their subordinates' document frequency or idf is calculated by the logarithm self-perceptions, and assign a quantifiable value to it, a of the total number of documents divided by the number requirement that many organizations would find this of documents term i appears at least once. A term that requirement too taxing on their supervisors. Irma Becerra Fernandez Ph.D. 3-4 Expert Seeker Graphical User Interface Web Based (.HTML) Generate Update Employee Employee Profiles Profiles Data Mining User Specified Data *.gif, *.bmp, *.jpeg,*.doc Skills PRMS Database Electronic Document Management X 500 Directory HR Database GPES Figure 1:Expert Seeker Architecture Another challenge in developing People-Finder KMS 1. Taxonomies should easily describe a knowledge deals with the development of knowledge taxonomies. area. Taxonomy is the study of the general principles of 2. Taxonomies should provide minimal descriptive scientific classification. Knowledge taxonomies allow text. organizing knowledge or competency areas in the 3. Taxonomies should facilitate browsing, not organization. In the case of People-Finder systems, the complicate them. taxonomy is used to describe, and catalog people's 4. Taxonomies should have the appropriate level of knowledge, an important design consideration. granularity and abstraction. If the level is to high then it Furthermore, knowledge taxonomies could be critical in will be too complicated for the user, but if the level is too the People-Finder system's success [Bec00]. People- low it will not properly describe the knowledge areas. finder KMS in place have addressed this consideration HP's CONNEX [Bec00] has been one of the successful keeping in mind that: systems in place that has developed a fairly functional taxonomy to describe people's knowledge. According to Irma Becerra Fernandez Ph.D. 3-5 Carrozza [e-mail 1999] HP’s knowledge taxonomy is follow-up email, 1999], as he describes a possible based on standards such as the U.S. Library of Congress scenario: Classifications (available online at http://lcweb.loc.gov) and the INSPEC Index (available online at You are working in a project to build a new http://www.iee.org.uk/publish/inspec), but is customized cryogenic handling storage facility. You encounter a to their business area. Other firms have followed this problem, where upon testing, a valve fails. There is model and have created their own taxonomies, as in the a design problem. You have two choices: case for Microsoft's SpuD and for BA&H's Knowledge ◆ The first choice is to go back through the same On-Line. According to Remeikis [phone interview, Sept. process with the same company and NASA 3, 1999], this effort was successful. In contrast, the engineers working the problem National Security Agency used a taxonomy based on a or standard from the Department of Labor (O*Net – ◆ The second choice is to use Expert Seeker to available online at organize the Rapid Answer Collaborative http://www.doleta.gov/programs/onet). Finally, Knowledge Expert Team (RACKET). Using the according to Timothy Horst, Vice-President and expertise keyword ‘cryogenics’ Expert Seeker Manager of Construction Resources and Technologies, finds the following experts: Bechtel Construction Operations Incorporated also 1. A collection of scientists from the developed a taxonomy for their Knowledge Bank. It is University of Arizona for cryogenics based on standards developed by the National Center for studies; Construction Education Research (NCCER – available 2. A valve manufacturing expert from a online at http://www.nccer.org ), but is only being used plant in Detroit; to catalog skills of manual workers [Phone interview, 3. A cryogenic expert that worked on August 17 and 26, 1999]. While a number of work problems during shuttle that classification standards have been developed, that could transferred to Marshall Space Flight be used to organize knowledge areas, we have not been Center. able to apply any of these standards directly, without 4. In addition, the Expert Seeker uncovers some thought and further development of the taxonomy. a collection of technical white papers A deep analysis of the People-Finder KMS in place and lessons learned that NASA has reveals that many attempts to create knowledge published from similar projects. taxonomies are unsuccessful [Remeikis phone interview The RACKET collaborates by video 1999] or sub-optimal [Carrozza phone interview and teleconference and the Internet to pinpoint the follow-up e-mail, 1999]. design problem, identify a feasible solution, and fixes the design problem in two days. 8 Practical Applicability of Expert Seeker 9 Conclusions and Future Work The Expert Seeker application has completed it’s first year development. The need for such a system to locate Results from the KSC Knowledge Management experts in an organization of more than 10,000 Assessment revealed the importance of a system to individuals, all with exellent qualifications, in order to identify experts within the organization. Expert Seeker reduce the time and effort spent in resolving issues represents an important first step towards achieving our pertaining to the R&D conducted at the different NASA objective of automatically and intuitively discovering centers. A preliminary usability study by NASA officials and identifying intellectual capital within the [Naus phone interview and follow-up e-mail, 2000] organization. While several systems in place today rely revealed minor weaknesses in the graphical-user on self-assessment, we look at the potential of artificial interface that since have been corrected. Suggestions also intelligence (AI) technologies, in particular data mining focused on methods to access information, such as the and clustering techniques, to uncover and map capability of the system to allow combined searches. organizational expertise. Expert Seeker, when completely implemented, is Data mining technologies could contribute to updating expected to become an effective tool in the management employees' profiles. Based on the assumption that of knowledge required for new product and process authors of documents in the repository are subject matter development at NASA. Chris Carlson of NASA- experts, mining the electronic document repository could Kennedy Space Center, envisions how Expert Seeker contribute to keeping employees' profiles up-to-date in will be used in the future [Carlson phone interview and an unobtrusive way. Furthermore, clustering techniques Irma Becerra Fernandez Ph.D. 3-6 could be instrumental in clustering similar data objects [Bec99B] Becerra-Fernandez, I. Searchable Answer together [Meh99]. In this case clusters of expertise, Generating Environment (SAGE): A Knowledge could reveal expertise areas that may not be currently Management System for Searching for Experts in defined. The use of clustering techniques provides the Florida. Proceedings of the 12th Annual International potential of creating a domain dictionary of “pseudo- Florida Artificial Intelligence Research Symposium keywords” that could serve to increase the semantic (FLAIRS) - Knowledge Management Track, Orlando, domain of the keywords, and which could be used to Florida (May, 1999). identify relationships that may not be necessarily [Bec00] Becerra-Fernandez, I., The Role of Artificial obvious. Another application of this clustering notion is Intelligence Technologies in the Implementation of the development of a “super” concept, which would People-Finder Knowledge Management Systems, allow to group experts together, developing a group-level Knowledge Based Systems special issue on Artificial of expertise. Given the individual areas of expertise, Intelligence in Knowledge Management, forthcoming in these could be clustered together into groups of expertise 2000. or virtual “centers of excellence”. In the case of Expert [Kot98] Kotnour, T. Presentation at the Partners in Seeker, grouping of experts within KSC or GSFC with Education Conference, Cocoa Beach, Florida (October, complementing expertise areas could result in virtual 1998). “centers of excellence”. This effort could reveal areas of [Meh99] Mehrotra, M., Alvarado, S., and Wainwright, strength that could otherwise go unnoticed in the W. Laying a Foundation for Software Engineering of organization. Knowledge Bases in Spacecraft Ground Systems. Proceedings of the 12th Annual International Florida Acknowledgements Artificial Intelligence Research Symposium (FLAIRS), Orlando, Florida (May, 1999). The author wishes to acknowledge NASA-Kennedy Space Center and NASA-Headquarters under the Faculty Awards for Research (FAR-99), grant number NAG10- 0259, as well as NASA-Goddard Space Flight Center and the CESDIS, contract number NAS5-32337 and subcontract number 5555-97-74, for financial support for the development of Expert Seeker. The author also wishes to acknowledge James Jennings, Shannon Roberts, Gregg Buckingham, Milt Halem, Susan Hobbard, Nancy Laubethal, and Jim Green, all of NASA who championed this initiative at KSC and GSFC. Finally, the author wishes to acknowledge the contributions of the students who work in the FIU Knowledge Management Lab and who collaborated in this research, specifically Claudia Alarcon, Greg Copeland, Hector Hartmann, Nelson Ospina, Haniel Pulido, Gabriela Rizzo, Juan Rodriguez, Hernan Santiesteban, and Luis Felipe Villegas. References [Ala99] Alavi, M., Leidner, D. Knowledge Management Systems: Issues, challenges, and benefits. Communications of the Association for Information Systems, 1, article 7 (1999). [Bec99A] Becerra-Fernandez, I. This is rocket science: The Knowledge Repository of Kennedy Space Center. Presentation at the Delphi’s International Knowledge Management Summit (IKMS’99), San Diego, California (March, 1999). Irma Becerra Fernandez Ph.D. 3-7