Case Study of Project Outcome Prediction
                          for an IT Vendor

                  Tomoyuki Kawamura                                                             Kenichi Takano
   Graduate School of System Design and Management                         Graduate School of System Design and Management
                    Keio University                                                         Keio University
                   Yokohama, Japan                                                         Yokohama, Japan
                 tomoyuki@z5.keio.jp                                                   k.takano@sdm.keio.ac.jp



                      I. INTRODUCTION                               B. Project Outcome Prediction Model
    Researchers have found that approximately 70% of                    The data to create the prediction model were collected by
information systems development projects in Japan have failed,      assessing the completed projects using the risk assessment
thus increasing the demand for solutions that will raise            sheet and the outcome assessment sheet. As a result, data from
expected project success rates. It is said that to improve          88 projects were collected. Then, each project was classified as
success rates, support should be provided by the organization       a success/failure by considering the assessment results of the
to which the projects belong. The study aims to identify            outcome assessment sheet. As a result of the classification, 43
projects that an organization should support preferentially by      projects (49%) were classified as successes, and 45 projects
predicting a project outcome. Several researches have               (51%) were classified as failures.
demonstrated that they could predict project outcomes using             The project outcome prediction model was created by
risk assessment results in projects with specific characteristics   applying logistic regression analysis using the 88 project data.
such as in-house embedded software development. In this             Logistic regression analysis was performed with the stepwise
study, we try to predict project outcomes using the results of      selection method by utilizing the classification results of
risk assessment at the establishment of requirements stage for      success/failure as the response variable and utilizing the
the projects of a specific IT vendor in Japan.                      assessment results of 17 items collected by the risk assessment
                                                                    sheet as the explanatory variables. As a result, a model using
        II. PROJECT OUTCOME PREDICTION SYSTEM                       five risk assessment items was obtained. Then, the degree of
    The project outcome prediction was performed for                generalization error was checked by applying the 10-fold
“Company A,” one of Japanese IT vendors. In Company A,              cross-validation method to the model. As a result, the
risk management specialists of the organization support the         predictive accuracy of the model is 73.9%.
identification of project risks at the establishment of
requirements stage and then determine the degrees of their                                    III. CONCLUSION
participation in the project. We try to develop a project               The research aimed to develop a project outcome
outcome prediction system including “risk assessment sheet”         prediction system including “risk assessment sheet” and
and “project outcome prediction model” to predict project           “project outcome prediction model” in order to identify
outcome at the stage.                                               projects that an IT vendor should support preferentially. As a
                                                                    result, 73.9% of the project outcomes were predicted correctly.
A. Risk Assessment Sheet and Outcome Assessment Sheet               Considering that the research object is an IT vendor whose
    The risk assessment sheet enumerates the risk assessment        projects have various characteristics and that the prediction is
items of a project at the establishment of requirements stage.      conducted in an early project stage, the system is considered to
The sheet was created by referring to the framework of              have achieved the research purpose. Details of the study can be
McLeod and MacDonell [1] and the knowledge of Company A.            found in our previous paper [2].
As a result, a risk assessment sheet consisting of 17 risk
assessment items was created.                                                                    REFERENCES
    The outcome assessment sheet enumerates assessment              [1]   L. McLeod and S. G. MacDonell, “Factors that Affect Software Systems
items for project outcomes at the end of the project. The sheet           Development Project Outcomes: A Survey of Research,” ACM
                                                                          Computing Surveys, Vol.43, No.4, Article24, 2011.
is used for the development of the project outcome prediction
                                                                    [2]   T. Kawamura and K. Takano, “Development of Project Outcome
model. This research characterizes project outcome as the                 Prediction System for an IT Vendor,” Proceedings of the 22st Asia-
difference between expected and actual measures of quality,               Pacific Software Engineering Conference, in print.
cost, and scheduling.




         Workshop on Alternate Workforces for Software Engineering (WAWSE 2015)                                                 55