Researchers have found that approximately 70% of information systems development projects in Japan have failed, thus increasing the demand for solutions that will raise expected project success rates. It is said that to improve success rates, support should be provided by the organization to which the projects belong. The study aims to identify projects that an organization should support preferentially by predicting a project outcome. Several researches have demonstrated that they could predict project outcomes using risk assessment results in projects with specific characteristics such as in-house embedded software development. In this study, we try to predict project outcomes using the results of risk assessment at the establishment of requirements stage for the projects of a specific IT vendor in Japan.
The project outcome prediction was performed for “Company A,” one of Japanese IT vendors. In Company A, risk management specialists of the organization support the identification of project risks at the establishment of requirements stage and then determine the degrees of their participation in the project. We try to develop a project outcome prediction system including “risk assessment sheet” and “project outcome prediction model” to predict project outcome at the stage.
The risk assessment sheet enumerates the risk assessment items of a project at the establishment of requirements stage. The sheet was created by referring to the framework of McLeod and MacDonell [1] and the knowledge of Company A. As a result, a risk assessment sheet consisting of 17 risk assessment items was created.
The outcome assessment sheet enumerates assessment items for project outcomes at the end of the project. The sheet is used for the development of the project outcome prediction model. This research characterizes project outcome as the difference between expected and actual measures of quality, cost, and scheduling.
Kenichi Takano
The project outcome prediction model was created by applying logistic regression analysis using the 88 project data. Logistic regression analysis was performed with the stepwise selection method by utilizing the classification results of success/failure as the response variable and utilizing the assessment results of 17 items collected by the risk assessment sheet as the explanatory variables. As a result, a model using five risk assessment items was obtained. Then, the degree of generalization error was checked by applying the 10-fold cross-validation method to the model. As a result, the predictive accuracy of the model is 73.9%.
The research aimed to develop a project outcome prediction system including “risk assessment sheet” and “project outcome prediction model” in order to identify projects that an IT vendor should support preferentially. As a result, 73.9% of the project outcomes were predicted correctly. Considering that the research object is an IT vendor whose projects have various characteristics and that the prediction is conducted in an early project stage, the system is considered to have achieved the research purpose. Details of the study can be found in our previous paper [2].