Efficient Use of KPIs in Data Science Projects

Project controlling is essential in data science projects, as the time required and costs are often difficult to estimate. Clients want to implement their use cases efficiently and successfully with machine learning methods. You can positively influence consistent project control with meaningful key performance indicators (KPIs). According to some studies (Opens in a new window), 85% of all data science projects fail, which requires early identification of obstacles.

In classical IT projects, many KPIs exist for project controlling, but these are not sufficient for data science projects. For this reason, in this article we present basic KPIs for IT projects and analyse them with regard to data science projects. As a result, we will see that specific KPIs can make the project controlling of data science projects more transparent.

The article first deals with the basics of project controlling and presents some KPIs from the software industry. Next, we use the presented KPIs and analyse them regarding data science projects using the process model CRISP-DM (CRoss-Industry Standard Process for Data Mining). The analysis also includes personal project experiences from data science projects.

Project controlling

Project control is a central component of project management and is used to support project decisions. It has the following objectives (cf. [1], p. 334):

• Coordination of project objectives

• Supporting project management and identifying deviations from the plan and their causes

• Evaluation of risks and initiation of measures to control or reduce risks.

The project controllers use instruments to control the project, including a cockpit. IT projects are usually managed on the basis of a project order. A project description usually defines the scope of the project and the expected project goals on the part of the client. However, the project description often only contains rough descriptions, which is why refinement is necessary.

Finally, we need to derive a roadmap with milestones and resource and cost planning. The milestones are essential in project controlling because they determine the timing of reviews and thus facilitate the classification of the work status in the overall project.

The KPIs to support project controlling are the main focus of this article. Experienced project managers usually rely on clearly defined and easily interpretable KPIs. In the next section we present a concept for deriving meaningful KPIs for IT projects.