{"id":118508,"date":"2017-12-28T05:26:25","date_gmt":"2017-12-28T05:26:25","guid":{"rendered":"https:\/\/livablesoftware.com\/?p=118508"},"modified":"2017-12-28T05:35:14","modified_gmt":"2017-12-28T05:35:14","slug":"gitana-git-software-project-inspector-database","status":"publish","type":"post","link":"https:\/\/livablesoftware.com\/gitana-git-software-project-inspector-database\/","title":{"rendered":"Gitana: A Software Project Inspector on top of a relational database"},"content":{"rendered":"

The development of software projects entails significant implementation and collaboration activities, typically supported by tools such as issue trackers, code review tools, and Version Control Systems. However, these tools only provide a partial view of the project and often lack of advanced querying mechanisms, thus hampering the analysis of the status of the project and endangering the decision making process on the best way to drive the development process. We present Gitana, a software project inspector able to import the activity of the different support tools into a single relational database, thus providing a central point to perform all kinds of cross-cutting analysis on the software project data.<\/p>\n

This work has been accepted as al Original Software Publication (OSP) in the Science of Computer Programming Journal, you can find the official paper here<\/a> or just continue reading.<\/p>\n

Introduction<\/h2>\n

Software development processes are supported by a plethora of tools. Each tool helping to manage the complexity of a specific aspect of the project:\u00a0source code is tracked by using Version Control Systems (VCSs) such as Git, development tasks are reported in issue trackers, and collaboration and coordination activities are normally developed in forums, mailing lists or chats. While all these tools store relevant information, they are actually silos that provide partial views of the project and lack of advanced query and data integration mechanisms. Thus, project managers often struggle to understand how projects perform, a must to make knowledgeable decisions about the development process<\/p>\n

In this paper we present Gitana, a project inspector that analyzes the support tools used in software projects and imports the information in a relational database, thus providing a central point to perform all kinds of cross-cutting analysis on project data. The current version of the tool provides support to inspect Git repositories, Bugzilla\/GitHub issue trackers, Eclipse forums and Slack instant messages. To ensure efficiency, Gitana comes with an incremental propagation mechanism that refreshes the database content with the latest modifications available on the data sources. The approach also incorporates exporters to enable further data analysis with third-party tools.<\/p>\n

Problems and Background<\/h2>\n

Nowadays, software development activities are independently tracked and recorded in different tools. These tools have limited support for querying their internal data and typically rely on scripts (e.g., git commands) or APIs (e.g., the Bugzilla API) to do so. As a result, any advanced analysis of the data quickly becomes a hard and complex task. Especially to perform any data integration or combination operation involving more than one tool, as it is typically the case. As consequence, comparison of data from different projects is also challenging.<\/p>\n

This situation limits the scope of existing research studies and tools devoted to understanding (and improving) software development (see our summary of all we have learned by mining GitHub<\/a> plus some concerns [1]). For instance, GHTorrent<\/a> [2] is a dataset only devoted to analyzing GitHub repositories, the work presented by Kahani et al. [3] target the analysis of Eclipse forums and Wang et al. [4] study the context of StackOverflow. Examples of more general tools are GrimoireLab<\/a> and Kibble<\/a>. They are similar to Gitana in their goals though they target a different user profile. In short, Gitana is the only one providing SQL-based access to the data which is a plus for users not familiar with Python libraries for data manipulation (e.g., Panda) and NoSQL storage engines (e.g., ElasticSearch) as required in those other tools. Therefore, we believe Gitana is especially useful for users (like, for instance, project managers) that want an easy way to explore the project data (and compare data across projects) by easily writing and running adhoc queries.<\/p>\n

Software Framework and Architecture<\/h2>\n

We propose a unified access to the information made available by all support tools. Figure 1 depicts our approach. From a global conceptual schema for project-related concepts, we derived a relational database, which is then populated from a variety of partial sources through an incremental update mechanism. The database can be manually explored using standard SQL and\/or used as the input for different kinds of deeper analysis processes such as complex network analysis and OLAP multidimensional analysis. Gitana architecture is divided into four main components (black-filled elements in Figure 1).<\/p>\n

\"\"<\/p>\n

Figure 1: Our approach.<\/strong><\/p>\n