Software Engineering Needs Agile Experimentation: A New Practice and Supporting Tool

Abstract

This article proposes a novel software engineering practice called Agile Experimentation. It aims mostly small experiments in a business driven software engineering environment where a developer is a scarce resource and the impact of the experimentation on the return-of-investment driven software project needs to be minimal. In such environment the tools used for the sake of research need to have virtually no negative impact on the developers, but simultaneously those tools need to collect high quality data to perform sound enough quantitative analyses. In order to fulfill those requirements, and to support the Agile Experimentation practice, we co-developed a tool called NActivitySensor that gathers the data about the developers activities in a widely used Integrated Development Environment—Visual Studio. The proposed Agile Experimentation practice and the developed tool complement each other to support lightweight experimentation in real-world software development settings.

Appendix A—NActivitySensor

This appendix contains description of the details of NActivitySensor Visual Studio Add-in. NActivitySensor is available as an extension [9] at Visual Studio Gallery. Visual Studio Gallery is a tools, controls, and templates distribution platform used in Visual Studio. NActivitySensor is maintained and supported as a free extension by its creators. It can be installed from withing Visual Studio by using “Extension and Updates...” from the “Tools” menu (see Fig. 1). After successful installation NActivitySensor writes the activities log to a new output window with the same name (see Fig. 2). It is possible to configure the extension to write the activity log into a database. The database engine supported in NActivitySensor is Microsoft SQL Server. The database connection string is set in NActivitySensor.dll.config in the extension installation directory (which is $\texttt {C:Users}\backslash \texttt {Account}\_\texttt {Name}\backslash \texttt {AppData}\backslash \texttt {Local}\backslash \texttt {Microsoft}\backslash \texttt {Visual}{} \texttt {Studio} \texttt {14.0}\backslash \texttt {Extensions})$. The key for the connection string is stored in NActivitySensor.MSSql.ConnectionString. Also it is possible to configure the extension on a project level. In order to do it a configuration file in the Visual Studio solution directory is needed. It needs to be called NActivitySensor.config. Example configuration file is showed on Listing 1.

Fig. 1

NActivitySensor installation in Visual studio 2015

Fig. 2

NActivitySensor output window in Visual studio 2015 IDE

Table 1 Microsoft Visual Studio events hooked in NActivitySensor

The internal storage format for the activities is JSON. JSON stands for (JavaScript Object Notation) and is a widely used as a data exchange format. It was used because it is both: (1) easy to read and write by a human and a machine and (2) its structure does not have to be defined beforehand. We are using JSON to store various activities with varied format. For example Listing 2 shows the log for the activity ‘DocumentOpened’ (opening a document in IDE) and Listing 3 shows the ‘BuildBegin’ (starting of project building in Visual Studio).

Table 1 shows all hooked-up events recorded by the NActivitySensor.

The data is stored in the Microsoft SQL Server database. Additionally the data is echoed back into VS output windows as a fall-back for non functioning database or network (it is possible to record the data from the output window to a file).