Software Engineering bots are applications able to react to external stimuli such as events triggered by tools or messages posted by users and run automated tasks in response, working as an interface between users and services.
Bots often include conversational capabilities to interact with end-users through textual messages (in chatbots) or speech (in voicebots) in the same communication channels as their human counterparts. Bots can support technical and social activities in software engineering, including communication and decision-making.
We are witnessing a massive adoption of bots in a variety of domains, including e-commerce, customer service, and education. Software development is no exception [1], [2]. Given the essential complexity of software projects and the large community of people around them (stakeholders, designers, developers and, let’s not forget, end-users), there are plenty of opportunities for bots to jump in and tame this complexity by (semi)automating repetitive tasks.
We often see bots working on software repositories, e.g., to manage pull requests, acting as Q&A bots, e.g., for information retrieval, or integrated in software development environments, e.g., automating bug repair [3].
Automation is even more relevant for open-source projects, which typically face sustainability issues. The adoption of bots may help free some responsibilities of the open-source maintainers and allow them to focus on the most critical tasks, benefiting the long-term health of open source. In open-source (or inner-source) projects, bots can leverage the public availability of software assets, including source code, discussions, issues and comments, to target more significant contributions.
All these topics in the intersection between bots and software engineering was the focus of a special issue at IEEE Software co-organized together with Emad Shihab, Stefan Wagner, Marco A. Gerosa and Mairieli Wessel. As part of our editorial we included a discussion on the future challenges in the development of bots for software engineering.
Future challenges for bots in software engineering
The growth in popularity and contribution of bots is undeniable. The number of libraries, platforms, and reusable bots keeps mounting up. Nevertheless, to fully unleash the potential of bots in software engineering, we would like to draw attention to several technical and socioeconomic open challenges.
Regarding technical challenges, we need better systems to facilitate the coordination and collaboration of bots in the same project, as right now, each bot behaves in an independent way, and they can have conflicting actions. This challenge requires defining bot-specific coordination and integration policies.
Quality evaluation of bots is another key area. Mainly when bots include conversational capabilities, bot testing implies redefining many of the classical testing concepts as we need to test the behavioral part of the bot, the conversational component, and the combination of the two [4], [5].
Finally, security and privacy also pose relevant challenges. As we must be able to trust the bots we add to our projects, we need techniques that assure that bots will not perform malicious activities, leak data and request the bare minimum permissions.
Beyond technical aspects, we need to better understand users’ perception of bots and how to optimize human-bot collaboration. Bots will need to get better communication and cognitive skills. For instance, when interacting with users, bots should be able to show empathy and re-act differently depending on the result of the sentiment analysis of the conversation they are having so far. Another example would be the need to learn and mimic the specific idiosyncrasy of a project (including its vocabulary and natural language use) to increase their chances of being accepted.
At the same time, bots could help in promoting social diversity in the project. As an example, they could identify and better support contributions from community minorities in the project. Finally, they should be able to explain their behavior to improve their trustworthiness.
The economic impact of bots in a project also deserves special attention. We do not have good economic models to evaluate the return-on-investment (ROI) of adopting a certain bot. If we could estimate the value of a bot for a project, it would be much easier to have rational discussions with project owners considering the cost-benefit analysis of integrating it. Even if some bots are released as open-source software, this doesn’t mean there is no cost to adopt them. For example, developers often disregard the cost of learning how to use a bot properly.
So far, we have mostly discussed the im-pact of bots on software engineering. But as bots are software components themselves, bot development could and should benefit from well-grounded software engineering practices. What the best practices are for this specific type of software component remains to be seen. For example, it is still unclear how bots, especially collaborative and cognitive bots, will be tested. Bots are becoming smarter, and we know that the creation of smart software applications poses a specific set of additional challenges [6]. We hope the community can benefit from this special issue’s papers and keep working on innovating in this increasingly important field.
References
- M. Storey and A. Zagalsky, “Disrupting developer productivity one bot at a time,” in Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016
- L. Erlenhov, F. G. de Oliveira Neto, and P. Leitner, “An empirical study of bots in software development: characteristics and challenges from a practitioner’s perspective,” in ESEC/FSE ’20: 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering.
- Santhanam, T. Hecking, A. Schreiber, and S. Wagner, “Bots in software engineering: a systematic mapping study,” PeerJ Computer Science, vol. 8, no. e866, 2022.
- Riccio, G. Jahangirova, A. Stocco, N. Humbatova, M. Weiss, and P. Tonella, “Testing machine learning based systems: a systematic mapping,” Empir. Softw. Eng., vol. 25, no. 6, pp. 5193–5254, 2020.
- J. Cabot, L. Burgueño, R. Clarisó, G. Daniel, J. Perianez-Pascual, and R. Rodríguez-Echeverría, “Testing challenges for NLP-intensive bots,” in 3rd IEEE/ACM International Workshop on Bots in Software Engineering, BotSE@ICSE 2021
- Ozkaya, “What is really different in engineering ai-enabled systems?” IEEE Softw., vol. 37, no. 4, pp. 3–6, 2020.
Featured image generated with DALL·E using the prompt a picture of a robot doing pair programming with a human developer