Automating third-party library migrations

Zorchenkov Alexey Mikhailovich Leading Engineer, Huawei Tech Company LLC 141002, Russia, Moskovskaya oblast', g. Mytishchi, ul. Beloborodova, 15, kv. 81 zorchenkov@gmail.com

Accepted abbreviations?

1      Introduction

Modern software systems rely heavily on third-party libraries to save time, reduce implementation costs, and improve software quality by offering extensive, reliable, and state-of-the-art functionality. ^[1],[2],[3]. However, since software systems are developing rapidly, there is a need for appropriate tools that provide reliable and effective methods for developers to make decisions when replacing old and outdated libraries with modern ones. This process of replacing a library with another, while maintaining the same code behavior, is known as library migration ^[4],[5]. It is widely known that the migration process between libraries is a rather difficult, error-prone and time-consuming process ^{[6],[2],[3],[1]}. Developers need to study the API of the new library and related documentation to find the correct API method(s) to replace the corresponding functionality in the current implementation belonging to the API of the outdated library. Developers often need to spend considerable time to make sure that new features do not have any side effect. For example, previous work has shown that developers usually spend up to 42 days migrating between libraries ^[7].

Typically, software companies assign migration tasks to developers who have more experience in order to reduce the risks of adverse side effects in the product. For example, [Figure 1] shows that developers who have more than ten years of experience,

Figure 1

they perform migrations more often than developers with less experience. The data is based on migrations collected by Alrubaye et al. ^[6] containing information about developers who previously performed migration tasks, such as names, email addresses, years of experience and migration dates. In addition, I found that 95.3% of the 57,447 Java projects used at least one third-party library (API). On average, there are 65 API update or migration processes for each project. Previously, a number of migration approaches and methods were proposed in order to determine the replacement of an outdated API with a newer version of the same API ^{[8],[9],[10],[11]}.

Other studies recommend which library to choose when replacing: ^{[12],[13],[14],[15],[16]}. However, such approaches do not provide clear guidance to developers on how to specifically perform flexibly configurable migration at the method level. In fact, method-level recommendations have been the focus of many studies, but only to recommend the same library in different programming languages and operating systems. ^{[17],[18],[19]}.

Obviously, there is a need for a more comprehensive recommendation methodology that does not depend on either the library or the programming language, i.e. it takes two different libraries as input and shows how to replace one with another at the method level.

In this article, I present a new machine learning method - a model trained on migrations previously performed by developers and recommends migrations at the API level for similar migration contexts. The model accepts two different libraries as input and outputs potential mappings between their API methods as a result. The main idea is to reuse valuable knowledge on migrations available in previous migrations performed manually by developers in other open source projects, i.e. to learn from the "Wisdom of the Crowd". The model uses predefined features related to the similarity of method signatures and their corresponding API documentation to build the algorithm.

2                Methodology The migration rule is denoted by a pair: the source (remote) library and the target (added) library and is represented as . For example, represents a Migration Rule where the easymock library migrates to the new mockito library. For a given migration rule, we denote – a set of methods belonging to , where , and – a set of methods belonging to , where . Our goal is to find correspondences between and . In such a way that each source method is mapped to the corresponding target method, this process is called method mapping.

Figure 2: Approach to the recommendation of mapping methods

Figure 3

Methods that require a significant amount of data at the training and verification stages (logistic regression and neural networks) showed the worst accuracy. On the contrary, enhanced decision trees are known for excellent performance on relatively small datasets due to the use of a set of Decision Trees and a weighted selection mechanism.

Figure 4

2.9 Setting parameters.

Figure 5

Setting parameters significantly affects learning in a given task ^[30]. In this regard, I have adjusted the training to improve accuracy. Since training is used based on a Two-class reinforced decision tree, I started the setup with the following parameters: the maximum number of leaves is 20, the minimum number of leaf instances is 10, the learning rate is 0.2, the number of trees is 100. Further, the training parameters were consistently changed until the error stabilized at 0.5% [Figure 4] with the number of leaves 6, the minimum number of leaf instances 47, the learning rate 0.14, the number of trees 233. [Figure 5] shows a comparison of training with and without parameter settings. The training schedule with parameter settings is located further from the curve and the accuracy increases by 2.3%: from 90.7% to 93.0% 3                Known methods of recommendations for mapping methods of library API migrations 3.1 Considered approaches. Consider the implementation of the other three best approaches for comparison with MIG. ·      Learning to Rank (LTR). We use the same set of features, and the same data set, evaluate each pair of methods belonging to the source and target API. The evaluation function is a linear combination of features whose weights are trained on known comparisons of migration methods. , where is the result of training on previously marked method correspondences. For the fairness of comparing LTR with other algorithms, only the method that received the maximum score is taken into account. ·      Text mining based approach (TMAP). The TMAP ^[19] method ranks method correspondences based on five attributes: , where is obtained by applying to the class description of the source method and the target method that creates and , then the cosine similarity to and is applied . it is obtained by applying to the descriptions of the source method and the target , creating and and calculating the cosine similarity between and . · Method Signature (MS). This method calculates the similarity of method signatures for all possible combinations as follows ^[28]: , where it calculates the similarity at the token level ^[8] between two types of return values, and counts the longest common sequence between two given method names ^[29]. 3.2 Results. The calculation of the accuracy of the mappings obtained by MIG and the best previously known approaches (LTR, TMAP ^[19] and MS ^[20]) is presented in [Table 1]. MIG has the best accuracy among all approaches according to all the rules, varying from 80% to 98%, which is significantly higher than other methods by 39.52% (as the difference between the average accuracy of MIG for migrations of 8 libraries is 86.98% and the best average for other methods is 47.46%).

Table 1

To explain the difference in the accuracy of the results of the approaches under consideration, the result of migration from json to gson is presented in [Figure 6]. In [Figure 6] (A), all 4 methods were able to correctly recommend replacing the method with . MS was able to find the correct replacement because the return types of both methods are the same. In addition, the names of the methods are very similar. TMAP was able to recommend the correct target method because they have a similar description and name . LTR gave the correct result due to the similarity of the descriptions , the signature of the input parameters and the type of return value . These three features have relatively high weights in comparison with other features, which significantly increases the accuracy of the ranking algorithm. In contrast to [Figure 6] (B), only MIG was able to recommend the correct target method "void addProperty(String property, Number value)". LTR recommends "void addProperty(String property, String value)" because the input parameter "String value" has more similarity with the source method of the software and than the similarity of "Number value" with the correct target method, taking into account that the other attributes have the same values for both target methods. The error occurred due to the polymorphic nature of the method: LTR was able to recommend the correct name of the method, but not the one with the correct parameters. TMAP recommended "JsonElement parse(JsonReader json)" because they had more similarity with the source method than the correct target method. MS recommended "JsonElement get(String memberName)" because it has a higher similarity of signatures with the original "put" method than the correct target method with the original method. In both cases [Figure 6] (A) and (B), MIG recommended the correct mapping of the method because he "learned" these types of patterns through numerous decision trees.

Figure 6

The analysis of the results showed that the following contexts are quite complex for all approaches: ·        Reloading the method. Refers to two methods that have the same names, but differ in the number of parameters, the type of parameters or the sequence of parameters. ·        Polymorphic methods. They are redefined in the class hierarchy, where the method of the child class has the same name and number of parameters with the method of the base class, but with different types. ·        Generalized methods (Generics). Methods with input and output parameters that allow the method to work with objects of various types. Also very difficult are the cases when the source and target methods differ in name, type of return value and even input parameters. Finally, there are methods without proper documentation that present difficulties for TMAP, LTR and MIG. [Figure 7] shows the dependence of the accuracy of MIG recommendations on the amount of data used for training. As the amount of training data increases, the accuracy increases from 83.3% with 10% of the training data to 92% with 90% of the data used for training. This confirms the independence of the features and the sufficiency of even a small amount of data to achieve good MIG accuracy.

Figure 7

4 Conclusion This study examines the problem of recommending a comparison of methods when migrating between third-party libraries. A new approach is described that recommends the comparison of methods between two unknown libraries using features extracted from the lexical similarity between method names and textual similarity of method documentation. I evaluated the result by checking how this approach and three other most commonly used approaches ^[19],[28] and ^[33] recommend a comparison of migration methods for 8 popular libraries. I have shown that the proposed approach shows much better accuracy and performance than the other 3 methods. Qualitative and quantitative analysis of the results shows an increase in accuracy by 39.51% in comparison with other well-known approaches. As part of future research, I plan to significantly expand the number of migrations used along with a wider set of binary classifiers. I also plan to expand the number of features by also including the context of using methods in the program code.