The requirements engineering process is a crucial stage of the software development life cycle. It involves various stakeholders from di erent professional backgrounds, particularly in the requirements elicitation phase. Each stakeholder carries distinct domain knowledge, causing them to di erently interpret certain words, leading to cross-domain ambiguity. This can result in misunderstanding amongst them and jeopardize the entire project. This paper proposes a natural language processing approach to nd potentially ambiguous words for a given set of domains. The idea is to apply linear transformations on word embedding models trained on di erent domain corpora, to bring them into a uni ed embedding space. The approach then nds words with divergent embeddings as they signify a variation in the meaning across the domains. It can help a requirements analyst in preventing misunderstandings during elicitation interviews and meetings by de ning a set of potentially ambiguous terms in advance. The paper also discusses certain problems with the existing approaches and discusses how the proposed approach resolves them.
In the context of software engineering, requirements engineering (RE) is the process of describing the intended behaviour of a software system along with the associated constraints [Pre10]. One of its phase is requirements elicitation, which has been termed as the most di cult, critical, and communication-intensive aspect of software development [AS05]. It requires interaction between di erent stakeholders through various techniques like brainstorming sessions and facilitated application speci cation technique. A stakeholder is any person with a vested interest in the project, such as potential users, developers, testers, domain experts, and regulatory agency personnel [SM12]. As these stakeholders come from di erent professional backgrounds and carry di erent domain knowledge, cross-domain ambiguity can occur amongst them. One may assign an interpretation to another's expression di erent from the intended meaning. This results in misunderstanding and distrust in requirements elicitation meetings, and costly problems in the later stages of the software life cycle [WMGWF13].
The study of variation of word meanings across domains as an NLP problem is termed as Synchronic Lexical Semantic Change (LSC) Detection [SHDTSIW19]. The rst attempt to apply it for dealing with cross-domain ambiguity in RE was by Ferrari et al. (2017) who used Wikipedia crawling and word embeddings to estimate ambiguous computer science (CS) terms vis-a-vis other application domains [FDG17]. Mishra and Sharma extended this work by focusing on various engineering subdomains [MS19]. Another approach was suggested by Ferrari et al. (2018) which also considered the ambiguity caused by non-CS domain-speci c words and addressed some of the technical limitations of the previous work [FEG18]. This approach was later extended to include quantitative evaluation of the obtained results [FE19]. An alternative approach which doesn't require domain-speci c word embeddings was suggested by Toews and Holland [TH19].
This paper proposes a natural language processing (NLP) approach based on linear transformation of word embedding spaces. Word embedding is a vector representation of a word capable of capturing its semantic and syntactic relations. A linear transformation can be used to learn a linear relationship between two word embedding spaces. The proposed approach produces a ranked list of potentially ambiguous terms for a given set of domains. It constructs a word embedding space for each domain using corpora composed of Wikipedia articles. It then applies linear transformations on these spaces in order to align them and construct a uni ed embedding space. For each word in a set of target words, an ambiguity score is assigned by applying a distance metric on its domain-speci c embeddings.
The remainder of this paper is organised as follows: Section 2 provides some background on ambiguity in RE and linear transformation of word embedding spaces. The existing approaches to cross-domain ambiguity detection are brie y explained in Section 3. The motivation behind the proposed approach is discussed in Section 4, whereas the apporach itself is outlined in Section 5. The experimental setup and results are presented and discussed in Section 6, and the conclusion and details of planned future work are provided in Section 7. 2 2.1
Ambiguity refers to the ability of a natural language (NL) expression to be interpreted in multiple manners. As requirements elicitation is a communication-intensive process, ambiguity is a major negative factor as it can lead to an unclear and incomplete requirements. Most of the existing literature on ambiguity in RE is focused on written requirement documents, and the role of ambiguity in oral NL during elicitation interviews has not been investigated thoroughly [FSG16]. Ambiguity can cause misunderstanding situations during elicitation interviews, where the requirements analyst does not understand the customer's expression or interprets it incorrectly. The latter phenomenon is known as subconscious disambiguation and is one of the major causes of requirements failure [GW89]. It is di cult to identify unless the interpretation by the analyst is not acceptable in his or her mental framework [FSG16]. The problem of cross-domain ambiguity can be seen as a special case of subconscious disambiguation which is caused due to di erent domain knowledge. 2.2
Word embedding is a collective term for language modelling techniques that map each word in the vocabulary to a dense vector representation. Contrary to one-hot representation, word embedding techniques embed each word into a low-dimensional continuous space and capture its semantic and syntactic relationships [LXT+15]. It is based on the distributional hypothesis proposed by Harris which states that words appearing in similar linguistic contexts share similar meanings [Har54].
One of the most popular word embedding techniques is skip-gram with negative sampling (SGNS) [MSC+13]. It trains a shallow two-layer neural network which, given a single input word w, predicts a set of context words c(w). The context for a word wi is the set of words surrounding it in a xed-size window, i.e. fwi L; ; wi 1; wi+1; ; wi+Lg, where L is the context-window size. Each word w is associated with vectors uw 2 RD and vw 2 RD, called the input and output vectors respectively. If T is the number of windows in the given corpus, then the objective of the skip-gram model is to maximize In the negative sampling method, p(wt+ijwi) is de ned as
T 1 X T
X p(wOjwI ) = log (uTwI vwO ) + A linear transformation can be used to learn a linear mapping from one vector space to another. Its use for combining di erent word embedding spaces was rst explored by Mikolov et al. who used it for bilingual machine n translation [MLS13]. They used a list of word pairs fxi; yigi=1, where yi is the translation of xi. Then they learned a translation matrix W by minimizing the following loss function n X jxiW i=1 yij This approach can also be used for aligning monolingual word embeddings. If one assumes that the meaning of most words remains unchanged, linear regression can be used to nd the best rotational alignment between two word embedding spaces. Failure to properly align a word can be then used to identify a change in meaning. This is the basis for the proposed approach towards identifying cross-domain ambiguous words. Similar approaches have been used to detect linguistic variation in the meaning of a word with time and to develop ensemble word embedding models [KARPS15, MSL17].
Signi cant work has been done to improve the linear transformation method. Dimension-wise mean centering has been shown to improve the performance of linear transformation methods in downstream tasks [ALA16]. Xing et al. noticed a hypothetical inconsistency in the distance metrics used in the optimization objectives in the work of Mikolov et al.: dot product for training word embeddings, Euclidean distance for learning transformation matrix, and cosine distance for similarity computations [XWLL15]. It was solved by normalizing the word embeddings and by requiring the transformation matrix to be orthogonal. The optimal orthogonal transformation matrix which maps X to Y can be found through the solution of the well-known Orthogonal Procrustes problem, which is given by
W = U V T where XT Y = U V T is the singular value decomposition (SVD) factorization of XT Y [Sch66]. 3
Synchronic LSC detection refers to the measurement of variation of word meanings across domains or speaker communities [SHDTSIW19]. The latter has been studied by making use of the large-scale data provided by communities on online platforms such as Reddit [TF17].
Research works on cross-domain ambiguity detection have been limited to its applicability in RE. The rst approach was suggested by Ferrari et al. (2017) who employed Wikipedia crawling and word embeddings to estimate the variation of typical CS words (e.g., code, database, windows) in other domains [FDG17]. They used Wikipedia articles to create two corpora: a CS one and a domain-speci c one, replaced the target words (top-k most frequent nouns in the CS corpus) in the latter by a uniquely identi able modi ed version, and trained a single language model for both corpora. Cosine similarity was then used as a metric to estimate the variation in the meaning of the target words when they are used in the speci ed domain. However, this approach su ers from the following drawbacks: the inability to identify non-CS cross-domain ambiguous words, the need to construct a language model for each combination of domains, and the need to modify the domain-speci c corpus.
Their work was extended by Mishra and Sharma who applied it on various subdomains of engineering with varying corpus size [MS19]. They identi ed the most suitable hyperparameters for training word embeddings on corpora of three di erent classes: large, medium, and small, based on the number of documents. They then used the obtained results to identify a similarity threshold for ambiguous words.
Ferrari et al. (2018) suggested an approach based on developing word embedding spaces for each domain, and then estimating the variation in the meaning of a word by comparing the lists of its most similar words in each domain [FEG18]. This approach addressed the above-mentioned drawbacks of the previous one. It was later extended by Ferrari and Esuli, with the major contribution being the introduction of a quantitative evaluation of the approach [FE19].
An alternative approach which does not require domain-speci c word embeddings was suggested by Toews and Holland [TH19]. It estimates a word's similarity across domains through context similarity. This approach does require trained word embeddings, but they are not required to be domain-speci c, which allows it to be used on small domain corpora as well. If D1 and D2 are two domain corpora, then the context similarity of a word w is de ned as simc(w) =
center(c1) center(c2) kcenter(c1)k kcenter(c2)k center(c) = 1
X IDFD(w) vw jcj w2c (5) (6) where c1
D1 and c2
4
The motivation behind the proposed linear transformation based approach is based on the following factors: The approaches suggested by Ferrari et al. (2018) and Toews and Holland judge a word's meaning from its local context rather than a global one. This leads them to wrongly assign a high ambiguity score to a word having distinct, yet similar, nearest words in di erent domains. A particular example of this problem is the high score assigned to proper names such as Michael ; although they are near to other proper nouns in all domains, but the exact lists vary widely. Such approaches also fail in the opposite scenario in which the meaning of the nearest words themselves change. This can happen in the case of ambiguous clusters. For example, a lot of topics in arti cial intelligence, such as neural networks and genetic algorithms, are inspired by biology. Due to this, certain words appear together in both these domains but carry di erent interpretations. However, the approach proposed by this paper relies on the global context rather than the local one, which resolves the issues mentioned above.
The proposed approach can work for more than two domains as opposed to the approaches suggested by Ferrari et al. (2017) and Toews and Holland [FDG17, TH19].
The approach proposed by Ferrari et al. (2018) assumes the meaning of the neighbouring words to be the same across domains, whereas a linear transformation based approach works on a much weaker assumption that the meaning of most words remains the same across domains.
Schlechtweg et al. evaluated various synchronic LSC detection models on SURel, a German dataset consisting of the meaning variations from general to domain-speci c corpus determined through manual annotation [HSSiW19, SHDTSIW19]. Their study found linear transformation to perform much better than other alignment techniques such as word injection (proposed by Ferrari et al. (2017)) and vector initialization. 5
Given a set of domains D = fD1; ; Dng, the approach requires a word embedding space Si corresponding to each domain Di. The rst step is to align the embedding spaces (subsection 5.1) and then determine the set of target words (subsection 5.2). The nal step is to assign a cross-domain ambiguity score to each target word (subsection 5.3). This step determines a transformation matrix Mi for each domain-speci c word embedding space Si which maps it to a uni ed embedding space. It uses an algorithm devised by Muromagi et al. [MSL17] which iteratively nds the transformation matrices M1; M2; ; Mn and the common target space Y . It performs the following two steps in each iteration: 1. The transformation matrices M1; M2;
; Mn are calculated using equation 4. 2. The target space is updated to be the average of all transformed spaces:
where nw is the number of domain-speci c embedding spaces with word w as part of its vocabulary.
These steps are repeatedly performed as long as the change in average normalised residual error, which is given by
i=1 is equal to or greater than a prede ned threshold . 5.2
The approach for identifying the set of target words TD has been presented in Algorithm 1.
procedure SelectWords(C; k; )
TD ; for wi 2 Vocab(C1) [ [ Vocab(Cn) do if POS(wi) 2 fN N; V B; ADJ g then counts = fFreq(C1; wi); ; Freq(Cn; wi)g c1; c2 Top2Values(counts) if c1 > k ^ c2 > c1 then
TD TD [ fwig return TD
Y (w) = 1 nw
X Si(w)Mi nw i=1 n 1 X kSiMi n pjSij d
Y k (7) (8) (9)
This step requires two numerical parameters, k and . To be considered a target word, w must satisfy three conditions:
It must be a content word, i.e. noun, verb, or adjective.
Its maximum frequency in a domain corpus, i.e. fmax = max(counti(w)), should be greater than or equal to k.
It should have a frequency of at least fmax in any other domain corpus. 5.3
This step assigns an ambiguity score to each word in TD based on their cross-domain ambiguity across the corpora C = fC1; ; Cng. The algorithm for the same is reported in Algorithm 2.
The idea is as follows. For each word w in the set of target words TD, the cosine distance for each unordered pair of its transformed embeddings is calculated, which is given by cosineDistance(vi; vj ) = 1
vi vj kvikkvj k
The average of all these cosine distances, weighted by the sum of the word frequencies in the corresponding domain corpora, is the ambiguity score assigned to the word w. All words in TD are sorted according to their score and a ranked list AD is produced.
procedure AssignAmbiguityScores(TD; M; S) Score ; for w 2 TD do
V ; for Si 2 S do if w 2 Si then
V V [ fMiSi(w)g U 0 C 0 for vi 2 V do for vj 2 V n vi do c counti(w) + countj (w) U U + c CosineDistance(vi; vj )
C C + c
Score[w] U=C AD Sort(TD; Score) return AD 6 6.1
To showcase the working of the proposed approach, this paper considers the same hypothetical project scenarios that were used by Ferrari and Esuli [FE19]. They involve ve domains: computer science (CS), electronic engineering (EE), mechanical engineering (ME), medicine (MED), and sports (SPO).
Light Controller [CS, EE]: an embedded software for room illumination system Mechanical CAD [CS, ME]: a software for designing and drafting mechanical components. Medical Software [CS, MED]: a disease-prediction software.
Athletes Network [CS, SPO]: a social network for athletes.
Medical Device [CS, EE, MED]: a tness tracker connected to a mobile app Medical Robot [CS, EE, ME, MED]: a computer-controlled robotic arm used for surgery.
Sports Rehab Machine [CS, EE, ME, MED, SPO]: a rehabilitation machine targeted towards athletes.
The rst four scenarios can be thought of as an interview between a requirements analyst with a CS and a domain expert, whereas the other three scenarios can be regarded as group elicitation meetings involving stakeholders from multiple domains. 6.2
The Wikipedia API for Python1 was used to construct the domain corpora by scraping articles belonging to particular categories. A maximum subcategory depth of 3 and a maximum article limit of 20,000 was set while creating each domain corpus.2 Each article text was converted to lowercase and all non-alphanumeric words and stop words were removed, followed by lemmatization. The article count, word count, and vocabulary size for each domain corpus are reported by Table 1.
1https://pypi.org/project/wikipedia/ 2Since Category:Computer science is a subcategory of Category:Electronic engineering, it was excluded while creating the EE corpus to avoid extensive overlap with the CS corpus.
implementation of the word2vec SGNS algorithm with word embedding dimension d = 50, context window size L = 10, negative sampling size = 5, and minimum frequency fmin = 10. These hyperparameters were deliberately kept equal to the values used by Ferarri et al. (2018) to ensure a fair comparison between their approach and the one proposed by this paper. Training of the word embeddings was followed by length normalization and dimension-wise mean centering. For aligning the word embedding spaces, the threshold was set to 0:001. The plot of average normalized residual errors for each project scenario is depicted in Figure 1. The parameters for identifying target words were set as k = 1000 and = 0:5. These hyperparameter values were chosen by the authors through informal experiments. The source code and the domain-speci c word2vec models can be found in the project repository4.
scenario are reported along with their ambiguity scores in the online Appendix 1, which is available on the project repository. In order to study the cases of disagreement between the approaches proposed by this paper and Ferrari et al. (2018), the top-5 words with the largest absolute di erences between the assigned ranks have been reported for each scenario by Table 2. The number of target words for each project scenario have also been mentioned in parenthesis.
It can be observed that most of the cases of disagreement have a higher rank, i.e. relatively lower ambiguity score assigned by the linear transformation approach proposed by this paper. Most of such cases are proper names such as robert, peter, and daniel. This is because of the problems associated with local-context approaches discussed in Section 4, and the low ambiguity score given to such words by the proposed approach is in line with the expected behavior of a global-context approach. 7
Ambiguous requirements are a major hindrance to successful software development and it is necessary to avoid them from the elicitation phase itself. Although this problem has been studied extensively, cross-domain ambiguity has attracted research only in recent times. This paper explores the applicability of a global-context approach, which makes use of linear transformation to map various domain-speci c language models into a unied embedding space, to solve this problem. From an NLP perspective, this paper is the rst attempt to apply an LSC detection method on more than two domains and also the rst work to logically and empirically compare the linear transformation method with the KNN-based one proposed by Ferrari et al. (2018).
A major challenge in applying this work in the eld of requirements engineering is the suitability of Wikipedia articles as the corpora source. The future work should be primarily directed towards identifying better corpora 3https://radimrehurek.com/gensim/ 4https://github.com/vaibhav29498/Cross-Domain-Ambiguity-Detection sources which can make this area of study more applicable for the industry. Other planned future work includes a systematic quantitative evaluation of the proposed approach, extending the approach to consider multi-word phrases, and de ning an ambiguity threshold. [ALA16]
Mikel Artetxe, Gorka Labaka, and Eneko Agirre. Learning principled bilingual mappings of word embeddings while preserving monolingual invariance. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2289{2294, Austin, Texas, November 2016. Association for Computational Linguistics.
K.K. Aggarwal and Yogesh Singh. Software Engineering. New Age International (P) Limited, 2005.
Alessio Ferrari, Beatrice Donati, and Stefania Gnesi. Detecting domain-speci c ambiguities: An NLP approach based on wikipedia crawling and word embeddings. In 2017 IEEE 25th International Requirements Engineering Conference Workshops (REW), pages 393{399, Sep. 2017. Alessio Ferrari and Andrea Esuli. An NLP approach for cross-domain ambiguity detection in requirements engineering. Automated Software Engineering, 26(3):559{598, Sep 2019.
Alessio Ferrari, Andrea Esuli, and Stefania Gnesi. Identi cation of cross-domain ambiguity with language models. In 2018 5th International Workshop on Arti cial Intelligence for Requirements Engineering (AIRE), pages 31{38, Aug 2018.
Alessio Ferrari, Paola Spoletini, and Stefania Gnesi. Ambiguity and tacit knowledge in requirements elicitation interviews. Requirements Engineering, 21(3):333{355, Sep 2016.
Donald C. Gause and Gerald M. Weinberg. Exploring Requirements: Quality Before Design. Dorset House Publishing Co., Inc., New York, NY, USA, 1989.
Zellig S. Harris. Distributional structure. Word, 10(2-3):146{162, 1954. [KARPS15] [MLS13] [MSC+13]
Anna Hatty, Dominik Schlechtweg, and Sabine Schulte im Walde. SURel: A gold standard for incorporating meaning shifts into term extraction. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 1{8, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
Vivek Kulkarni, Rami Al-Rfou, Bryan Perozzi, and Steven Skiena. Statistically signi cant detection of linguistic change. In Proceedings of the 24th International Conference on World Wide Web, WWW '15, pages 625{635, Republic and Canton of Geneva, Switzerland, 2015. International World Wide Web Conferences Steering Committee.
Yitan Li, Linli Xu, Fei Tian, Liang Jiang, Xiaowei Zhong, and Enhong Chen. Word embedding revisited: A new representation learning and explicit matrix factorization perspective. In Proceedings of the 24th International Conference on Arti cial Intelligence, IJCAI'15, pages 3650{3656. AAAI Press, 2015.
Tomas Mikolov, Quoc V. Le, and Ilya Sutskever. Exploiting similarities among languages for machine translation. ArXiv, abs/1309.4168, 2013.
S. Mishra and A. Sharma. On the use of word embeddings for identifying domain speci c ambiguities in requirements. In 2019 IEEE 27th International Requirements Engineering Conference Workshops (REW), pages 234{240, Sep. 2019.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Je rey Dean. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, NIPS'13, pages 3111{3119, USA, 2013. Curran Associates Inc.
Avo Muromagi, Kairit Sirts, and Sven Laur. Linear ensembles of word embedding models. In Proceedings of the 21st Nordic Conference on Computational Linguistics, pages 96{104, Gothenburg, Sweden, May 2017. Association for Computational Linguistics.
Roger S. Pressman. Software Engineering: A Practitioner's Approach. McGraw-Hill, 2010. Peter H. Schonemann. A generalized solution of the orthogonal procrustes problem. Psychometrika, 31(1):1{10, Mar 1966.
Yogesh Singh and Ruchika Malhotra. Object-Oriented Software Engineering. PHI Learning, 2012. Marco Del Tredici and Raquel Fernandez. Semantic variation in online communities of practice. In IWCS 2017 - 12th International Conference on Computational Semantics - Long papers, 2017. Daniel Toews and Leif Van Holland. Determining domain-speci c di erences of polysemous words using context information. In Joint Proceedings of REFSQ-2019 Workshops, Doctoral Symposium, Live Studies Track, and Poster Track co-located with the 25th International Conference on Requirements Engineering: Foundation for Software Quality (REFSQ 2019), Essen, Germany, March 18th, 2019., 2019. [XWLL15]