Natural language interactions between humans and robots are intended to be situated in the sense that both user and robot can access and refer to the shared environment. Contextual knowledge plays a key role in resolving the ambiguities inherent in interpretation tasks. In addition, we expect the interpretation produced to be well-founded, e.g., that all mentions of entities in the environment (as perceived by the robot) are correctly grounded. In this paper, we propose the application of a transformer-based architecture that combines the input utterance with a linguistic description of the environment to produce interpretations and references to the environment in an end-to-end fashion. Experimental results demonstrate the robustness of the proposed methodology, overcoming previous approaches in which linguistic interpretation and grounding are composed of possible complex processing chains.
In a world that is moving towards a widespread use of virtual assistants (at home, for work, or as a hobby) and robotic platforms that perform dificult or risky tasks for humans, making sure these technologies understand human language becomes increasingly important. Virtual assistants are designed to satisfy a user need that is often informational or merely entertaining, like in the daily requests made to search for translation of individual words or the requests for popular songs. Understanding a command or the title of a song turns out to be crucial to satisfying these needs in a natural manner. The quality of such interpretation processes is important, especially in critical scenarios, e.g. involving robotic platforms that perform sensitive, medical or critical tasks usually carried out under speech-based control.
The use of natural language to control these platforms or teach them the movements or actions needed to perform a task could be a key factor in the not-too-distant future. Today, there are countless domestic robots whose job is to perform domotic tasks, such as automatic cleaning or even cooking. As suggested in [2], domestic robots have to contend with complex problems such as (1) self-localization and navigation in complex environments, (2) precise recognition of objects and people, (3) manipulation of physical objects, and (4) meaningful interaction with humans to satisfy their needs, being them physical or abstract (Human Robot Interaction).
It is necessary to make these home automation assistants aware of their surroundings and the elements in them. In order to correctly interpret a sentence such as “Take the volume on the table near the window′′ (1) it is necessary to support capability for entity association able to retrieve objects mentioned in the command (such as volume, table and window) and possibly disambiguate between entities of the same type. For example, in the case where several tables exist in the environment, it is necessary to explicitly guide the interpretation towards the one that is next to the window.
Several works, such as [3] proposed specific methods for a Grounded language interpretation of robotic commands. We investigate here the approach recently proposed in [4], namely Grounded language Understanding via Transformers (GrUT): this method suggests adopting a Transformer-based architecture (e.g., BART presented in [5]) that can produce the linguistic interpretation of an utterance by taking in input i) the transcription of the input command, ii) a linguistic description of the entities from the map involved in the command and iii) a linguistic description of the robot’s capabilities.
GrUT is appealing as it drastically reduces the need for task-specific engineering of the model (such as [3]): it only requires a way to linguistically describe a map, so that the Transformer generates interpretations consistent both with the utterance and the map. In other words, the same command can generate diferent linguistic interpretations when coupled with diferent map descriptions. Let us consider the utterance in example 1. Whenever the volume is actually next to the table, the input utterance is extended with the additional synthetic text “v1 is a volume, t1 is a table and v1 is near t1” so that GrUT produces the following interpretation: Taking(︀ Theme("the volume on the table near the window"))︀ . On the contrary, GrUT generates “Bringing(︀ Theme("the volume”),Goal("on the table near the window”))︀ whenever these objects are far from each other, that can be expressed by “v1 is a volume, t1 is a table and v1 is far from t1”. The final interpretation is consistent with Frame Semantics [ 6]: in the first case, the robot is expected to move towards the table (i.e., t1) and take the volume (i.e., v1), while in the second case it is expected to take the volume that is far away from the table and bring it over there. However, while the actual state of the environment afects the interpretation process, GrUT only produces just an approximate linguistic efect, expressed by texts fragments as fillers of the output argument predicate structure. A further step is still required to ground each fragment to the intended entity triggered by the predicate.
In this paper, we propose an end-to-end grounded interpretation process1, so that GrUT is expected to produce indexed representations in the robot KB, such as Taking(︀ Theme(v1))︀ or Bringing(︀ Theme(v1), Goal(t1))︀ . It is worth noting that the connection between words in a spoken command and the entities (here referred to as linguistic grounding) is not a trivial task. In general, as in [3], it is assumed that each entity in a map is denoted by one or more labels to enable such a grounding: for example, one or more linguistic references, such as volume or book, correspond to the object v1. These are used in [3] and [4] to retrieve all entities involved by a command from the map. The simplest approach here is to select all and only those entities whose denotation coincides with one of the command words, as applied in [4]. Unfortunately, this assumption is quite unrealistic, as for the role of synonyms or paraphrases used to refer to 1We released an extended version of GrUT at https://github.com/crux82/grut objects in user commands: “take the handbook . . . ” or “take the tome . . . ” can be equivalently used in natural language interaction. To overcome this limitation, a more complex retrieval function is explored in this paper. It makes use of more expressive associations between several linguistic labels including words that are highly similar according to a neural semantic similarity function. To the best of our knowledge, this expands recent research like [7] and [8] as it is the ifrst end-to-end technique for Fully Grounded Linguistic Interpretation, made dependent on an explicit (logical) description of the environment.
Results indicate that the adoption of expressive functions for linguistic grounding enables an end-to-end process that is even more robust than an a-posteriori application of the linguistic grounding process to the interpretations produced by GrUT.
In the rest of this paper, section 2 summarizes the related work, section 3 presents the proposed extension of GrUT, section 4 reports the experimental evaluation, while section 5 derives some conclusions.
The semantic interpretation of texts or spoken utterances is generally modeled as a Semantic
Role Labeling (SRL) task, which consists of identifying all the expressed linguistic predicates
(such as Bringing vs. Taking evoked by the verb "to take") and their corresponding semantic
arguments (such as "the volume" or "on the table near the window") in order to perform a deep
semantic interpretation of a human-generated utterance [9]. Data-driven approaches for SRL
have gotten a lot of attention since the pioneering work of Gildea and Jurafsky [10], leading
to multiple benchmarking initiatives [11, 12, 13]. As in [
The authors of [
All of the aforementioned methods, however, simply consider linguistic evidence. According
to the concept of Grounded Semantic Role Labeling (G-SRL) in [
It’s interesting to note that texts are annotated using the Frame Semantics theory [6], which
[3] suggests can be exploited by the robot to directly derive the necessary plan and action
primitives. However, [3] adopts the traditional SRL processing chain. Additionally, their output
still only exists at the linguistic level: labeling only defines roles associated with words, not
with actual objects, whereas interpretation depends on correlations between words and objects
in the environment. As suggested in [
Recently, in [4] Grounded language Understanding via Transformers (GrUT) is proposed as a sequence-to-sequence (seq2seq) approach for GSRL, sensitive to the map information in form of linguistic descriptions and capable of directly perform Grounding during interpretation, efectively linking entities in the map with the Arguments predicted. In this paper, we extend GrUT to make it end-to-end, thus allowing richer interpretations that are also grounded in the environment. Moreover, diferent and more expressive policies to retrieve entities from the map are investigated.
As discussed in the previous section, the semantic interpretation of spoken commands (and in general texts) strongly benefits from the application of Transformer-based architectures such as BART [5]. In a nutshell, a Transformer is applied to interpretation processes by taking in input a text expressed in natural language, and “translating” it into an artificial text reflecting the underlying linguistic predicate. In order to extend the application of Transformer to Grounded SRL tasks, the idea behind GrUT [4] is to use a natural language description of the map and add it to the input sentence. If we want to make the interpretation sensitive to the entities, their proprieties, position and relational information (such as proximity or distance), we first need a way to refer to them. In our context, one entity is known through its (English) noun (possibly its most commonly used lexical reference, e.g. the word volume) as well as its conceptual type. The association with the environment (i.e. the grounding) is realized through its identifier (Existence Constraint, ) that is linked to the position of the corresponding physical object in the environment. For example, the map to be paired with the command in (1) can give rise to the following description: EC: "1, also known as volume or book, is an instance of the class Book, 1, also known as table, is an instance of the class Table and 1, also known as window, is an instance of the class Window.". Moreover, if book 1 and the table 1 are close to each other in the environment, a further declaration of a Proximity Constraint ( ) acting over them will be added: PC: "1 is near 1 and 1 is near 1" Finally, for each selected entity, a description of whether the property containing other objects is true (Containability Constraint, ) will be added. For illustrative purposes, imagine the existence of a hypothetical cup 1: CC: "1 can contain other objects" The entire description is a micro-story2 useful for the SRL model to disambiguate between the diferent situations. Notice that only when the spatial constraint is true, the correct interpretation for the ambiguous situation (1) finally corresponds to the role labeled logical form:
Taking(Theme(”the volume on the table near the window”)). (2)
Since the book 1, referenced through the noun volume, is close to 1, it is interpreted thus
as the Theme of the Taking predicate. In this work, only these 3 constraints are defined and
used, but in the future a wider list of properties will be explored. It is worth noticing that
the linguistic description of the map enables the use of highly accurate transformers (such as
BART [5] and T5 [
In this paper, we extend the above method in two directions. First, we make GrUT an end-toend architecture. To this end, the transformer is trained to generate a predicate that expresses both the semantic information and the object involved. In other words, while the input is kept consistent with that used by GrUT, the output is:
Taking(︀ Theme(1)︀) and it is expected that the transformer autonomously learns this transformation. This schema produces a diferent input in the case that book 1 is far from table 1. Part of the map description will be PC: "1 is far from 1 and 1 is near 1", while the output is expected to be significantly diferent, i.e.,
Bringing(︀ Theme(1), Goal(1)︀) 2All the constraints are appended to the input, each of which is divided by a "#" delimiter character. (3) (4) Algorithm 1 GrUT compilation Algorithm 1: procedure construct_input(Sentence = (1, ..., ||), LexSim. , Threshold ) 2: ← ∅ 3: for = 1, ..., || do ◁ All entities that can be potentially referred to by a word in the command are collected to be considered in the map description. The implementation of get_candidate_entities is in Algorithm 2 4: if () == Noun then 5: ← ∪ get_candidate_entities(, , ) 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: ← ”” for ∈ do ← +" # "+_ ()+" also known as "+__ ()+
" is an instance of class "+_() ← ”” for ∈ do if () then
← +" # "+_ ()+" can contain other objects" ← ”” for 1 ∈ do for 2 ∈ do if 1 ̸= 2 ∧ (1, 2) < then
← +" # "+_ (1)+" is near "+_ (2) ← − { 1} return + + + ◁ Existence Constraints ◁ Containability Constraints ◁ Proximity Constraints
In addition, this paper extends the grounding process to improve the robustness and
applicability of GrUT. In fact, GrUT assumes that each entity in the environment is enriched
with a set of lexical references () = {1, . . . , }, used to link words (1, . . . , ||) in
the sentence . As an example let us consider the volume 1 and its corresponding lexical
reference (1) = {volume, book}. A robust linguistic grounding function is essential for
GrUT (and our extended counterpart) to build the map description. In this sense, the algorithm
2 expresses the policy adopted to retrieve the entities involved in the utterance. In a nutshell,
we propose to retrieve all entities that have significant lexical similarity with each noun in .
We will experimentally evaluate three LexicalSimilarity functions, characterized by incremental
levels of expressiveness:
1. Exact Match: the simplest function corresponds to the naive exact match between two
input strings. It produces a Boolean result, i.e., 1 if the two strings are equal, and 0
otherwise. It allows retrieving all entities that have at least one lexical reference perfectly
matching one word in the command; it is the function used in GrUT and, while it is very
precise, it fails when the user refers to entities using synonyms, e.g., referring to 1 with
handbook, tome or manual. Despite its simplicity, it assumes significant efort in map
construction: in many practical scenarios, we cannot assume that all possible lexical
references are defined for all entities.
2. Levenshtein similarity: a “soft” string matching that sometimes captures also
morphological relatedness between input word pairs. We defined a Levenshtein Similarity
Algorithm 2 Entity Retrieval Algorithm
1: procedure get_candidate_entities(Word , LexicalSimilarity , Threshold )
2: _ ← ∅
3: for ∈ _ do
4: for _ ∈ () do
5: if (, _ ) > then
6: _ ← _ ∪
7: return _
◁ For each lexical reference
() in the form
(, ) = 1 −
(, )
|(, )|
(5)
which is based on the well-known Levenshtein distance () between input strings
and . While Levenshtein distance produces values ranging from 0 to the length of the
longest word (between the two), again ranges between 0 (totally diferent strings)
and 1 (the same string). This similarity function is more robust in linking slightly diferent
input strings (e.g., book vs handbook) and it may capture some sort of morphological
analogy between words, but it fails when the user refers to entities using synonyms.
3. Neural Semantic similarity: it corresponds to the cosine similarity3 between
embeddings representing both words [
To prevent smoothed measures (such as the one based on word embeddings) from causing an excessive number of entities to be retrieved (e.g., from a map containing dozens of objects), we applied a threshold : as a result, only entities with significant similarity to one of the command words are retrieved. Our method should be robust even in cases where multiple entities are retrieved for the same word. Although the input text may contain redundant entities in the map description, the arguments in the output should contain only relevant entities. As a result, the transformer is expected to select only valid entities (using the attention mechanism of the encoder) to produce the correct output.
The goal of the following evaluation is to demonstrate the efectiveness of the proposed extension of GrUT and the diferent impacts of the lexical similarity functions we considered.
3The cosine similarity function ranges between [
The proposed approach is evaluated here in a home automation scenario, where a robot is supposed to receive spoken commands and is required to interpret commands in order to perform the expected actions. Examples are picking up a book on a table, taking out the rubbish, or looking for the keys. The evaluation is applied to the HuRIC4 dataset, which consists of 656 voice commands in English coupled with interpretations, in terms of predicates and arguments, to which the Grounding processes of the previous chapter were applied, i.e. linking them with identifiers of entities in the surrounding environment. In HuRIC, predicates are defined according to a subset of the semantic frames of FrameNet [6] and the corresponding arguments are selected. On average, each entity is represented by 1.37 lexical references. Inspired by the previous evaluation in [4] we adopted a 10-fold cross-validation scheme with 80/10/10 data split between training/validation/test and the following aspects of the extended version of GrUT are evaluated: • Frame Prediction (FP), i.e. the ability of the models to correctly generate the names of frames evoked by the voice command; it is measured as the F1-measure, where Precision and Recall reflect the capability of GrUT to recover the correct frame(s) expressed in the spoken command. • Argument Identification and Classification (AIC) as an Exact Match (AIC-ExM) evaluation in which the ability of the systems to correctly generate the names of the Arguments evoked by the command and to associate them all with the entities that evoke that Argument is evaluated; AIC-ExM is measured as the F1-Measure of produced arguments that perfectly corresponds to the gold-standard in their complete form, including frame, type of argument and corresponding grounded entities. • Argument Identification and Classification (AIC) as a more relaxed evaluation, in which the models are required to be able to associate with each Argument at least the correct Entity Head (AIC-Enty); it is measured as the F1-measure, where Precision and Recall reflect the capability of GrUT to recover the correct arguments, i.e., having the same argument type and grounded entity of the ones in the spoken command.
We compared the impact of the three proposed Lexical Similarity functions involved in the entity retrieval step. For each function, a specific threshold was estimated on the validation set by maximizing the F1-measure of the entity retrieval step. In this subtask, Precision is measured as the average percentage of entities that are correctly retrieved when constructing the map description, while Recall is measured as the average percentage of entities that were expected to be retrieved as mentioned in the command. In particular, under the Exact Match policy a definition of EM = 0.50 was applied, while under the policy based on Levenshtein Similarity and the Word Embedding one, Lev = 0.80 and WE = 0.55 are respectively used. In Table 1, the values of the main parameters used to train the Transformer are reported as the configuration maximizing the AIC-ExM on the development set, on average.
The experimental results are reported in Table 2. In the first rows, GrUT + ExPostGrounding represents our strong baseline and it is based on GrUT which, in [4] was demonstrated competitive with state-of-the-art models for Grounded Semantic Role Labeling, such as the one proposed in [3]. GrUT generates logic forms expressing the interpretation of commands at a linguistic level and in [4] it was reported to achieve 92.28% of F1 in the FP task, 88.41% in the Argument Identification and Classification Exact match (AIC-ExM) and 93.29% as the score in recovering the Semantic Head of the individual arguments.
To implement our baseline, we re-used the predictions of GrUT and applied the linguistic grounding a-posteriori: for each argument in the produced logic form, such as Goal(“on the table near the window” ), the first noun is selected (here the table) and the entity maximizing the lexical similarity is selected to replace the argument. If no entity is retrieved (or the threshold is not exceeded) the related argument is removed. We considered the three Lexical Similarity functions, i.e. based on Exact Match (EM), Levenshtein Similarity (LS) and semantic similarity estimated over Word Embeddings (WE).
The application of the grounding function causes a significant performance drop: as an example, AIC-ExM drops from 88.41% to 80.91%. This is mainly due to the 10% of entities in HuRIC whose lexical reference does not match any of the words used in a candidate argument.
The values raise from 78.62% and 80.00% for AIC tasks at argument level (indeed simpler tasks), when using the String Matching Retrieval method, to 80.91% and 82.22% for the W2V Retrieval method, showing that the word embeddings and the vector similarity function are useful for recovering some connections. The small diference between a grounding function based on the Exact Match and the one based on WEs suggests that lexical references in HuRIC are generally the same words used in the commands. The diferent entity retrieval policies do not afect the Frame Prediction subtask.
When applying our proposed model, namely GrUT-End-to-End, the Transformer-based model outperforms the baselines: +50% of error reduction on AIC tasks between ExPostGrounding 2 at 80.91% and End-to-End 2 at 90.03%. The lexical grounding based on neural representations seems indeed robust in retrieving entities, while the attention mechanism within the transformer efectively grounds the correct interpretation. The transformer efectively learns how to map entities in the descriptions of the map associated with the input command to the correct entities in the produced interpretations. It also seems to improve the results of GrUT5. The diferences between the EM, LS and WE are here more evident. When the Exact Match fails, no entity is retrieved and the argument cannot be included in the final command. On the contrary, smoother measures like the one based on WEs may introduce a super-set of the correct entities, thus introducing some noise; however, the transformer seems efective in pruning those entities not involved in the command. The diferent retrieval policies slightly afect the FP tasks: the Exact Match generally ignores several entities not retrieved in the input, and it negatively afects the capability of frame disambiguation. The efect of LS allows improving the quality of frame disambiguation also against the WE: we speculate that if “extra” entities not mentioned in the command do help the overall grounding process from one point of view, from the other these do not help the frame prediction substep. The error analysis summarized by the example in 3 confirms that most of the misinterpretations are due to errors in the linguistic grounding phase. This example refers to a simple map with two entities, 6 that is instance of the Book class, with the only lexical reference volume and 6 that is instance of the Room class, with the only lexical reference guest room. Given the command "take the book to the bed room”, the GrUT-End-to-End model adopting the Exact Match is not able to retrieve any entity, and, even though it is able to infer the correct predicate and 5The evaluation of this paper follows a slightly diferent policy than [ 4] : here the “simpler” measure just requires that since an interpretation include the correct frame, arguments and mentioned entities; in [4] evaluation is correct only when also all words in the text are correctly mapped to their corresponding frames and arguments. arguments, it simply rewrites the input text. When using Levenshtein Similarity, the "guest room" is correctly linked so that the second argument is correctly grounded. The adoption of the cosine similarity applied in the Neural Word Embedding space allows generating the input "take the book to the bed room # 6 also known as volume is an instance of class BOOK & 6 also known as guest room is an instance of class BEDROOM & 6 is far from 6", that leads to the correct interpretation. Most of the errors involve entities whose lexical references are exactly one and cannot be retrieved by the linguistic grounding function because they are uncommon nouns, e.g. the lexical reference volume for the Book type entity.
This paper presents an End-to-End sequence-to-sequence process for Grounded Semantic Role Labeling. The proposed approach suggests providing the input text, which expresses the user’s utterance enriched by a description of the surrounding environment expressed in natural language, as the input of a Transformer-based architecture. Correspondingly, the desired output is a logical form in which the entities of the environment are correctly associated with the command and grounded as well. Several policies have been applied as strategies for retrieving the entities from the map that are involved by the input utterance.
The experimental results confirm the robustness of the presented methods, especially when compared with traditional architectures chaining the linguistic interpretation of the utterance and then the linguistic grounding of the involved entity. This result is widely applicable as it does not require a costly adaptation to a particular scenario or domain.
Future work will extend the proposed methodology to consider additional properties of the environment (e.g., object properties crucial to disambiguate multiple instances of the same class, such as multiple books in a map), the user profile, or information extracted during the previous interactions between the user and the robot (exploiting the dialogue history).
We would like to thank the “Istituto di Analisi dei Sistemi ed Informatica - Antonio Ruberti" (IASI) for supporting the experimentations through access to dedicated computing resources. Claudiu Daniel Hromei is a Ph.D. student enrolled in the National Ph.D. in Artificial Intelligence, XXXVII cycle, course on Health and life sciences, organized by the Universitá Campus Bio-Medico di Roma. [2] M. E. Foster, Natural language generation for social robotics: opportunities and challenges, Philosophical Transactions of the Royal Society B: Biological Sciences 374 (2019) 20180027. doi:10.1098/rstb.2018.0027. [3] A. Vanzo, D. Croce, E. Bastianelli, R. Basili, D. Nardi, Grounded language interpretation of robotic commands through structured learning, Artif. Intell. 278 (2020). doi:10.1016/j. artint.2019.103181. [4] C. D. Hromei, C. Lorenzo, D. Croce, R. Basili, Embedding contextual information in seq2seq models for grounded semantic role labeling, in: (submitted at the) AIxIA 2022 - Advances in Artificial Intelligence - 21th International Conference of the Italian Association for Artificial Intelligence, 2022. [5] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, L. Zettlemoyer, Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension, ArXiv abs/1910.13461 (2020). [6] C. J. Fillmore, Frames and the semantics of understanding, Quaderni di Semantica 6 (1985) 222–254. [7] A. Kalyanpur, O. Biran, T. Brelof, J. Chu-Carroll, A. Diertani, O. Rambow, M. Sammons, Open-domain frame semantic parsing using transformers, 2020. URL: https://arxiv.org/ abs/2010.10998. doi:10.48550/ARXIV.2010.10998. [8] R. Blloshmi, S. Conia, R. Tripodi, R. Navigli, Generating senses and roles: An end-toend model for dependency- and span-based semantic role labeling, in: Z. Zhou (Ed.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, ijcai.org, 2021, pp. 3786–3793.
URL: https://doi.org/10.24963/ijcai.2021/521. doi:10.24963/ijcai.2021/521. [9] M. Palmer, D. Gildea, N. Xue, Semantic Role Labeling, Synthesis Lectures on Human Language Technologies, Morgan & Claypool Publishers, 2010. URL: https://doi.org/10. 2200/S00239ED1V01Y200912HLT006. doi:10.2200/S00239ED1V01Y200912HLT006. [10] D. Gildea, D. Jurafsky, Automatic Labeling of Semantic Roles, Computational Linguistics 28 (2002) 245–288. URL: https://doi.org/10.1162/089120102760275983. doi:10.1162/ 089120102760275983. [11] C. Baker, M. Ellsworth, K. Erk, SemEval-2007 task 19: Frame semantic structure extraction, in: Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval2007), Association for Computational Linguistics, Prague, Czech Republic, 2007, pp. 99–104.
URL: https://aclanthology.org/S07-1018. [12] X. Carreras, L. Màrquez, Introduction to the CoNLL-2005 shared task: Semantic role labeling, in: Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005), Association for Computational Linguistics, Ann Arbor, Michigan, 2005, pp. 152–164. URL: https://aclanthology.org/W05-0620. [13] S. Pradhan, A. Moschitti, N. Xue, O. Uryupina, Y. Zhang, Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes, in: Joint Conference on EMNLP and CoNLL-Shared Task, 2012, pp. 1–40. [14] D. Das, D. Chen, A. F. T. Martins, N. Schneider, N. A. Smith, Frame-Semantic Parsing, Computational Linguistics 40 (2014) 9–56. URL: https://doi.org/10.1162/COLI_a_00163. doi:10.1162/COLI\_a\_00163. [15] D. Marcheggiani, I. Titov, Encoding sentences with graph convolutional networks for tems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, 2013, pp. 3111–3119. URL: https://proceedings.neurips.cc/paper/2013/hash/ 9aa42b31882ec039965f3c4923ce901b-Abstract.html.