Expanding on the end-to-end memory network for goal-oriented dialogue

Abstract

A series of end-to-end models have been proposed in order to satisfy therequirements of the Dialog System Technology Challenge: building an end-to-end dialog system for goal-oriented applications. While these modelshave proven to be a good solution for such tasks, they perform worse whendealing with out-of-vocabulary tasks and none-synthetic data. Additionally,they rely heavily on the use of an underlying knowledge base to achieve goodresults.We propose two new models that build on the end-to-end memory net-work architecture. The goal of these two models is to better handle out-of-vocabulary tasks and none-synthetic data. The first model changes thebag-of-words representation of the data, into a paragraph vector represen-tation treating all data sentences as unseen sentences. We call this modelthe distributed bag-of-words end-to-end memory network (DbowN2N). Thesecond model adds a bidirectional long short-term memory layer at thebeginning of the model used for named entity recognition, to capture thekeywords in the sentences before feeding it through the memory network.We call this network the key-tagging end-to-end memory network (KTN2N).In our experiments, the DbowN2N model achieves similar results to thatof the state of the art regular memory network, suggesting that bag-of-wordsrepresentation of the sentences are as effective as Distributed bag-of-wordsrepresentations for dealing with tasks like this. The KTN2N model achievesa considerable increase in accuracy over the plain memory network andcomparable results with state of the art memory networks such as the gatedmemory network and unified weight-tying memory network.