This paper describes a submission to the Word-in-Context competition for the
IJCAI 2019 SemDeep-5 workshop. The task is to determine whether a given focus
word is used in the same or different senses in two contexts. We took an
ELMo-inspired approach similar to the baseline model in the task description
paper, where contextualized representations are obtained for the focus words
and a classification is made according to the degree of similarity between
these representations. Our model had a few simple differences, notably joint
training of the forward and backward LSTMs, a different choice of states for
the contextualized representations and a new similarity measure for them. These
changes yielded a 3.5\% improvement on the ELMo baseline.