Maori loanwords are widely used in New Zealand English for various social functions by New Zealanders within and outside of the Maori community. Motivated by the lack of linguistic resources for studying how Maori loanwords are used in social media, we present a new corpus of New Zealand English tweets. We collected tweets containing selected Maori words that are likely to be known by New Zealanders who do not speak Maori. Since over 30% of these words turned out to be irrelevant, we manually annotated a sample of our tweets into relevant and irrelevant categories. This data was used to train machine learning models to automatically filter out irrelevant tweets.