This folder is made available as part of the following research paper to be published in Patterns journal.

Title: Automatic Sentiment Lexicon Creation for Airport Services Reviews Using Pointwise Mutual Information. 

Abstract of the paper:
The recent drastic increase in internet users has led to the explosive growth of opinion data. Hence, sentiment analysis has attracted much attention in the past few years. The analysis of users’ sentiment depends on the quality of sentiment lexicons. There are many existing general sentiment lexicons.
However, they cannot cover all the domain-specific words where the meaning of these words may differ according to the domain. Therefore, the automation of domain-specific sentiment lexicon creation has become an essential task. In this study, we propose a novel method to generate domain-specific sentiment lexicons for airport service reviews utilising the VADER sentiment lexicon dictionary. First, we scraped the data from the SKYTRAX website, which is a collection of reviews of around 600 airports. Then, data pre-processing techniques were employed including unigrams extraction and stopwords removal. 
Having done that, we employed pointwise mutual information to calculate the scores of the extracted unigrams. Then, we updated the default scores of VADER with the pointwise mutual information scores. We evaluated our results using the performance measures of accuracy, precision, recall, and F1-score. Two popular general sentiment lexicons are used as benchmarks. The results showed that our proposed lexicon dictionary for the domain of airport reviews outperformed the benchmarks with consistent considerable improvements achieving around 10% in accuracy and around 7% in F1-score.



We have performed our experiment on a Personal Computer (PC) with Intel(R) Core(TM) i7-9700 CPU with a processor speed of 3.00 GHz and 32 GHz RAM. Python Programming Language implemented in Jupyter Notebook is used for these experiments. 
We utilised libraires such as NLTK, Scikit-Learn for the analysis, and Beautiful Soup library for scraping the reviews.


## Contents of the repository
This repository contains resources used and described in the paper.

The repository is structured as follows:
> **NOTE:** this should follow the folders you have in this repository

Code:
This folder contains:
- A file called Source_code, which computes the VADER & AFINN sentiment analysis. It also computes the re-scoring calculations and results of PMI. We compare between these results and observed that updated VADER outperformed the others. 
After that, we applied 2 conditions to refine the dataset and extract the incorrectly labelled reviews, the updated VADER still outperform the others with steady improvemnts.

- A folder contains 2 output figures. The first figure called "VADER scores distribution" which shows the distributions of default VADER's scores. And the second figure called "Distribution of Updated VADER with PMI" which shows the distribution for the updated VADER incorporating PMI scores.


Data:
This folder contains:
- Dataset used for this paper called "skytrax_reviews". We only utalised the review text for the sentiment analysis.
- Also, sentiment scores file called "sentiment_scores" is used for the analysis of Vader where Vader scores have been extracted.
This dataset has been scrapped from SKYTRAX website "https://www.airlinequality.com/review-pages/a-z-airport-reviews/". 







