Final Group Project: AirBnB analytics

Exploratory Data Analysis (EDA)

In the R4DS Exploratory Data Analysis chapter, the authors state:

“Your goal during EDA is to develop an understanding of your data. The easiest way to do this is to use questions as tools to guide your investigation…EDA is fundamentally a creative process. And like most creative processes, the key to asking quality questions is to generate a large quantity of questions.”

Conduct a thorough EDA. Recall that an EDA involves three things:

• Looking at the raw values.
  • dplyr::glimpse()
• Computing summary statistics of the variables of interest, or finding NAs
  • mosaic::favstats()
  • skimr::skim()
• Creating informative visualizations.
  • ggplot2::ggplot()
    • geom_histogram() or geom_density() for numeric continuous variables
    • geom_bar() or geom_col() for categorical variables
  • GGally::ggpairs() for scaterrlot/correlation matrix
    • Note that you can add transparency to points/density plots in the aes call, for example: aes(colour = gender, alpha = 0.4)

You may wish to have a level 1 header (#) for your EDA, then use level 2 sub-headers (##) to make sure you cover all three EDA bases. At a minimum you should address these questions:

• How many variables/columns? How many rows/observations?
• Which variables are numbers?
• Which are categorical or factor variables (numeric or character variables with variables that have a fixed and known set of possible values?
• What are the correlations between variables? Does each scatterplot support a linear relationship between variables? Do any of the correlations appear to be conditional on the value of a categorical variable?

At this stage, you may also find you want to use filter, mutate, arrange, select, or count. Let your questions lead you!

In all cases, please think about the message your plot is conveying. Don’t just say “This is my X-axis, this is my Y-axis”, but rather what’s the so what of the plot. Tell some sort of story and speculate about the differences in the patterns in no more than a paragraph.

listings <- listings %>% 
  mutate(price = parse_number(price))

listings <- listings %>%
  mutate(cleaning_fee = case_when(
    is.na(cleaning_fee) ~ ______, 
    TRUE ~ cleaning_fee
  ))

listings <- listings %>%
  mutate(prop_type_simplified = case_when(
    property_type %in% c("Apartment","______", "______","______") ~ property_type, 
    TRUE ~ "Other"
  ))
  

listings %>%
  count(property_type, prop_type_simplified) %>%
  arrange(desc(n))        

Mapping

Visualisations of feature distributions and their relations are key to understanding a data set, and they can open up new lines of exploration. While we do not have time to go into all the wonderful geospatial visualisations one can do with R, you can use the following code to start with a map of your city, and overlay all AirBnB coordinates to get an overview of the spatial distribution of AirBnB rentals. For this visualisation we use the leaflet package, which includes a variety of tools for interactive maps, so you can easily zoom in-out, click on a point to get the actual AirBnB listing for that specific point, etc.

The following code, having created a dataframe listings with all AirbnB listings in Munich, will plot on the map all AirBnBs where minimum_nights is less than equal to four (4). You could learn more about leaflet, by following the relevant Datacamp course on mapping with leaflet

leaflet(data = filter(listings, minimum_nights <= 4)) %>% 
  addProviderTiles("OpenStreetMap.Mapnik") %>% 
  addCircleMarkers(lng = ~longitude, 
                   lat = ~latitude, 
                   radius = 1, 
                   fillColor = "blue", 
                   fillOpacity = 0.4, 
                   popup = ~listing_url,
                   label = ~property_type)

Regression Analysis

For the target variable \(Y\), we will use the cost for two people to stay at an Airbnb location for four (4) nights.

Create a new variable called price_4_nights that uses price, cleaning_fee, guests_included, and extra_people to calculate the total cost for two people to stay at the Airbnb property for 4 nights. This is the variable \(Y\) we want to explain.

Use histograms or density plots to examine the distributions of price_4_nights and log(price_4_nights). Which variable should you use for the regression model? Why?

Fit a regression model called model1 with the following explanatory variables: prop_type_simplified, number_of_reviews, and review_scores_rating.

• Interpret the coefficient review_scores_rating in terms of price_4_nights.
• Interpret the coefficient of prop_type_simplified in terms of price_4_nights.

We want to determine if room_type is a significant predictor of the cost for 4 nights, given everything else in the model. Fit a regression model called model2 that includes all of the explanantory variables in model1 plus room_type.

Diagnostics, collinearity, summary tables

As you keep building your models, it makes sense to:

1. Check the residuals, using autoplot(model_x)

2. As you start building models with more explanatory variables, make sure you use `car::vif(model_x)`` to calculate the Variance Inflation Factor (VIF) for your predictors and determine whether you have colinear variables. A general guideline is that a VIF larger than 5 or 10 is large, and your model may suffer from collinearity. Remove the variable in question and run your model again without it.

3. Create a summary table, using huxtable (https://mam2021.netlify.app/example/modelling_side_by_side_tables/) that shows which models you worked on, which predictors are significant, the adjusted \(R^2\), and the Residual Standard Error.

4. Finally, you must use the best model you came up with for prediction. Suppose you are planning to visit the city you have been assigned to over reading week, and you want to stay in an Airbnb. Find Airbnb’s that are apartment with a private room, have at least 10 reviews, and an average rating of at least 90. Use your best model to predict the total cost to stay at this Airbnb for 4 nights. Include the appropriate 95% interval with your prediction. Report the point prediction and interval in terms of price_4_nights.

• if you used a log(price_4_nights) model, make sure you anti-log to convert the value in $

Deliverables

• By midnight on Tuesday 21 Sep 2021, you must upload on Canvas a short presentation (max 4-5 slides) with your findings, as some groups will be asked to present in class. You should present your Exploratory Data Analysis, as well as your best model. In addition, you must upload on Canvas your final report, written using R Markdown to introduce, frame, and describe your story and findings. You should include the following in the memo:

  • Executive summary
  • Background information and summary of the data
  • Explanation, description, and code for each individual plot/table/regression model
  • Summary of the process of analysis
  • Comparison of various models, using huxtable::huxreg()
  • Rationale for the final model; significance and some diagnostics
  • Applying your model to predict price_4_nights

Remember to follow R Markdown etiquette rules and style; don’t have the Rmd output extraneous messages or warnings, include summary tables in nice tables (use kableExtra), and remove any placeholder texts from past Rmd templates.

Performing peer review

This group assignment will be graded on a rubric. Part of your grade (5/30) will be the outcome of a peer review as discussed below. As part of this assignment you will be reviewing, commenting on, and marking other students’ assignments. Specifically, you will email the Rmd of your work to the group who worked on the same city, and you will have to assess the Rmd they send you.

## 
## -- Column specification --------------------------------------------------------
## cols(
##   Category = col_character(),
##   `Needs Work` = col_character(),
##   Satisfactory = col_character(),
##   Excellent = col_character()
## )

Acknowledgements

• The data from this lab is from insideairbnb.com
• The material on peer review is derived in part from UBC STAT 545, licensed under the CC BY-NC 3.0 Creative Commons License.