Assignment 0

  1. 0

    Part 0: Set-up. Create a project folder with sub-folders Data, Code, Assignment, Solutions. Save the two data files in Data. Open a new R script first thing, set the working directory with setwd(...), install rio and load it with library(rio). Submit a PDF solution set and the .R file; the first line of both has the student ID and assignment number, and "group:" plus names if you worked with others.

  2. 1a

    Q1: Read in the data. Read exp2015_v2025.xlsx into R as an object called mydata. (A 2015 survey of household costs and incomes in Israel: 23,134 rows, 78 columns.)

  3. 1b

    Create newdata containing only: sex age national marital ussrborn schoolty schooly diploma weekwhrs weekwrk workstat incomems chld017.

  4. 2a

    Q2: Data types. Use summary to find each variable's type. Which are characters?

  5. 2b

    Convert each character variable (including age) to a factor with as.factor.

  6. 2c

    Create age_numeric, a numeric version of the factor age (hint: as.numeric).

  7. 3a

    Q3: Subset the data. Subset the observations in newdata to those with age between 25 and 62. Refer to the variable description file to interpret the values of age_numeric. Why do we need to limit our observations to this age range?

  8. 3b

    Limit the observations in newdata to individuals with less than 25 years of education (individuals with more than 24 years are considered data-entry errors).

  9. 4a

    Q4: Explore the data. Create a table with the following summary statistics for each numeric (non-factor) variable: average, median, minimum, maximum.

  10. 4b

    What can be learned about the variable incomems from this table?

  11. 4c

    What percentage of the observations are women? (hint: use sex)

  12. 4d

    What percentage of the observations are for people born in the USSR? (hint: use ussrborn)

  13. 5a

    Q5: New variables. Create a variable called emp which takes the value 0 if the person is unemployed and 1 otherwise (hint: use workstat).

  14. 5b

    Create a new variable for hourly wage called wage_per_hour (hint: use weekwrk, weekwhrs, incomems).

  15. 5c

    Drop (delete) the variable age_numeric you created in 2c (hint: use subset).

  16. 5d

    Remove observations that have an outlier hourly wage (higher than 1,000 NIS), while keeping those that have hourly wage of NA. Use the command is.na.

  17. 5e

    Create a new variable sex_by_nation that classifies individuals into one of the following groups (and NA otherwise): Jewish men, Arab men, Jewish women, Arab women. Save it as a factor.

  18. 6

    Q6: Save the cleaned data. The object newdata will be used in other problem sets for this class. Save the cleaned version in your Data folder as HW1-cleanData.RData using the command export.

  19. 7a

    Q7: Explore across different groups. Calculate the average hourly wage for each of the groups in 5e.

  20. 7b

    Describe in a couple of sentences the wage gaps between these different groups. What are some things that could cause these gaps?

  21. 7c

    For the same groupings in 5e, calculate the employment rates (hint: use emp from 5a).

  22. 8

    Q8: Marriage in the labour market. Through independent analysis, answer: does being married provide an advantage or a disadvantage in the labour market?

    a. Address differences in both wage and employment between married and unmarried people.

    b. Does your answer change between men and women?

  23. 9a

    Q9: Expectation, variance, covariance. XX is a random variable with mean μX\mu_X and variance σX2\sigma_X^2, and YY has mean μY\mu_Y and variance σY2\sigma_Y^2. Let Cov(X,Y)=σXY\text{Cov}(X,Y) = \sigma_{XY}. Evaluate:

    a. E[4+6X]\mathbb{E}[4 + 6X]

  24. 9b

    b. (μX)2+σX2(\mu_X)^2 + \sigma_X^2

  25. 9c

    c. Var(11−3X)\text{Var}(11 - 3X)

  26. 9d

    d. Cov(X−8, Y−2)\text{Cov}(X - 8,\, Y - 2)

  27. 9e

    e. Cov(6X, 2Y)\text{Cov}(6X,\, 2Y)

  28. 9f

    f. Var(8X−6Y)\text{Var}(8X - 6Y)

  29. 10a

    Q10: Summation identities. Prove the following equalities hold.

    a.

    ∑i=1n(Xi−Xˉ)=0\sum_{i=1}^{n} (X_i - \bar{X}) = 0
  30. 10b

    b.

    ∑i=1n(Xi−Xˉ)2=∑i=1n(Xi−Xˉ) Xi\sum_{i=1}^{n} (X_i - \bar{X})^2 = \sum_{i=1}^{n} (X_i - \bar{X})\, X_i
  31. 10c

    c.

    ∑i=1n(Xi−Xˉ)(Yi−Yˉ)−∑i=1nXiYi=−n Xˉ Yˉ\sum_{i=1}^{n} (X_i - \bar{X})(Y_i - \bar{Y}) - \sum_{i=1}^{n} X_i Y_i = -n\,\bar{X}\,\bar{Y}
  32. 11a

    Q11: A weighted average of independent variables. X1,…,XnX_1, \dots, X_n are independent random variables with Var(Xi)=σ2\text{Var}(X_i) = \sigma^2 and E[Xi]=μ\mathbb{E}[X_i] = \mu for all ii. Let

    Y=∑i=1nbiXin,Y = \frac{\sum_{i=1}^{n} b_i X_i}{n},

    where b1,…,bnb_1, \dots, b_n are non-random real numbers.

    a. Find E[Y]\mathbb{E}[Y] in terms of μ\mu and the bib_i's.

  33. 11b

    b. Find Var(Y)\text{Var}(Y) in terms of σ2\sigma^2 and the bib_i's.

Solutions are Alfie's submitted work, not an official key.

Part B (R data handling) comes from Alfie's own R script, re-run on the raw data to check the outputs. Part A (statistics review, Q9 to Q11) has no solution in the source and is left as TODO. The cleaned newdata object from this assignment is reused in later problem sets.

Course review: Lec_13-Exam Review