Contengency table

Contengency table of responded (yes, no) vs the type of question

Test Results

Here’s the test results

chisq.test(conTable)
## 
##  Pearson's Chi-squared test
## 
## data:  conTable
## X-squared = 1848.9, df = 4, p-value < 2.2e-16
chisq.test(conTable, simulate.p.value = TRUE, B = 10000)
## 
##  Pearson's Chi-squared test with simulated p-value (based on 10000
##  replicates)
## 
## data:  conTable
## X-squared = 1848.9, df = NA, p-value = 9.999e-05

Residuals

Post-hoc analysis

Using Fisher’s Exact. Agrees nicely with the results from the logistic regression (next section)

update No longer working because fifer package removed

Logit model

Run a bimodal regression based on Question type + grade + race + gender + number of user's responses + Campaign number (e.g. is it the first message sent, second sent, last message sent in the campaign). In the future, can look at how category and/or keywords play into this.

The next table (Table 6) shows the model in a more pretty table format. Table 7 shows the same model, but with an output more similar to SPSS.

Prediction

Based on modeling only the significant variables from above, which has a higher AIC (second model’s AIC=12,331) than the model above (AIC=8,090). Note that the figure shows three points in time (beginning, midpoint, and end), but assumes userResponses is the same across the board

Multilevel logit model

The next output below models the fixed effects of variables of interest (e.g. question type) while accounting for random effects (userID). Note: the model doesn’t converge if gender, race, or grade is included, presumably because those effects are accounted for in the random effects of the userID

## Generalized linear mixed model fit by maximum likelihood (Laplace
##   Approximation) [glmerMod]
##  Family: binomial  ( logit )
## Formula: Replied ~ Type + (1 | userID)
##    Data: df
## 
##      AIC      BIC   logLik deviance df.resid 
##  12952.6  12999.4  -6470.3  12940.6    18068 
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -4.1588 -0.3957 -0.1975 -0.0403 24.7967 
## 
## Random effects:
##  Groups Name        Variance Std.Dev.
##  userID (Intercept) 2.119    1.456   
## Number of obs: 18074, groups:  userID, 414
## 
## Fixed effects:
##                     Estimate Std. Error z value Pr(>|z|)    
## (Intercept)          -5.3448     0.2489 -21.474   <2e-16 ***
## TypeKnowledge         0.5810     0.2626   2.212   0.0269 *  
## TypeKnowledge w/ EX   3.4863     0.2392  14.572   <2e-16 ***
## TypeQuiz              4.2730     0.2426  17.613   <2e-16 ***
## TypeT/F               4.7069     0.2411  19.524   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr) TypKnw TKw/EX TypeQz
## TypeKnowldg -0.842                     
## TypKnwlw/EX -0.942  0.876              
## TypeQuiz    -0.934  0.864  0.967       
## TypeT/F     -0.943  0.870  0.974  0.966

Gender

Grade

Race

Combo

Gender

Grade

Race

Note that unlike gender & grade, I didn’t drop the Other race because it’s representing Two or more races

Compare to Census data (2017 ACS 5-year estimates) from Galveston County: