[SOLVED] How to speed up GLM estimation?

Issue

I am using RStudio 0.97.320 (R 2.15.3) on Amazon EC2. My data frame has 200k rows and 12 columns.

I am trying to fit a logistic regression with approximately 1500 parameters.

R is using 7% CPU and has 60+GB memory and is still taking a very long time.

Here is the code:

glm.1.2 <- glm(formula = Y ~ factor(X1) * log(X2) * (X3 + X4 * (X5 + I(X5^2)) * (X8 + I(X8^2)) + ((X6 + I(X6^2)) * factor(X7))), 
  family = binomial(logit), data = df[1:150000,])

Any suggestions to speed this up by a significant amount?

Solution

There are a couple packages to speed up glm fitting. fastglm has benchmarks showing it to be even faster than speedglm.

You could also install a more performant BLAS library on your computer (as Ben Bolker suggests in comments), which will help any method.

Answered By – Gregor Thomas

Answer Checked By – Katrina (BugsFixing Volunteer)

Leave a Reply

Your email address will not be published. Required fields are marked *