[SOLVED] How to download multiple files using r by iteratively updating string from url using a list

Issue

I want to download over 100 files from https://biogeo.ucdavis.edu/data/gadm3.6/Rsf/gadm36_AGO_1_sf.rds. The only string that changes in the url is "AGO", which is to be replaced by another string such as "DZA" taken from a list. For example, https://biogeo.ucdavis.edu/data/gadm3.6/Rsf/gadm36_DZA_1_sf.rds will download another file. I would appreciate any help to create a function that downloads iteratively using r and writes the files in the working directory.

List of strings to iterate over:

strin<-structure(list(iso3c = c("DZA", "AGO", "BWA", "BDI", "CMR", "CPV"
)), class = c("tbl_df", "tbl", "data.frame"), row.names = c(NA, 
-6L))

My attempt:

list1 = strin %>% unlist(.)

for (i in list1){
  urls <- lapply(list1
                 , function(i) paste0("https://biogeo.ucdavis.edu/data/gadm2.8/rds/", i, "_adm2.rds" ) %>% as.character(.)
  )
}

for (u in urls){
  url <- lapply(urls, function(u) GET(u,write_disk(tempfile(fileext = ".RDS")))
  )
}

infolist1 =rownames_to_column(
                  data.frame(
                    as.matrix(
                      unlist(
                        lapply(url, `[`, c('url', 'status_code', 'content'))
                        )
                      )
                    )
                  , var = "rowname")

infolist1$rowname= gsub(".*\\.","",infolist1$rowname) 
colnames(infolist1) = c("rowname", "data1")

c2 = infolist1 %>%  subset(., rowname == "status_code", select = c(data1)) 
c3= infolist1 %>%   subset(., rowname == "content", select = c(data1)) 

infolist1 = cbind(c2,c3)  %>% unlist(.)
rm(list=c('c2', 'c3'))
colnames(infolist1) = c("urlx", "filestatus", "filepath")

#Error message:

Error in colnames<-(tmp, value = c("urlx", "filestatus", "filepath" :
attempt to set ‘colnames’ on an object with less than two dimensions’

Solution

You show a lot of extra code that extends past the question from your title.

To download the files, I would use download.file

strin<-structure(list(iso3c = c("DZA", "AGO", "BWA", "BDI", "CMR", "CPV"
)), class = c("tbl_df", "tbl", "data.frame"), row.names = c(NA, 
-6L))

for (ss in strin$iso3c)
    download.file(
        paste0("https://biogeo.ucdavis.edu/data/gadm2.8/rds/", ss, "_adm2.rds"),
        paste0(ss, "_adm2.rds"))

This stores all files (except for "CPV_adm2.rds" which doesn’t seem to exist) in the current working folder.

Answered By – Maurits Evers

Answer Checked By – Terry (BugsFixing Volunteer)