Hi Holger,

My two cents:

• "rvest" (https://cran.r-project.org/web/packages/rvest/rvest.pdf and https://www.reed.edu/data-at-reed/resources/R/rvest.html) and/or "XML" (http://www.omegahat.net/RSXML/shortIntro.pdf) for webscraping.

• Still in webscraping territory, "RSelenium" (https://www.rdocumentation.org/packages/RSelenium/versions/1.7.1 and http://johndharrison.github.io/RSOCRUG/) for dealing with "dynamic" websites (e.g. when you need to interact with the website for it to generate the data you want).

• "tm" (https://cran.r-project.org/web/packages/tm/vignettes/tm.pdf), "quanteda" (https://cran.r-project.org/web/packages/quanteda/vignettes/quickstart.html), or "tidytext" (https://cran.r-project.org/web/packages/tidytext/vignettes/tidytext.html and their great open-source book: https://www.tidytextmining.com/) for text-mining.

• As for machine learning, there are plenty (see https://cran.r-project.org/web/views/MachineLearning.html). Almost as many as there are algorithms. A nice general library, with several algorithms and simple sintax, is offered by the "caret" package (http://topepo.github.io/caret/index.html).

Hope it helps.

Best,

José

I think R is equipping itself for handling big data.

https://www.analyticsvidhya.com/blog/2015/08/list-r-packages-data-analysis/

https://www.computerworld.com/article/2921176/business-intelligence/great-r-packages-for-data-import-wrangling-visualization.html

https://support.rstudio.com/hc/en-us/articles/201057987-Quick-list-of-useful-R-packages

https://www.quora.com/What-are-some-good-R-packages-for-analysis-of-qualitative-data

Article Data management by using R: Big data clinical research series

Holger, please consider applying as an exert in the next summer school of computational social sciences https://bigsss-css.jacobs-university.de/ (call should open soon). I will send you a PM with gesis stuff.

type "method(class = "big data")", it may help. Also try ??big data. All at the console

Hi Holger,

I'ld like to add "rattle" to your summary: https://rattle.togaware.com/

A very easy to use GUI - and for this not to be seen as a stand-alone-package - for Data Mining in R.

Recommended as "three-klick-quick-'n-dirty" - overview before diggin' deeper.

Best regards

Andreas

Hi Holger,

My two cents:

• "rvest" (https://cran.r-project.org/web/packages/rvest/rvest.pdf and https://www.reed.edu/data-at-reed/resources/R/rvest.html) and/or "XML" (http://www.omegahat.net/RSXML/shortIntro.pdf) for webscraping.

• Still in webscraping territory, "RSelenium" (https://www.rdocumentation.org/packages/RSelenium/versions/1.7.1 and http://johndharrison.github.io/RSOCRUG/) for dealing with "dynamic" websites (e.g. when you need to interact with the website for it to generate the data you want).

• "tm" (https://cran.r-project.org/web/packages/tm/vignettes/tm.pdf), "quanteda" (https://cran.r-project.org/web/packages/quanteda/vignettes/quickstart.html), or "tidytext" (https://cran.r-project.org/web/packages/tidytext/vignettes/tidytext.html and their great open-source book: https://www.tidytextmining.com/) for text-mining.

• As for machine learning, there are plenty (see https://cran.r-project.org/web/views/MachineLearning.html). Almost as many as there are algorithms. A nice general library, with several algorithms and simple sintax, is offered by the "caret" package (http://topepo.github.io/caret/index.html).

Hope it helps.

Best,

José

Hi Holger,

I would also add to your list:

• robotstxt: Provides functions to download and parse ‘robots.txt’ files. Ultimately the package makes it easy to check if bots (spiders, crawler, scrapers, …) are allowed to access specific resources on a domain. Just to keep yourself legal ;) (https://github.com/ropensci/robotstxt).
• skimr: provides a frictionless approach to summary statistics which conforms to the principle of least surprise, displaying summary statistics the user can skim quickly to understand their data. (https://github.com/ropensci/skimr)
• streamR: Functions to access Twitter's filter, sample, and user streams, and to parse the output into data frames. (https://github.com/pablobarbera/streamR)
• rtweet: R client for accessing Twitter’s REST and stream APIs.
• instaR: This package provides a series of functions that allow R users to access Instagram's API to search for media that mention specific hashtags or were sent from a certain location, to download recent pictures from a given users, and to capture lists of followers. (https://github.com/pablobarbera/instaR)

Hope it helps.

Best,

Víctor