Oracle has just released version 4.5.0 of the Big Data Lite VM which, when it comes to R, still suffers from the issues we had pinpointed for the previous version 4.4.0 (and then some).
The first attempt to install the additional packages fails with a ‘cannot open URL’ error:
[oracle@bigdatalite ~]$ scripts/install_additional_packages.sh
Installing additional packages
running
'/usr/lib64/R/bin/R --slave --no-restore -e install.packages("http://cran.fhcrc.org/src/contrib/gtools_3.5.0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
trying URL 'http://cran.fhcrc.org/src/contrib/gtools_3.5.0.tar.gz'
Error in download.file(p, destfile, method, mode = "wb", ...) :
cannot open URL 'http://cran.fhcrc.org/src/contrib/gtools_3.5.0.tar.gz'
In addition: Warning message:
In download.file(p, destfile, method, mode = "wb", ...) :
unable to resolve 'www-proxy.us.oracle.com'
[...]
Fortunately, the warning about the proxy helps to locate the issue, which is a forgotten proxy setting in the provided script install_additional_packages.sh (highlighted):
# Install additional open-source R packages for HOL exercises
# Main packages are arules, arulesViz and forecast plus their dependencies
export http_proxy=http://www-proxy.us.oracle.com:80
# Command no longer needed since the installation of ORAAH configures Java and RJava
# echo Configuring JAVA Environment for R
# sudo R CMD javareconf
echo Installing additional packages
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/gtools_3.5.0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/gdata_2.17.0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/whisker_0.3-2.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/xtable_1.8-2.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/digest_0.6.9.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/doParallel_1.0.10.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/foreach_1.4.3.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/gridBase_0.4-7.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/iterators_1.0.8.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/pkgmaker_0.22.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/rngtools_1.2.4.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/registry_0.3.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/stringi_1.1.1.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/magrittr_1.5.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/stringr_1.0.0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/irlba_2.0.0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/scatterplot3d_0.3-37.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/lmtest_0.9-34.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/vcd_1.4-1.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/TSP_1.1-4.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/qap_0.1-0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/gclus_1.3.1.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/dendextend_1.1.8.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/bitops_1.0-6.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/caTools_1.17.1.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/gplots_3.0.1.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/seriation_1.2-0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/quadprog_1.5-5.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/zoo_1.7-13.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/reshape2_1.4.1.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/gtable_0.2.0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/dichromat_2.0-0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/plyr_1.8.3.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/munsell_0.4.3.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/labeling_0.3.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/scales_0.4.0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/ggplot2_2.1.0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/RColorBrewer_1.1-2.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/NMF_0.20.6.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/foreach_1.4.3.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/igraph_1.0.1.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/Archive/arulesViz/arulesViz_1.03.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/Archive/tseries/tseries_0.10-34.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/fracdiff_1.4-2.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/Archive/Rcpp/Rcpp_0.12.2.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/Archive/RcppArmadillo/RcppArmadillo_0.6.200.2.0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/nnet_7.3-12.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/colorspace_1.2-6.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/timeDate_3012.100.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/Archive/forecast/forecast_7.0.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Rscript --verbose -e 'install.packages("http://cran.fhcrc.org/src/contrib/sandwich_2.3-4.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
Here is what happens if we comment out the proxy setting (line 3 above) and rerun the script; the first 5 packages are successfully installed, but when we reach doParallel (line 16), we get an error for missing dependencies:
running
'/usr/lib64/R/bin/R --slave --no-restore -e install.packages("http://cran.fhcrc.org/src/contrib/doParallel_1.0.10.tar.gz",repos=NULL,dependencies=TRUE,lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",type="source")'
trying URL 'http://cran.fhcrc.org/src/contrib/doParallel_1.0.10.tar.gz'
Content type 'application/x-gzip' length 173022 bytes (168 KB)
==================================================
downloaded 168 KB
ERROR: dependencies ‘foreach’, ‘iterators’ are not available for package ‘doParallel’
* removing ‘/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library/doParallel’
Warning message:
In install.packages("http://cran.fhcrc.org/src/contrib/doParallel_1.0.10.tar.gz", :
installation of package ‘/tmp/RtmpaSUC0i/downloaded_packages/doParallel_1.0.10.tar.gz’ had non-zero exit status
Why the missing dependencies? Well, as we have stressed (apparently not strongly enough) in the recent past, the combination of repos=NULL,dependencies=TRUE arguments, as it happens here, is invalid, as clearly indicated in the documentation:
dependencies logical indicating whether to also install uninstalled packages which
these packages depend on/link to/import/suggest (and so on recursively).
Not used if repos = NULL.
And of course, the fact that both dependencies of doParallel are indeed queued for installation further down (lines 17 & 19) does not change the issue that doParallel will not be installed.
Out of the 50 requested additional packages, 17 cannot be installed; most of them fail due to missing dependencies, as above, but arulesViz fails because there is a typo in the file name. At the end of the day, out of the 3 “main” packages (as mentioned in the comments of the installation script) arules, arulesViz, and forecast, only arules will be available (which comes preinstalled, albeit in an old version).
We have argued in the past why installing R packages in such a way makes little sense; we will not repeat the arguments here. So, if one wants the additional packages installed, here is a simple R script to do so:
pkgs = c("gtools",
"gdata",
"whisker",
"xtable",
"digest",
"doParallel",
"gridBase",
"pkgmaker",
"rngtools",
"registry",
"stringi",
"magrittr",
"stringr",
"irlba",
"scatterplot3d",
"lmtest",
"vcd",
"TSP",
"qap",
"gclus",
"dendextend",
"bitops",
"caTools",
"gplots",
"seriation",
"quadprog",
"zoo",
"reshape2",
"gtable",
"dichromat",
"plyr",
"munsell",
"labeling",
"scales",
"ggplot2",
"RColorBrewer",
"NMF",
"igraph",
"arulesViz",
"arules",
"tseries",
"fracdiff",
"RcppArmadillo",
"nnet",
"colorspace",
"timeDate",
"forecast",
"sandwich"
)
install.packages(pkgs, dependencies=TRUE,
repos="http://cran.fhcrc.org",
lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",
type="source")
where we have left some packages to be installed as dependencies, and we install again arules in its latest version (needed for arulesViz).
Run the script from your home folder, with
Rscript --verbose 'scripts/additional_packages.R'
See the last part of this post for explanation of the (minor) warnings.
A quick inspection shows that almost all of the packages in our list have been successfully installed, with two exceptions: RcppArmadillo, due to incompatibility with the existing g++ compiler, and forecast, which depends on RcppArmadillo. We can easily amend this by downloading a previous version of the former – just append the following lines in the above R script:
install.packages("http://cran.fhcrc.org/src/contrib/Archive/RcppArmadillo/RcppArmadillo_0.6.200.2.0.tar.gz",
repos=NULL,
lib="/u01/app/oracle/product/12.1.0.2/dbhome_1/R/library",
type="source")
install.packages("forecast", repos="http://cran.fhcrc.org")
Christos - Iraklis is one of our resident Data Scientists. He holds advanced graduate degrees in applied mathematics, engineering, and computing. He has been awarded both Chartered Engineer and Chartered Manager status in the UK, as well as Master status in Kaggle.com due to "consistent and stellar results" in predictive analytics contests.
Latest posts by Christos - Iraklis Tsatsoulis (see all)
- Streaming data from Raspberry Pi to Oracle NoSQL via Node-RED - February 13, 2017
- Dynamically switch Keras backend in Jupyter notebooks - January 10, 2017
- sparklyr: a test drive on YARN - November 7, 2016
[…] article was first published on R – Nodalpoint, and kindly contributed to […]
I am very happy to discover that Oracle, for their new Big Data Lite VM 4.6, have included the script above for installing the additional R packages