Is Microsoft R Open faster than CRAN R?


I was finally able to install Microsoft R Open on a machine that features Ubuntu Server 16.04 so now let's test!

Microsoft© website states that "From its inception, R was designed to use only a single thread (processor) at a time. Even today, R works that way unless linked with multi-threaded BLAS/LAPACK libraries." That's true.

But also they state that "Compared to open source R, the MKL offers significant performance gains, particularly on Windows.". Microsoft R open is, to the date, the best option to install R on Windows in a painless way. I'm not so sure about the significant performance gains, specially because their website does not show different results on different hardware under a controlled experiment. So, if you are going to talk about significant differences please respect statistics.

My own results of Microsoft test

This post complements an older post. Like my referred post, this article does not present evidence to close the long discussion on benchmarks nor provides a careful test on different machines.

The benchmarks presented here are an adaptation of Simon Urbanek's benchmarks, and I'll be using the package that is used for the results exposed in Microsoft's website.

It is important to notice that I did repeat each test 100 times and the results presented here correspond to the median of the repetitions of each test. This was made after a suggestion from George Vega Yon.

This is how the test was made:

R built with MKL versus Microsoft R Open

Operations Median Time R built with Intel MKL Microsoft R Open
Cholesky factorization 0.8190 0.8270
Linear discriminant analysis 2.9425 3.1610
Matrix multiplication 4.2075 4.2745
Principal component analysis 4.6750 4.7980
QR decomposition 6.1480 5.9570
Singular value decomposition 4.1445 4.3415

R built with OpenBLAS versus Microsoft R Open

Operations Median Time R built with OpenBLAS Microsoft R Open
Cholesky factorization 0.8960 0.8260
Linear discriminant analysis 3.0875 3.1625
Matrix multiplication 4.0005 4.2675
Principal component analysis 6.0795 4.7720
QR decomposition 5.7610 5.9560
Singular value decomposition 6.4725 4.3335


These partial results show that CRAN R (aka R Open Source) built with Intel MKL numerical libraries performs better that Microsoft R Open. But, how better? This can't be extrapolated to other systems but Ubuntu.

For some operations R augmented with alternative math kernel (MKL in this case) performs better than Microsoft R Open.


I used this script to obtain the results: