Artificial Intelligence

So, how come we can use TensorFlow from R?

November 26, 2024
aitoolsadmin



Which computer language is most closely associated with TensorFlow? While on the TensorFlow for R blog, we would of course like the answer to be R, chances are it is Python (though TensorFlow has official bindings for C++, Swift, Javascript, Java, and Go as well).

So why is it you can define a Keras model as

library(keras)
model <- keras_model_sequential() %>%
  layer_dense(units = 32, activation = "relu") %>%
  layer_dense(units = 1)

(nice with %>%s and all!) – then train and evaluate it, get predictions and plot them, all that without ever leaving R?

The short answer is, you have keras, tensorflow and reticulate installed.
reticulate embeds a Python session within the R process. A single process means a single address space: The same objects exist, and can be operated upon, regardless of whether they’re seen by R or by Python. On that basis, tensorflow and keras then wrap the respective Python libraries and let you write R code that, in fact, looks like R.

This post first elaborates a bit on the short answer. We then go deeper into what happens in the background.

One note on terminology before we jump in: On the R side, we’re making a clear distinction between the packages keras and tensorflow. For Python we are going to use TensorFlow and Keras interchangeably. Historically, these have been different, and TensorFlow was commonly thought of as one possible backend to run Keras on, besides the pioneering, now discontinued Theano, and CNTK. Standalone Keras does still exist, but recent work has been, and is being, done in tf.keras. Of course, this makes Python Keras a subset of Python TensorFlow, but all examples in this post will use that subset so we can use both to refer to the same thing.

So keras, tensorflow, reticulate, what are they for?

Firstly, nothing of this would be possible without reticulate. reticulate is an R package designed to allow seemless interoperability between R and Python. If we absolutely wanted, we could construct a Keras model like this:

<class 'tensorflow.python.keras.engine.sequential.Sequential'>

We could go on adding layers …

m$add(tf$keras$layers$Dense(32, "relu"))
m$add(tf$keras$layers$Dense(1))
m$layers
[[1]]
<tensorflow.python.keras.layers.core.Dense>

[[2]]
<tensorflow.python.keras.layers.core.Dense>

But who would want to? If this were the only way, it’d be less cumbersome to directly write Python instead. Plus, as a user you’d have to know the complete Python-side module structure (now where do optimizers live, currently: tf.keras.optimizers, tf.optimizers …?), and keep up with all path and name changes in the Python API.

This is where keras comes into play. keras is where the TensorFlow-specific usability, re-usability, and convenience features live.
Functionality provided by keras spans the whole range between boilerplate-avoidance over enabling elegant, R-like idioms to providing means of advanced feature usage. As an example for the first two, consider layer_dense which, among others, converts its units argument to an integer, and takes arguments in an order that allow it to be “pipe-added” to a model: Instead of

model <- keras_model_sequential()
model$add(layer_dense(units = 32L))

we can just say

model <- keras_model_sequential()
model %>% layer_dense(units = 32)

While these are nice to have, there is more. Advanced functionality in (Python) Keras mostly depends on the ability to subclass objects. One example is custom callbacks. If you were using Python, you’d have to subclass tf.keras.callbacks.Callback. From R, you can create an R6 class inheriting from KerasCallback, like so

CustomCallback <- R6::R6Class("CustomCallback",
    inherit = KerasCallback,
    public = list(
      on_train_begin = function(logs) {
        # do something
      },
      on_train_end = function(logs) {
        # do something
      }
    )
  )

This is because keras defines an actual Python class, RCallback, and maps your R6 class’ methods to it.
Another example is custom models, introduced on this blog about a year ago.
These models can be trained with custom training loops. In R, you use keras_model_custom to create one, for example, like this:

m <- keras_model_custom(name = "mymodel", function(self) {
  self$dense1 <- layer_dense(units = 32, activation = "relu")
  self$dense2 <- layer_dense(units = 10, activation = "softmax")
  
  function(inputs, mask = NULL) {
    self$dense1(inputs) %>%
      self$dense2()
  }
})

Here, keras will make sure an actual Python object is created which subclasses tf.keras.Model and when called, runs the above anonymous function().

So that’s keras. What about the tensorflow package? As a user you only need it when you have to do advanced stuff, like configure TensorFlow device usage or (in TF 1.x) access elements of the Graph or the Session. Internally, it is used by keras heavily. Essential internal functionality includes, e.g., implementations of S3 methods, like print, [ or +, on Tensors, so you can operate on them like on R vectors.

Now that we know what each of the packages is “for”, let’s dig deeper into what makes this possible.

Show me the magic: reticulate

Instead of exposing the topic top-down, we follow a by-example approach, building up complexity as we go. We’ll have three scenarios.

First, we assume we already have a Python object (that has been constructed in whatever way) and need to convert that to R. Then, we’ll investigate how we can create a Python object, calling its constructor. Finally, we go the other way round: We ask how we can pass an R function to Python for later usage.

Scenario 1: R-to-Python conversion

Let’s assume we have created a Python object in the global namespace, like this:

So: There is a variable, called x, with value 1, living in Python world. Now how do we bring this thing into R?

We know the main entry point to conversion is py_to_r, defined as a generic in conversion.R:

py_to_r <- function(x) {
  ensure_python_initialized()
  UseMethod("py_to_r")
}

… with the default implementation calling a function named py_ref_to_r:

Rcpp : You just write your C++ function, and Rcpp takes care of compilation and provides the glue code necessary to call this function from R.

So py_ref_to_r really is written in C++:

.Call(`_reticulate_py_ref_to_r`, x)
}

which finally wraps the “real” thing, the C++ function py_ref_to_R we saw above.

Via py_ref_to_r_with_convert in #1, a one-liner that extracts an object’s “convert” feature (see below)

Extending Python Guide.

In official terms, what reticulate does it embed and extend Python.
Embed, because it lets you use Python from inside R. Extend, because to enable Python to call back into R it needs to wrap R functions in C, so Python can understand them.

As part of the former, the desired Python is loaded (Py_Initialize()); as part of the latter, two functions are defined in a new module named rpycall, that will be loaded when Python itself is loaded.

Global Interpreter Lock, this is not automatically the case when other implementations are used, or C is used directly. So call_python_function_on_main_thread makes sure that unless we can execute on the main thread, we wait.

That’s it for our three “spotlights on reticulate”.

Wrapup

It goes without saying that there’s a lot about reticulate we didn’t cover in this article, such as memory management, initialization, or specifics of data conversion. Nonetheless, we hope we were able to shed a bit of light on the magic involved in calling TensorFlow from R.

R is a concise and elegant language, but to a high degree its power comes from its packages, including those that allow you to call into, and interact with, the outside world, such as deep learning frameworks or distributed processing engines. In this post, it was a special pleasure to focus on a central building block that makes much of this possible: reticulate.

Thanks for reading!

Leave a Reply

Your email address will not be published. Required fields are marked *