Will Google Drop TensorFlow?

google open source TensorFlow in 2015. Major organizations such as Uber, Airbnb, NASA, GE Healthcare and Twitter have co-opted TensorFlow to optimize their operations. The open-source Python library is mainly used for the training and inference of deep neural networks. For example, GE Healthcare uses TensorFlow to increase the speed and accuracy of MRIs in identifying specific body parts.

Meta AI released PyTorch, an open-source machine learning platform, in 2016. PyTorch enables faster prototyping than TensorFlow. In addition, it is more closely integrated with the Python ecosystem than TensorFlow, and the debugging experience is much simpler. PyTorch can be traced using one of the many widely available Python debugging tools. However, in the case of TensorFlow, developers need to learn tfdbg (TensorFlow debugger) to evaluate TensorFlow expressions at runtime.

In addition, Pytorch provides a comprehensive set of tools, libraries, pre-trained models, and datasets for every stage of development, enabling the developer community to quickly create and deploy AI innovations at scale.

Source: the gradient

Furthermore, the flaws in TensorFlow helped the rise of PyTorch. Although TensorFlow has upgraded the binaries, the latest version of TensorFlow 2.9.0 does not work on CUDA 11.7 (the latest version).

JAX, the rising star

In 2020, DeepMind Technologies, a subsidiary of Alphabet Inc, said it is using JAX to accelerate AI/ML research, sparking discussions about the future of TensorFlow.

Only After execution or JAX is a relatively new machine learning framework developed by Google Research Teams for high-performance numerical computing. The Python library API is based on NumPy, a collection of functions used in scientific computing.

JAX includes an extensible system of configurable function transformations such as differentiation, vectorization, and JIT compilation to support research into machine learning† ML researchers often apply a single function to a lot of data. The automatic vectorization feature in JAX simplifies it. Furthermore, the NumPy-based API and JIT compilation enable researchers to easily scale high-performance computing to one or more accelerators.

JAX offers a simpler design for dealing with the most complex machine learning problems. It is highly optimized and supports large scale data parallelism that allows for the distribution of complex technical issues across multiple TPUs rather than running separate pieces of code in different chips. With JAX you get access to so many TPUs. Furthermore, in JAX, the code written for one project can be reused for other projects.

Does JAX Overshadow TensorFlow?

JAX is flexible, easy to apply and enables rapid experimentation with new algorithms and architectures. However, JAX faces several challenges. First, JAX has no option to simply load and pre-process data and requires TensorFlow or PyTorch to handle much of the installation. Second, JAX does not allow higher-level model abstraction and has no implementation portability. Third, the JAX library is not rich and for most projects, researchers have to write the entire code from scratch.

On the other hand, TensorFlow has built an extensive infrastructure over the years – be it open source projects, pre-trained models, tutorials, higher level abstractions (via Keras), and portability to deployment destinations. In May 2022, Google released TensorFlow 2.9 with several new and improved features:

The latest version has integrated the oneDNN performance library with TensorFlow to achieve better performance on Intel CPUs.

DTensor, a new one TensorFlow API for distributed model processing, can move seamlessly from data parallelism to model parallelism.

TraceType for tf.function makes it easy to understand retracing rules.

A new experimental version of the Keras Optimizer API provides a more unified and comprehensive catalog of built-in optimizers.

Improved determinism performance

New support for Windows WSL2.

“JAX is a powerful framework for general numerical/scientific calculations (eg matrix operations, etc.) at scale. Being more than just a deep learning framework, it offers much more flexibility compared to TensorFlow through both high and low level APIs. On the other hand, as it is still under development and not as mature as Tensorflow, it is currently mainly used by researchers and not developers,” said Anshuman Gupta, VP Data Science, MiQ Digital India.

TensorFlow has been adopted in various industries such as healthcare, retail, social media, education, etc. Several products have been built on TensorFlow with the help of Keras–a widely used deep learning library to create machine learning models. On the other hand, JAX is a young framework.

If JAX can provide the feasibility and flexibility to run complex models in the future, then Google could switch to JAX as the optimal framework over TensorFlow.

Leave a Comment

Your email address will not be published.