Why Swift May Be the Next Big Thing in Deep Learning

Swift for TensorFlow is Much More Than Just a LibrarySwift for TensorFlow is not just TF for another language.

It is essentially another branch (in git sense) of the Swift language itself.

This means that S4TF is not a library; it is a language in its own right, with features built into it that support all functionality needed for TensorFlow.

For example, S4TF has a very powerful automatic differentiation system in it, which is one of the foundations of deep learning needed for calculating gradients.

Contrast this with Python, where automatic differentiation is not a core component of the language.

Some of the features developed initially as part of S4TF were later integrated into the Swift language itself.

⚡️Swift is FastWhen I first learned that Swift runs as fast as C code, I was astonished.

I knew that C was highly optimized and allowed to achieve very high speed, but that comes at the cost of micro-managing memory, which leads to C being not memory safe).

Besides, C is not a language that is very easy to learn.

Now, Swift runs as fast as C in numerical computation, and it doesn’t have memory safety issues, and it is much easier to learn.

LLVM compiler behind Swift is very powerful and has very efficient optimizations that will ensure your code will run very fast.

????.You Can Use Python, C and C++ Code in SwiftSince Swift for machine learning is at a very early stage of its life, this means there are not many machine learning libraries for Swift.

You shouldn’t worry about it too much, because Swift has amazing Python interoperability.

You simply import any Python library in Swift, and it just works.

Similarly, you can import C and C++ libraries into Swift (for C++, you need to make sure that header files are written in plain C, without C++ features).

To summarize, if you need specific functionality, but it is not implemented in Swift yet, you can import corresponding Python, C or C++ package.

Impressive!⚙️ Swift Can Go Very Low-LevelIf you ever used TensorFlow, most likely you did it through a Python package.

Underneath the hood, Python version of TensorFlow library has C code underneath.

So when you call any function in TensorFlow, at some level you hit some C code.

This means there is a limit of how low you can go trying to inspect the source code.

For example, if you want to see how convolutions are implemented, you won’t be able to see Python code for that, because that’s implemented in C.

In Swift, that is different.

Chris Lattner called Swift “ syntactic sugar for LLVM [assembly language]”.

This means that in essence, Swift sits very close to the hardware, and there are no other layers of code written in C in between.

This also means that the Swift code is very fast, as was described above.

It all leads to you as developer being able to inspect code from a very high level to a very low level, without the need to go into C.

????.What’s NextSwift is only one part of the innovation in deep learning happening at Google.

There is another component that is very closely related: MLIR, which stands for Multi-Level Intermediate Representation.

MLIR will be Google’s unifying compiler infrastructure, allowing to write code in Swift (or any other supported language) and compile it to any supported hardware.

Currently, there are a plethora of compilers for different target hardware, but MLIR will change that, allowing not only for code reusability but also for writing custom low-level components of the compiler.

It will also allow researchers to apply machine learning to optimize low-level algorithms:While MLIR acts as a compiler for ML, we also see it enabling the use of machine learning techniques within compilers as well!.This is particularly important as engineers developing numerical libraries do not scale at the same rate as the diversification of ML models or hardware.

Imagine being able to use deep learning to help optimize low-level memory tiling algorithms on data (a task similar to what Halide is trying to accomplish).

Moreover, this is only the beginning and other creative applications of machine learning in compilers away!SummaryIf you are into deep learning, then Swift is a language you should probably start learning.

It brings many advantages as compared to Python.

Google is investing heavily into making Swift a key component of its TensorFlow ML infrastructure, and it is very likely Swift will become the language of deep learning.

So, starting to get involved with Swift early will give you a first-mover advantage.

Originally published at https://pechyonkin.


.. More details

Leave a Reply