CV4Code: Sourcecode Understanding via Visual Code Representations
We present CV4Code, a compact and effective computer vision method for
sourcecode understanding. Our method leverages the contextual and the
structural information available from the code snippet by treating each snippet
as a two-dimensional image, which naturally encodes the context and retains the
underlying structural information through an explicit spatial representation.
To codify snippets as images, we propose an ASCII codepoint-based image
representation that facilitates fast generation of sourcecode images and
eliminates redundancy in the encoding that would arise from an RGB pixel
representation. Furthermore, as sourcecode is treated as images, neither
lexical analysis (tokenisation) nor syntax tree parsing is required, which
makes the proposed method agnostic to any particular programming language and
lightweight from the application pipeline point of view. CV4Code can even
featurise syntactically incorrect code which is not possible from methods that
depend on the Abstract Syntax Tree (AST). We demonstrate the effectiveness of
CV4Code by learning Convolutional and Transformer networks to predict the
functional task, i.e. the problem it solves, of the source code directly from
its two-dimensional representation, and using an embedding from its latent
space to derive a similarity score of two code snippets in a retrieval setup.
Experimental results show that our approach achieves state-of-the-art
performance in comparison to other methods with the same task and data
configurations. For the first time we show the benefits of treating sourcecode
understanding as a form of image processing task.