TITLE: Notation for element-wise function application
QUESTION [3 upvotes]: Is there some kind of specific notation I could use to specify that function $f$ is applied to each element of matrix $W$ and not to a matrix as a whole. Specifically, I am writing about applying activation function for a layer of neural network. Or just writing this is clear enough: $f(W)$?

REPLY [0 votes]: I think there are no common notation to describe elementwise function application, but for some binary operators like $+, \cdot$ it is common to use $\oplus, \odot$ in neural networks. Some programming languages like Julia adopt dot operator f.(W), X .* Y to denote elementwise application. $f \circ W$ is not appropriate because
$$
\exists W, \exists i, j, \exp(\text{get}_{ij}(W)) = \exp (W_{ij}) \ne (\exp (W))_{ij} = \text{get}_{ij}(\exp(W)),
$$
this means $\exp \circ \text{get}_{ij} \ne \text{get}_{ij} \circ \exp$.
I personally like to use adding a symbol $\circ$ or others to the bottom or top of the function like $\overset{\circ}{f}(\{ a_i \}_{i \in I}) := \{ f(a_i) \}_{i \in I}, \overset{\circ}{f}(W) = ( f(W_{ij}) )_{ij}$ to mean an elementwise function.
{"set_name": "stack_exchange", "score": 3, "question_id": 2324764}