This metaphor is saying that the person is talking to someone who is not paying attention and they are getting frustrated.
Consider the statement "Are you not listening to me?". Based on this "Am I talking to a brick wall?" a true statement? Yes, no, or I don't know?