Increase formality of this text:
More concerned about what he thinks your not giving him emotionally then mad.
When you are more concerned with what he thinks, you are not giving yourself to him emotionally.