Neglecting is the opposite of swimming, so this contradiction suggests that the speaker does not miss swimming in the woman's eyes, but instead misses neglecting her.
Consider the statement "I miss neglecting in her eyes and watching her.". Is the statement "I miss swimming in her eyes and watching her." false? Yes/no/unknown