To leave it at that means to take no further action, so in this context they are suggesting to stop and not continue.
Consider the staetement "Let's just stop here and not continue any further.". Based on this statement, is "Let's just leave it at that." a true statement? Yes/no/unknown