To not put too fine a point on it means to speak bluntly, but the speaker in this context is mincing words
Consider the statement "Er... well, to mince words, yes, I said.". Based on this "Er... well, not to put too fine a point on it, yes,I said." a true statement? Yes, no, or I don't know?