Not on your life is used to refuse very firmly to do something, which is what rafferty is doing in this context.
Consider the statement "If it was, would you accept? Absolutely not, rafferty.". Is the statement "If it was, would you accept? not on your life, rafferty." false? Yes/no/unknown