This phase I/II trial studies the side effects and how well high-dose brachytherapy works in treating patients with prostate cancer that has not spread to other parts of the body. Brachytherapy is a type of radiation therapy in which radioactive material sealed in needles, seeds, wires, or catheters is placed directly into or near a tumor and may be a better treatment in patients with prostate cancer.