This research is being done to see how effective high-dose rate endorectal brachytherapy (HDRBT) is in treating cancer of the lowest part of the bowel (rectum). In this study we want to try to decrease side effects and shorten the course of radiation treatment for patients with cancer of their rectum by using a high-dose rate endorectal brachytherapy (HDRBT). This is a different form of radiation than what is normally given. With HDRBT, the radiation is given through an applicator placed into the bowel next to the tumor. The radiation is directed at the tumor and a small area around it.