PRIMARY OBJECTIVES: I. To determine the safety, tolerability and recommended Phase II dose of temsirolimus in combination with vorinostat in patients with metastatic, hormone refractory, chemoresistant prostate cancer. II. To obtain preliminary evidence of response in prostate cancer patients treated with temsirolimus and vorinostat. SECONDARY OBJECTIVES: I. To determine the partial and complete objective response rates in metastatic hormone-refractory, chemo-resistant prostate cancer patients with measurable disease treated with temsirolimus and vorinostat. II. To determine the progression free survival and overall survival in patients with metastatic hormone refractory, chemo-resistant prostate cancer. III. To determine the PSA response, the duration of PSA response, time to PSA progression, PSA doubling time and PSA slope in metastatic hormone refractory, chemo-resistant prostate cancer patients treated with temsirolimus and vorinostat. IV. To assess changes in expression levels of bone remodeling markers (N telopeptides and bone alkaline phosphatase) and angiogenesis-related gene and protein expression (VEGF/HIF1-alpha) in blood and circulating tumor cells, and when available, in tissue, and correlate them with cancer and treatment related outcomes. V. To assess the changes in tumor metabolism with FDG/IIC-Choline PET/CT scan. OUTLINE: Patients receive oral vorinostat once daily on days 1-14 and temsirolimus IV on days 1, 8, and 15. Courses repeat every 21 days in the absence of disease progression or unacceptable toxicity. After completion of study treatment, patients are followed at 30 days and then every 3 months thereafter.