This phase II trial is studying the side effects and how well RO4929097 works in treating patients with recurrent and/or metastatic epithelial ovarian cancer, fallopian tube cancer, or primary peritoneal cancer. RO4929097 may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth.