This phase I/II trial studies the side effects and best dose of romidepsin when given together with paclitaxel albumin-stabilized nanoparticle formulation and to see how well they work in treating patients with metastatic inflammatory breast cancer. Romidepsin may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth. Drugs used in chemotherapy, such as paclitaxel albumin-stabilized nanoparticle formulation, work in different ways to stop the growth of tumor cells, either by killing the cells or by stopping them from dividing. Giving romidepsin and paclitaxel albumin-stabilized nanoparticle formulation may be an effective treatment for inflammatory breast cancer.