This phase I/II trial is studying the side effects and best dose of sorafenib when given together with cetuximab and irinotecan and to see how well they work in treating patients with advanced or metastatic colorectal cancer. Sorafenib may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth. Monoclonal antibodies, such as cetuximab, can block tumor growth in different ways. Some block the ability of tumor cells to grow and spread. Others find tumor cells and help kill them or carry tumor-killing substances to them. Sorafenib and cetuximab may also stop tumor growth by blocking blood flow to the tumor. Drugs used in chemotherapy, such as irinotecan, work in different ways to kill tumor cells, either by killing the cells or by stopping them from dividing. Giving sorafenib together with cetuximab and irinotecan may kill more tumor cells