This is a 2-part prospective trial examining the ability of Stereotactic Body Radiation Therapy (SBRT) to induce a response to MK-3475, a humanized antibody to PD-1, in patients who progress on this antibody. Patients with metastatic melanoma will be enrolled after they have progressed on anti-PD-1 therapy. Patients with metastatic NSCLC (previously untreated with anti-PD-1 or anti-PD-L1 therapy) will be enrolled and treated with MK-3475 until they exhibit progression of disease. At this point (when patients have demonstrated progression of disease) a single target lesion will be selected and treated with SBRT, and then MK-3475 will be restarted and continued until there is further progression of disease. The first phase of the study is a radiation dose escalation with a constant dose of MK-3475. The second part of the study includes expansion cohorts of NSCLC and melanoma patients.