title: How to use PHP to Submit a Form and Retrieve a File from Another Site?
body: <p>I need to download a CSV file from <a href="http://www.carriersoftwaredata.com/Login.aspx?file=FHWA" rel="nofollow">http://www.carriersoftwaredata.com/Login.aspx?file=FHWA</a> to my local server on a daily basis. My plan is to implement a cron job running a php script to do this. However, the page requires a username and password be inputted before allowing the file to be downloaded. If I visit this page in my web browser, enter the username and password, and submit the form then my browser will show me a Download Dialog box and begin to download the file.</p>
  
  <p><strong>How can I use PHP to submit the form and download the provided file?</strong></p>
  
  <p>This is what I am doing currently to get the necessary $_POST information.</p>
  
  <pre><code>//url for fhwa db file
  $fhwa_url = 'http://www.carriersoftwaredata.com/Login.aspx?file=FHWA';
  
  //get the contents of the fhwa page
  $fhwa_login = file_get_contents($fhwa_url);
  
  //load contents of fhwa page into a DOMDocument object
  $fhwa_dom = new DOMDocument;
  if (!$fhwa_dom-&gt;loadhtml($fhwa_login))
  {
     echo 'Could not Load html for FHWA Login page.';
  }
  else
  {
     //create a post array to send back to the server - keys relate to the name of the input
     $fhwa_post_items = array(
        '__VIEWSTATE'=&gt;'',
        'Email'=&gt;'',
        'Password'=&gt;'',
        '__EVENTVALIDATION'=&gt;'',
     );
  
     //create an xpath object
     $xpath = new DOMXpath($fhwa_dom);
  
     //iterate through the form1 form and find all inputs
     foreach($xpath-&gt;query('//form[@name="form1"]//input') as $input)
     {
        //get name and value of input
        $input_name = $input-&gt;getAttribute('name');
        $input_value = $input-&gt;getAttribute('value');
  
        //check if input name matches a key in the post array
        if(array_key_exists($input_name, $fhwa_post_items))
        {
           //if the input name is Email or Password enter the defined email and password
           switch($input_name)
           {
              case 'Email':
                 $input_value = $email;
                 break;
              case 'Password':
                 $input_value = $pass;
                 break;
           }//switch
  
           //assign value to post array
           $fhwa_post[$input_name] = $input_value;
        }// if
     }// foreach
  }// if
  </code></pre>
  
  <p>This is how I am trying to submit the form - but it doesn't seem to be working in the way that I need. I'd like what is returned by stream_get_contents to be the contents of the CSV file I want to download.</p>
  
  <pre><code>   //get the url data and open a connection to the page
     $url_data = parse_url($fhwa_url);
     $post_str = http_build_query($fhwa_post);
  
     //create socket
     $fp = @fsockopen($url_data['host'], 80, $errno, $errstr, 30);
     fputs($fp, "POST $fhwa_url HTTP/1.0\r\n");
     fputs($fp, "Host: {$url_data['host']}\r\n");
     fputs($fp, "User-Agent: Mozilla/4.5 [en]\r\n");
     fputs($fp, "Content-Type: application/x-www-form-urlencoded\r\n");
     fputs($fp, "Content-Length: ".strlen($post_str)."\r\n");
     fputs($fp, "\r\n");
     fputs($fp, $post_str."\r\n\r\n");
     echo stream_get_contents($fp);
     fclose($fp);
  </code></pre>
  
  <p>Any help is definitely appreciated.</p>