title: Ruby Net::HTTP::Post submit form
body: <p>I'm working on creating a website scraper. There is a form used to change the current page. </p>
  
  <p>This is the way that I am submitting the form for the POST request, but it seems to be fetching the same page over and over again.</p>
  
  <p>Here is some sample code:</p>
  
  <pre><code>pages = {
   "total_pages" =&gt; 19,
   "p1" =&gt; '1234/1456/78990/123324345/12143343214345/231432143/12432412/435435/',
   "p2" =&gt; '1432424/123421421/345/435/6/65/5/34/3/2/21/1243',
  ..
  ..
  ..    
  }
  
  
  idx = 1
  p_count = pages["total_pages"]
  
  #set up the HTTP request to change pages to get all the auction results
  uri = URI.parse("http://somerandomwebsite.com?listings")
  http = Net::HTTP.new(uri.host, uri.port)
  req = Net::HTTP::Post.new(uri.request_uri)
  
  p_count.times do
    puts "On loop sequence: #{idx}"
    pg_num = "p#{idx}"
    pg_content = pages["#{pg_num}"]
    req.set_form_data({"page" =&gt; "#{pg_num}", "#{pg_num}" =&gt; "#{pg_content}"})
  
    response = http.request(req)
    page = Nokogiri::HTML(response.body)
    idx = idx + 1
  end
  </code></pre>
  
  <p>It looks like <code>page</code> never changes. Is there a way to see what the full request looks like each time I am looking to make sure that the proper params are getting passed? It seems like it's virtually impossible to determine anything about <code>req</code>.</p>