title: How to fill and submit form using Python 3 and only Python 3 libraries
body: <p>I am working on this as a free time project.</p>
  
  <p>I would like to use Python 3 to login to a site (fill form and submit) and then I plan to scrape the page for some data. The reason I am strictly looking for a solution in Python 3 is because I am trying to learn more about Python and figured I would go straight to Python 3. I have seen several great looking tools such as mechanize but they only seem to support Python 2.</p>
  
  <p>Planning to use this for a financial investment site but let's just use starbucks as an example.</p>
  
  <pre><code>def loginToStockSite(username, pwd):
      url = "https://www.starbucks.com/account/signin"
      values = {"Account.UserName" : username, 
                "Account.PassWord" : pwd}
      data = urllib.parse.urlencode(values)
      data = data.encode('utf-8')
      req = urllib.request.Request(url, data)
      sock = urllib.request.urlopen(req)
      htmlSource = sock.read().decode('utf-8')
      return htmlSource
  </code></pre>
  
  <p>I have gotten very confused from the various encode, decode, url opener, etc. examples. I have not found a solution that has worked for me yet.</p>
  
  <p>Thanks in advance for any help with my "for fun" project.</p>
  
  <hr>
  
  <p>Update</p>
  
  <p>Here is the html I get back (with a bunch cut out to fit the character limit for a question)</p>
  
  <pre><code>&lt;!DOCTYPE html&gt;
      &lt;head&gt;
          &lt;meta charset="utf-8" /&gt;
  
          &lt;title&gt;Account Sign In | Starbucks Coffee Company&lt;/title&gt;
          &lt;meta name="viewport" content="width=device-width, initial-scale=1" /&gt;
      &lt;/head&gt;
      &lt;div class="fields"&gt;
          &lt;div class="region size1of1"&gt;
              &lt;div class="validation_summary warning validation_medium"&gt;&lt;h2&gt;Please Enable Cookies to Continue&lt;/h2&gt;&lt;p&gt;To sign in to the Starbucks web site, please enable cookies in your web browser.&lt;/p&gt;&lt;/div&gt;
  
          &lt;div class="fields"&gt;
              &lt;form action="/account/signin" class="siren region size1of2" id="accountForm" method="post"&gt;
  
                  &lt;fieldset class="login_position"&gt;
                      &lt;legend class="hidden_visually"&gt;I have a Starbucks account.&lt;/legend&gt;
  
                      &lt;h2 class="existing_acc_h3"&gt;I have a Starbucks account.&lt;/h2&gt;
  
                                          &lt;div id="user_info" class="region size1of2 reset"&gt;
                                              Hi,
                                              &lt;span id="info_user"&gt;&lt;/span&gt;
                                          &lt;/div&gt;
  
                                          &lt;div class="size1of2"&gt;
                                              &lt;a id="not_me" href="#notme"&gt;Not You?&lt;/a&gt;
                                          &lt;/div&gt;
                                      &lt;/div&gt;
                                  &lt;/div&gt;
                              &lt;/div&gt;
  
                              &lt;div id="connect_cont" &gt;
                                  &lt;div id="text_cont" class="hidden"&gt;
                                      &lt;h3&gt;Have a Starbucks account?&lt;/h3&gt;
                                      &lt;p&gt;Connect your Facebook account to your Starbucks account by logging in here.&lt;/p&gt;
                                  &lt;/div&gt;
                              &lt;/div&gt;
                          &lt;/li&gt;
                          &lt;li&gt;
                              &lt;label for="Account_UserName"&gt;Username &lt;span class="required"&gt;*&lt;/span&gt;&lt;/label&gt;
                              &lt;label for="Account_UserName" class="hidden"&gt;Starbucks Username &lt;span class="required"&gt;*&lt;/span&gt;&lt;/label&gt;
                              &lt;input class="field_xlarge" id="Account_UserName" maxlength="200" name="Account.UserName" type="text" value="MY_USERNAME_WAS_HERE" /&gt;
                          &lt;/li&gt;
                          &lt;li&gt;
                              &lt;label for="Account_PassWord"&gt;Password &lt;span class="required"&gt;*&lt;/span&gt;&lt;/label&gt;
                              &lt;label for="Account_PassWord" class="hidden"&gt;Starbucks Password &lt;span class="required"&gt;*&lt;/span&gt;&lt;/label&gt;
                              &lt;input class="field_xlarge password" id="Account_PassWord" maxlength="200" name="Account.PassWord" type="password" value="MY_PASSWORD_WAS_HERE" /&gt;
                          &lt;/li&gt;
                          &lt;li&gt;
                              Forgot your &lt;a href="/account/forgot-username?AllowGuest=False"&gt;username&lt;/a&gt; or &lt;a href="/account/forgot-password?AllowGuest=False"&gt;password&lt;/a&gt;?
                              &lt;p class="privacy_policy hidden"&gt;
                                  &lt;a href="/about-us/company-information/online-policies/privacy-policy"&gt;Concerned about privacy?&lt;/a&gt;
                              &lt;/p&gt;
                          &lt;/li&gt;
                          &lt;li class="inline push"&gt;
                              &lt;input type="checkbox" id="Account.IsRememberMe" name="Account.IsRememberMe" value="True" class="checkbox" /&gt;
                              &lt;label for="Account.IsRememberMe"&gt;Keep me signed in.&lt;/label&gt;
                          &lt;/li&gt;
                      &lt;/ol&gt;
                  &lt;/fieldset&gt;
  
                  &lt;fieldset&gt;
                      &lt;input id="ReturnUrl" name="ReturnUrl" type="hidden" value="" /&gt;
                      &lt;input id="AllowGuest" name="AllowGuest" type="hidden" value="False" /&gt;
                      &lt;input id="isJavaScriptDisabled" name="isJavaScriptDisabled" type="hidden" value="True" /&gt;
                      &lt;span class="button"&gt;&lt;button type="submit"&gt;Sign In&lt;/button&gt;&lt;/span&gt;
                  &lt;/fieldset&gt;
  
                  &lt;fieldset class="submit"&gt;
                      &lt;div id="fb_container"&gt;
                          &lt;div id="fb_btn_cont"&gt;
                              Or log in using Facebook.
                              &lt;p&gt;&lt;a class="fb_button fb_button_medium" id="connect" href="#connect"&gt;&lt;span class="fb_button_text"&gt;Login with Facebook&lt;/span&gt;&lt;/a&gt;&lt;/p&gt;
                          &lt;/div&gt;
                      &lt;/div&gt;
                  &lt;/fieldset&gt;
              &lt;/form&gt;
  
              &lt;div class="region size1of2 block_login"&gt;
  
  
                      &lt;h2&gt;I need a Starbucks account.&lt;/h2&gt;
  
                          &lt;p&gt;&lt;span class="button"&gt;&lt;a href="/account/create"&gt;Create An Account&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;
  
  
                      &lt;p&gt;With a Starbucks account you can register and manage your Starbucks Cards and participate in &lt;a href="/card/rewards"&gt;My Starbucks Rewards&lt;/a&gt;.&lt;/p&gt;
  
                      &lt;ul class ="basic"&gt;
                          &lt;li&gt;Enjoy a free drink on your birthday&lt;/li&gt;
                          &lt;li&gt;Protect your balance if your Starbucks Card is missing or stolen.&lt;/li&gt;
                          &lt;li&gt;Transfer money between cards.&lt;/li&gt;
                          &lt;li&gt;Track your earnings in My Starbucks Rewards&lt;/li&gt;
                          &lt;li&gt;Reload your Card balance automatically&lt;/li&gt;
                      &lt;/ul&gt;
  
              &lt;/div&gt;
  
  
          &lt;/div&gt;
  
  
  
  
  
                  &lt;div class="fields"&gt;
      &lt;div class="region size1of1"&gt;
          &lt;ul id="breadcrumb"&gt;
              &lt;li&gt;&lt;a href="/card"&gt;Card&lt;/a&gt; .&lt;ul&gt;
                  &lt;li&gt;&lt;a href="/card/rewards"&gt;My Starbucks Rewards&lt;/a&gt; .&lt;ul&gt;
                      &lt;li&gt;View Your Stars&lt;/li&gt;
                  &lt;/ul&gt;&lt;/li&gt;
              &lt;/ul&gt;&lt;/li&gt;
          &lt;/ul&gt;
      &lt;/div&gt;
  &lt;/div&gt;
          &lt;/div&gt;
  
  
  
      &lt;div id="footer"&gt;
          &lt;div class="container"&gt;
  
      &lt;form id="search" method="get" action="/search"&gt;
          &lt;fieldset&gt;
              &lt;input id="searchbox" name="keywords" title="Search Keyword" maxlength="100" class="search_input" /&gt;
              &lt;span class="button button_search"&gt;&lt;button id="submit_search_util" type="submit"&gt;Search&lt;/button&gt;&lt;/span&gt;
          &lt;/fieldset&gt;
      &lt;/form&gt;
  
  
              &lt;div class="fields"&gt;
                  &lt;div class="region size5of6 suffix1of6"&gt;
                      &lt;div class="footer_categorical"&gt;&lt;ol class="blocks blocks-five-up"&gt;
  &lt;li&gt;&lt;h4&gt;
      &lt;a href="/shop/card"&gt;Buy a Card&lt;/a&gt;
  &lt;/h4&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;h4&gt;
      &lt;a href="/card"&gt;Manage Your Card&lt;/a&gt;
  &lt;/h4&gt;
  &lt;ol&gt;
  
      &lt;li&gt;&lt;a href="/Card#cardBalanceWrapper"&gt;Check Balance&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href="/card/reload/one-time"&gt;Reload Your Card&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href="/card/manage/transfer"&gt;Transfer Funds&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href="/card/manage/history"&gt;View Transactions&lt;/a&gt;&lt;/li&gt;
  
  &lt;/ol&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;h4&gt;
      &lt;a href="/card/rewards"&gt;My Starbucks Rewards&lt;/a&gt;
  &lt;/h4&gt;
  &lt;ol&gt;
  
      &lt;li&gt;&lt;a href="/account/create/register"&gt;Register Your Card&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href="/account"&gt;View Your Stars&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href="/card/rewards/gold"&gt;Keep Your Gold Benefits&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href="/card/rewards/rewards-program-ts-and-cs"&gt;Rewards Program Terms and Conditions&lt;/a&gt;&lt;/li&gt;
  
  &lt;/ol&gt;
  &lt;/li&gt;
  &lt;li&gt;&lt;h4&gt;
      Learn More
  &lt;/h4&gt;
  &lt;ol&gt;
  
      &lt;li&gt;&lt;a href="/card/card-terms-and-conditions"&gt;Card Terms and Conditions&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href="/card/egift"&gt;What is a Starbucks Card eGift?&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href="/customer-service/faqs/card"&gt;Card FAQs&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href="/account"&gt;Manage Your Account&lt;/a&gt;&lt;/li&gt;
      &lt;li&gt;&lt;a href="http://mystarbucksidea.force.com/ideaList?ext=0&amp;amp;lsi=0&amp;amp;category=Starbucks+Card"&gt;My Starbucks Idea&lt;/a&gt;&lt;/li&gt;
  
  &lt;/ol&gt;
  &lt;/li&gt;
  </code></pre>
  
  <p>I expected to get back a page like I get when I login in using a browser and have a div with the current balance on whatever the default gift card is that is registered.</p>
  
  <pre><code> &lt;div class="balance-amount numbers"&gt;
  </code></pre>
  
  <p>That is the div that is visible. I found the balance in another place in the page source also but I do not see it on the page.</p>
  
  <pre><code>&lt;p class="card_balance numbers"&gt;
  
                                  &lt;span&gt;$27.68&lt;/span&gt; &lt;span class="datestamp"&gt;3/10/2013 2:12 PM&lt;/span&gt;
  
                          &lt;/p&gt;
  </code></pre>
  
  <p>Anyways, I would like to input the login info and submit the form using Python 3 or post the data somehow to login and then (as a use case but outside what is needed to answer the question) I would pull the account balance from the html.</p>
  
  <p>I did find that my username and password are apparently in their corresponding fields and in the validation summary it says</p>
  
  <pre><code>&lt;div class="validation_summary warning validation_medium"&gt;&lt;h2&gt;Please Enable Cookies to Continue&lt;/h2&gt;&lt;p&gt;To sign in to the Starbucks web site, please enable cookies in your web browser.&lt;/p&gt;&lt;/div&gt;
  </code></pre>
  
  <p>I did see some examples that dealt with cookies. Is that the problem? I will research that possible solution. In the meantime, hope this helps you help me. Thanks.</p>