title: No such file or directory error (python)
body: <p>In my code I've set the path for the txt file to the script path, but for some reason after the program writes some txt files for a few of the links, it throws this error "FileNotFoundError: [Errno 2] No such file or directory:" I don't really understand why for some of the links it works, but for others it can't seem to find the directory.</p>
  
  <pre><code>from lxml import html
  import requests, os.path
  spath = os.path.dirname(__file__)  ## finds path of script
  main_pg = requests.get("http://www.nytimes.com/") ## input site here
  with open(os.path.join(spath, "Main.txt"),"w", encoding='utf-8') as doc: 
      doc.write(main_pg.text)
  tree = html.fromstring(main_pg.content)
  hrefs = tree.xpath('//a[starts-with(@href, "http:") or starts-with(@href,"https:") or starts-with(@href,"ftp:")]/@href')  ## To avoid non-absolute hrefs
  for href in hrefs:
      link_pg = requests.get(href)
      tree2 = html.fromstring(link_pg.content)
      doc_title = tree2.xpath('//html/head/title/text()')  ## selects title of text from each link
      with open(os.path.join(spath, "%s.txt"%doc_title), "w", encoding ='utf-8') as href_doc:
          href_doc.write(link_pg.text)
  </code></pre>