title: How can I read Wikipedia dump files similarly to how I can get information through the Mediawiki API?
body: <p>I've been trying to create a local Mediawiki instance of the english Wikipedia so that I can do lots of heavy and time consuming calls to the Mediawiki API (e.x. iterate over all pages and get their category and internal links)</p>
  
  <p>So far it hasn't worked out, there is always some problem after the SQL importing of the dumps is done or mid process, so I'm giving up on that for now but am looking into another solution.</p>
  
  <p>So say I want to go through all pages within a specific category and get each page internal links (what other wikipedia pages it links to), in <a href="https://github.com/mwclient/mwclient" rel="nofollow">mwclient</a> it's easy:</p>
  
  <pre><code>import mwclient
  site = mwclient.Site('en.wikipedia.org')
  
  def getPages(c, p):
      for page in c:
          if page.namespace == 0:
              p.append(page)
          elif page.namespace == 14:
              getPages(page, p)
          else:
              pass
  
  pages = []
  c1 = site.Pages["Category:Mathematics"]
  getPages(c1, pages)
  </code></pre>
  
  <p>But it requires me to have a running copy of Wikipedia, so I'm wondering if there is a similarly simple solution that can be executed against a dump (XML, SQL, DBPedia, or some other dump form), rather then communicating with a Mediawiki instance?</p>