Parsing HTML with C++


Revisited! Please see the new article here.





Here’s some sample code demonstrating Tidy and libxml++.


#include <tidy/tidy.h>
#include <tidy/buffio.h>

    // Initialize a Tidy document
    TidyDoc tidyDoc = tidyCreate();
    TidyBuffer tidyOutputBuffer = {0};

    // Configure Tidy
    bool configSuccess = tidyOptSetBool(tidyDoc, TidyXmlOut, yes)
        && tidyOptSetBool(tidyDoc, TidyQuiet, yes)
        && tidyOptSetBool(tidyDoc, TidyNumEntities, yes)
        && tidyOptSetBool(tidyDoc, TidyShowWarnings, no);

    int tidyResponseCode = -1;

    // Parse input
    if (configSuccess)
        tidyResponseCode = tidyParseString(tidyDoc, html.c_str());

    // Process HTML
    if (tidyResponseCode >= 0)
        tidyResponseCode = tidyCleanAndRepair(tidyDoc);

    // Output the HTML to our buffer
    if (tidyResponseCode >= 0)
        tidyResponseCode = tidySaveBuffer(tidyDoc, &tidyOutputBuffer);

    // Any errors from Tidy?
    if (tidyResponseCode < 0)

    std::string tidyResult = (char*)tidyOutputBuffer.bp;

    return tidyResult;


Step 2: Parse the XML with libxml++:


xmlpp::DomParser doc;

// 'response' contains your HTML

xmlpp::Document* document = doc.get_document();


Important note about namespaces

This won’t work: *p/text()



More info


See the libxml++ class references:

Revisited! Please see the new article here.

Join the Conversation


    1. Hi Mark – The code is pretty much usable as is. To convert this into a complete example, all you need to do is throw Step 2 into a ‘main’ method (excluding the #include line, of course), with Step 1 code prepended to the top of the file.

  1. Hello, maybe I am too newbie of this topic, but, how is it handled the case when one has a formatting tag inside another?

    <a href="#" rel="nofollow">Text plus <b>bold text</b></a>

    How is it parsed into XML?

    1. Hi Alberto,

      The b tag is just a child of the a tag, so you can write XPath to select it.
      This should do it:

      descendant-or-self::[local-name() = ‘a’]/following-sibling::*[local-name() = ‘b’ and (position() = 1)]

      That will select all b tags that are direct descendants of a tags.


      1. Hi,

        my concern is that there is both bold and non-bold text. Then, there is some content plus a child. How is this compliant with XML? And how does one recognize separately the part of the content which is outside the inner tag ( in this case) and that inside it?

  2. Ah, I see. With XPath, to select the text itself, you need to use the text() function. This is because XPath treats text as an additional node. This is untested, but I think if you want to select the “Text plus ” text, you can use:

    descendant-or-self::[local-name() = ‘a’]/text()

    and if you want to select the “bold text” try:


    As for whether it’s actually valid XML, I’m not sure but I don’t think you’ll have any trouble parsing it. If you do have trouble, let me know.


  3. Not sure if you’re still replying to this, but do you have the HTML example?

    xmlpp requires a DTD, so I used

    Now it’s throwing errors:

    terminate called after throwing an instance of ‘xmlpp::validity_error’
    Validity error:
    Line 289, column 7 (error):
    Value “center” for attribute align of img is not among the enumerated set
    Line 296, column 20 (error):
    No declaration for attribute data-form-id of element a
    Line 569, column 3 (error):
    Element img does not carry attribute alt

    1. Looking at my project from when I first used this code, I never had to specify a DTD. Starting with an instance of the DomParser (xmlpp::DomParser doc;), I simply write doc.parse_memory(CleanHTML(response));. As long as you are first passing your HTML to Tidy to clean it up, I wouldn’t think xmlpp would throw any errors.

  4. when i compile this doesn’t recognize the file “tiny.h ” and “buffio.h” :
    and i have them in the same folder that i’m compiling .

  5. Hello and thanks for your tutorial ! I’m having an issue with TidyHTML though… I’m using it as a library, as you do, and I’m having trouble “tidying” those new HTML5 tags. , , and the like…
    How could I overcome this ? Should I add those tags, and if yes, how would I do that ? Be as precise as you can please, for I am not a smart man 😀

    Thanks again for all this !

    1. Just a follow-up: according to the API documentation, tidyOptSetValue(tidyDoc, TidyBlockTags, “header”) should do the trick… But it’s not. What am I missing ? Is that a whole lot more complicated and do I also have to create those in tidyenum.h and tags.c ?

      Thanks by advance for your help !

      1. Okay yeah, definitely had to check my shit before making these comments. Good thing there’s moderation heh? Anyway, I was dumb, all the “new tags” should be put in the same spot separated by commas. Sorry for the spam – maybe you could add this how-to-add-new-tags thing in the article itself ?

        Anyway thanks a lot again!

  6. Hi ! I have a question if you still look after this thread. I’m kind of a “noob” in this kind of programmation. So here’s my question… I have used libcurl to get an html file. It’s now in the root of the project I’m coding and I don’t understand how to use your code to just convert my (for example) “file.html” into a xhtml format that I can later parse. The fact is I don’t understand where or wich, in your program, file do you convert, and what kind of output you get. Do you get an url to put somewhere in the programm and get a straight string in return, or just an “.html” file that get converted ? That the part I’m stuck on. And where do I specify the file I want o be converted…
    Hope you really can help me, thanks !

    1. Hi,

      The first function I define, CleanHTML, takes the contents of the HTML file itself. So, you just need to open the file and read its contents into a string. Then pass that to CleanHTML, which will produce XML. From there, just use the code in “Step 2” to actually parse the XML into a tree that you can traverse.

      Hope this helps

        1. I have one more question, but only dor the sake of my curiosity. What would happened if you would do something like CleanHTML(response) in the main method without the rest of step 2 and left it as is. Would anything happen ? If yes, when would the buffer clean itself ?

          Thanks again for your time.

    2. I used that curl library recently… By “parsing” the html file you meanr remove all the html tags and get only the text and links of the html?

      1. By that I mean look algorythmicaly into a wab page to find the desired information and then “extract it”. My programm need to change in serial the name of a list of files from a alphanumerical code to an actual “name”. And I know a site on which I can find the desired information from just typing the proper url with the alphanumerical code. I than want to take in a string the proper name to than rename the file. Hope this reply to your question.

  7. Tidy – … has a built-in SAX-style parser. (!!WRONG)

    tidyBufAppend( out, in, r ); is not sax style parsing, it’s just adding bytes to buffer;
    in fact you can see then in main()
    err = tidyParseBuffer(tdoc, &docbuf);
    which will parse the whole preloaded buffer

    so please don’t write certain things and confuse people…

  8. Can you please upload the project and its dependent projects such as glibmm and the gtk includes for the all the msvc project to compile properly.
    This would probably save everyone a huge amount of time gathering the right version from the internet.



Leave a comment

Leave a Reply