

We (the IE team) get a lot of questions about these files.  The first question is often "What are they?" The answer is quite simple: Index.dat files are binary files that Internet Explorer uses to save information about URLs.  They are basically hash tables that have been optimized to be very fast when dealing with URLs.  Given an URL, these files map the URL to a binary blob of data.  The binary blob of data is different, depending on what the particular index.dat file is used for. 

Index.dat files are for IE's internal usage only.  They are not designed or tested to be used by others.  The functions for interacting with them are private, except things like the UrlCache APIs which provide a layer of abstraction for interacting with some index.dat files.

More on this topic to follow.