Take the 2-minute tour ×

Hello I want to Compare two webpages using python script. how can i achieve it? thanks in advance!

share|improve this question
what do you want to compare? Do you just want to know whether they are exactly the same? Or whether they look the same? –  Sören Mar 8 '11 at 16:50
add comment

2 Answers

up vote 3 down vote accepted

First, you want to retrieve both webpages. You can use wget, urlretrieve, etc.:
wget Vs urlretrieve of python

Second, you want to "compare" the pages. You can use a "diff" tool as Chinmay noted. You can also do a keyword analysis of the two pages:

  1. Parse all keywords from page. e.g. How do I extract keywords used in text?
  2. Optionally take the "stem" of the words with something like:
  3. Use some math to compare the two pages' keywords, e.g. term frequency–inverse document frequency: http://en.wikipedia.org/wiki/Tf%E2%80%93idf with some of the python tools out there like these: http://wiki.python.org/moin/InformationRetrieval
share|improve this answer
Thanks for the detailed comment !! this helped a lot !! –  Arackna Mar 8 '11 at 17:14
add comment

What do you mean by compare? If you just want to find the differences between two files, try difflib, which is part of the standard Python library.

share|improve this answer
add comment

Your Answer


