Checking if website responds in python using a browser user agent
===

I am trying to come up with a script to check if a website responds and resolves dns using python. I want to be able to do this in a few sequential loops, however after trying to run a loop once, the second time i run the script, the websites that previously returned a successful response, now do not.

Below is my script:


    #! C:\Python27
    import socket,time
    
    localtime = time.asctime( time.localtime(time.time()) )
    
    
    def hostres(hostname):
        print "Attempting to resolve " + hostname 
        try:
            socket.gethostbyname(hostname)
            print "Resolved Successfully!"
        except socket.error:
            print "Could Not Resolve"
    
    print "*************************************************"
    print "Website loop starting.."
    print "Local current time :", localtime
    print "*************************************************"
    print ""
    
    text_file = open("sites.txt", "r")
    lines = text_file.readlines()
    for line in lines:
        hostres(line)
    text_file.close()

The contents of the text file are:

    www.google.com
    en.wikipedia.org
    www.youtube.com
    us.gamespot.com

I am thinking it is to do with these websites recognizing the script as a "bot" rather than a legitimate end-user, would it be correct to assume this? If so, how can I still check if the dns name resolves by looking up the name of the website (or IP, does not matter) and be able to run this without getting a false reading of "request failed" despite the fact that the service is fully accessible from a browser?

Thank you for your help!