Extending a basic web crawler to filter status codes and HTML - java
===

I followed a tutorial on writing a basic web crawler in Java and have got something with basic functionality.

At the moment it just retrieves the HTML from the site and prints it to the console.
I was hoping to extend it so it can filter out specifics like the HTML page title and the HTTP status code?

I found this library:
http://htmlparser.sourceforge.net/
... which I think might be able to do the job for me but could I do it without using an external library?

Here's what I have so far:

    public static void main(String[] args) {

		// String representing the URL
		String input = "";

		// Check if argument added at command line
		if (args.length >= 1) {
			input = args[0];
		}

		// If no argument at command line use default
		else {
			input = "http://www.my_site.com/";
			System.out.println("\nNo argument entered so default of " + input
					+ " used: \n");
		}
		// input test URL and read from file input stream
		try {

			testURL = new URL(input);
			BufferedReader reader = new BufferedReader(new InputStreamReader(
					testURL.openStream()));
			
			// String variable to hold the returned content
			String line = "";

			// print content to console until no new lines of content
			while ((line = reader.readLine()) != null) {
				System.out.println(line);
			}
		} catch (Exception e) {

			e.printStackTrace();
			System.out.println("Exception thrown");
		}
	}
