{'fraction_non_alphanumeric': 0.02680067001675042, 'fraction_numerical': 0.007537688442211055, 'mean_word_length': 5.424731182795699, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
DEVELOPMENT OF AN AGENT-BASED SCHEDULE COLLECTING SYSTEM FOR TRANSPORTATION NETWORK PLANNING

In this research, an agent system that can identify and collect schedule information provided through the Internet was developed. The agent system is divided into the following: Schedule Crawler, which extracts URLs (Uniform Resource Locators) from HTML (Hyper-Text Markup Language) documents, finds HTML pages, and identifies a schedule-providing page. Web Robot, which senses change of data from the identified schedule-providing page, extracts schedule data, and stores the extracted data in the schedule database. In addition, algorithm and heuristics that can perceive only schedule information from the concerned Web page were developed. To compare the performance of the system, an experiment was carried out for four shipping companies, and HTML pages with 135 shipping schedule information. As a result, a 99.8% identification rate for the schedule information pages was demonstrated, and a 92.3% success rate for the extraction of schedule data was exhibited in the concerned pages. The extracted schedules can be used as schedule information to design an intermodal transportation network.