Engineering Initiation Award:  Time-Average Markov Decision Processes:  Theory Algorithms and Applications
{'Code': '05010700', 'Directorate': {'LongName': 'Direct For Computer & Info Scie & Enginr'}, 'Division': {'LongName': 'Division of Computing and Communication Foundations'}}
{'SignBlockName': 'Dwight D. Fisher'}
Engineering Initiation Abstract - Ross                                          This research will investigate Markov Decision Processes                   (MDPs), particularly as they relate to telecommunications.  MDPs                have been proposed for a variety of fields in which decisions                   must be made sequentially in the presence of uncertainty:                       communication networks (routing, flow control, multiplexing);                   operations management (inventory control, machine replacement,                  highway maintenance, hospital admission scheduling); and                        statistics (sequential hypothesis testing), among others.                       However, industrial implementations of MDPs are surprisingly few,               particularly in the telecommunication industry.                                      The principal investigator will study a new formulation for                time-average MDPs that could overcome some of the apparent                      reasons for the lack of their application.  It should lead to                   simpler and more elegant proofs for the classical results, and                  should also be a powerful technique for obtaining new results and               algorithms with a wide range of applicability.  Regarding                       telecommunication systems, the study will address the application               of time-average MDPs to a variety of queueing models, based on                  the researcher's previous work with optimal flow control and                    optimal multiplexing for integrated communication networks.  It                 is also anticipated that the new formulation will broaden                       substantially the range of applicabability of MDPs to                           telecommunications.  Another important aspect of the research is                the simultaneous study of concrete applications of MDPs along                   with the mathematical theory of MDPs, thereby increasing                        synergistically the quality and quantity of the results.
{'FirstName': 'Keith', 'LastName': 'Ross', 'EmailAddress': 'keithwross@nyu.edu', 'StartDate': '09/01/1987', 'EndDate': None, 'RoleCode': 'Principal Investigator'}
{'Name': 'University of Pennsylvania', 'CityName': 'Philadelphia', 'ZipCode': '191046205', 'PhoneNumber': '2158987293', 'StreetAddress': 'Research Services', 'CountryName': 'United States', 'StateName': 'Pennsylvania', 'StateCode': 'PA'}
{'Code': '4096', 'Text': 'COMMUNICATIONS RESEARCH'}
{'url': 'https://www.nsf.gov/awardsearch/download?DownloadFileName=1987&All=true', 'xml': '8707620.xml'}