Learning for Dynamic Bidding in Cognitive Radio Resources
  In this paper, we model the various wireless users in a cognitive radio
network as a collection of selfish, autonomous agents that strategically
interact in order to acquire the dynamically available spectrum opportunities.
Our main focus is on developing solutions for wireless users to successfully
compete with each other for the limited and time-varying spectrum
opportunities, given the experienced dynamics in the wireless network. We
categorize these dynamics into two types: one is the disturbance due to the
environment (e.g. wireless channel conditions, source traffic characteristics,
etc.) and the other is the impact caused by competing users. To analyze the
interactions among users given the environment disturbance, we propose a
general stochastic framework for modeling how the competition among users for
spectrum opportunities evolves over time. At each stage of the dynamic resource
allocation, a central spectrum moderator auctions the available resources and
the users strategically bid for the required resources. The joint bid actions
affect the resource allocation and hence, the rewards and future strategies of
all users. Based on the observed resource allocation and corresponding rewards
from previous allocations, we propose a best response learning algorithm that
can be deployed by wireless users to improve their bidding policy at each
stage. The simulation results show that by deploying the proposed best response
learning algorithm, the wireless users can significantly improve their own
performance in terms of both the packet loss rate and the incurred cost for the
used resources.
