Distributed Parameter Map-Reduce
  This paper describes how to convert a machine learning problem into a series
of map-reduce tasks. We study logistic regression algorithm. In logistic
regression algorithm, it is assumed that samples are independent and each
sample is assigned a probability. Parameters are obtained by maxmizing the
product of all sample probabilities. Rapid expansion of training samples brings
challenges to machine learning method. Training samples are so many that they
can be only stored in distributed file system and driven by map-reduce style
programs. The main step of logistic regression is inference. According to
map-reduce spirit, each sample makes inference through a separate map
procedure. But the premise of inference is that the map procedure holds
parameters for all features in the sample. In this paper, we propose
Distributed Parameter Map-Reduce, in which not only samples, but also
parameters are distributed in nodes of distributed filesystem. Through a series
of map-reduce tasks, we assign each sample parameters for its features, make
inference for the sample and update paramters of the model. The above processes
are excuted looply until convergence. We test the proposed algorithm in actual
hadoop production environment. Experiments show that the acceleration of the
algorithm is in linear relationship with the number of cluster nodes.
