Designing and Implementing Data Warehouse for Agricultural Big Data
  In recent years, precision agriculture that uses modern information and
communication technologies is becoming very popular. Raw and semi-processed
agricultural data are usually collected through various sources, such as:
Internet of Thing (IoT), sensors, satellites, weather stations, robots, farm
equipment, farmers and agribusinesses, etc. Besides, agricultural datasets are
very large, complex, unstructured, heterogeneous, non-standardized, and
inconsistent. Hence, the agricultural data mining is considered as Big Data
application in terms of volume, variety, velocity and veracity. It is a key
foundation to establishing a crop intelligence platform, which will enable
resource efficient agronomy decision making and recommendations. In this paper,
we designed and implemented a continental level agricultural data warehouse by
combining Hive, MongoDB and Cassandra. Our data warehouse capabilities: (1)
flexible schema; (2) data integration from real agricultural multi datasets;
(3) data science and business intelligent support; (4) high performance; (5)
high storage; (6) security; (7) governance and monitoring; (8) replication and
recovery; (9) consistency, availability and partition tolerant; (10)
distributed and cloud deployment. We also evaluate the performance of our data
warehouse.
