{'fraction_non_alphanumeric': 0.030894152693028396, 'fraction_numerical': 0.017489069331667707, 'mean_word_length': 4.951958821847298, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Importance of wind-induced undercatch adjustment in a gauge-based analysis of daily precipitation over Japan

We produced a new gauge-based analysis of daily precipitation over Japan from 1981 to 2000 on a 0.01° grid. Wind-induced undercatch was adjusted at each observation station, and orographic effects were considered in the process of spatial interpolation from station data to the gridded product. The resulting gridded gauge-based analysis of precipitation was validated against observed river discharge data. The validation demonstrated that the product is reasonable, mostly because of the undercatch adjustment. Based on the resulting product, Japan receives nearly 2000 mm/year of precipitation on average, which is approximately 10% larger than commonly thought.


INTRODUCTION
Gauge-based analyses of precipitation, also referred to as gridded precipitation products, have many research and practical applications, including those related to climate change impact assessment, weather forecast validation, and water resources prediction. Several global-and continental-scale gauge-based analyses of precipitation are available (e.g., Chen et al., 2002;Xie et al., 2007), mostly on a 0.5°latitude-longitude grid or similar spatial resolution. With the exception of the analysis by Xie et al. (2007), most are monthly products. In addition, global or continental data sets are usually constructed; these cannot resolve small areas like Japan well. Therefore, for studies and applications in Japan, a similar analysis product, but on a finer spatial resolution and with a shorter temporal scale (e.g., daily), is necessary.
We developed a new gauge-based analysis of daily precipitation over all of Japan on a 0.01°grid. The development strategy partially followed the methods of Xie et al. (2007), who used a data product called PRISM, developed by Daly et al. (1994Daly et al. ( , 2002, to represent orographic effects on precipitation estimation. Because PRISM data are not available for Japan and because full details of the development method of PRISM could not be extracted from the reports by Daly et al. (1994Daly et al. ( , 2002, we used a slightly different series of methods to incorporate orographic effects based on the ideas of Daly et al. (1994Daly et al. ( , 2002. Although we present a series of methods for the spatial interpolation and incorporation of orographic effects, the key advantages of our study are the attempt to adjust for wind-induced undercatch of precipitation using high-density meteorological observations and the validation of the product against observed river discharge data. Gridded precipitation products are not necessarily validated against river discharge. We describe the data in Section 2, the adjustment for wind-induced undercatch in Section 3, the spatial interpolation and incorporation of orographic effects in Section 4, and the validation in Section 5; the final section contains a summary.

DATA
The Automated Meteorological Data Acquisition System (AMeDAS), operated by the Japan Meteorological Agency (JMA), acquires and archives hourly surface meteorological observations at a relatively high spatial density throughout Japan. For precipitation, AMeDAS collects data at a spatial resolution of approximately 17 km. We used AMeDAS data for 1981 2000 to produce a new gauge-based analysis of precipitation. The number of available AMeDAS observation stations was limited before 1980; thus, we used data collected from 1981 to 2000.
River discharge data were necessary for the validation process described in Section 5. We obtained these data from the official database of the Ministry of Land Infrastructure and Transportation (MLIT), using the following selection criteria. First, discharge data should be available for > 3 years to allow for comparison between mean annual discharge and mean annual precipitation. Second, information on the catchment area should be available for each discharge gauging station. Further, because we calculated basin-average precipitation, evaporation, and runoff using a digital river network on a 0.1°grid over Japan (called the J-TRIP network; Yoshimura et al., 2008), each catchment should be well represented on J-TRIP in terms of its catchment area and catchment shape. Third, observed river discharge data reflecting considerable human effects should be excluded from the validation process. For this criterion, we surveyed maps of each river basin to determine whether large artificial facilities like large dams existed. In addition, we verified runoff heights with the unit of "mm" from upstream to downstream if several discharge gauging stations were available within the same river basin. We subjectively omitted from the validation process river basins that had large dams and highly variable runoff heights from upstream to downstream, because such dams and runoff variations suggest human disturbance of the river. In total, we selected 59 river discharge gauges.
Elevation data on a 0.01°grid were prepared from an official digital elevation model (DEM) for Japan created by the MLIT.

BIAS ADJUSTMENT FOR WIND-INDUCED UNDERCATCH
Gauge measurement of precipitation usually involves systematic underestimation, mainly because of the effects of wind passing above the rain gauge. We adjusted for the wind-induced undercatch of precipitation using the following equation, based on the equation and coefficient used by Yokoyama et al. (2003): P is the amount of precipitation after adjustment, Pobs is the actual observed amount of precipitation at each site, m is a coefficient for each rain-gauge type and each precipitation type (rainfall or snowfall), and U is the wind speed at the height of the rain gauge at the site. Yokoyama et al. (2003) determined m for three major types of rain gauge used by the JMA: RT-1, RT-3, and RT-4. Of these, RT-3 is widely used on the main island of Japan and RT-4 is widely used in Hokkaido for meteorological observatories of JMA. Because it was difficult to determine the type of rain gauge used at each of the 787 stations, we simply assumed that RT-3 was used on the main island and RT-4 was used in Hokkaido. Exact specification of rain gauge type for all the stations would be achieved in future if collaboration with JMA is successful. To apply m to the above equation, we had to determine whether the precipitation was rainfall or snowfall. We separated rainfall and snowfall using the following simple equation, following the methods of Sugaya (1991): where Rrain is the percentage of rainfall (between 0 and 100) and T is surface air temperature at each hour. The wind speed at the height of the rain gauge was also necessary to determine the adjustment. However, wind speed is often measured at a height that differs from that of the rain gauge. Because it is difficult to obtain and manage specific information for each station, we estimated the wind speed at the rain gauge height based on the measured wind speed. Assuming that wind speed W is measured at approximately 5 m above the ground and provided that several surface parameters such as the roughness length and displacement height are within the ordinary range of values, the estimated wind speed U at the rain gauge height (~1 m) would be approximately 80% of W. Thus, we assumed Of course, the coefficient in the above equation may be 0.7 or 0.9, but the effect of such a difference on the precipitation adjustment was generally < 10%. Therefore, we arbitrarily chose 0.8. Adjusted values (= differences between after-adjustment and before-adjustment) were then calculated for each station for each day over Japan (Figure 1). Regional variations of the adjusted values are easily found in this figure. For example, large adjustment was carried out in snow-dominated northern Japan and in southern coasts where precipitation amount is originally large.

General strategy
We next briefly explain the series of methods used to construct the final product. First, the daily climatology of precipitation after undercatch adjustment was calculated for each observation station for each day of a year. Here, "climatology" means the mean value averaged over multiple years. By spatially interpolating the daily climatology for stations, a gridded analyzed field of daily climatology was computed over Japan for every day of a year. Interpolation was performed using an inverse-distance weighting method and with the incorporation of orographic effects. A gridded analyzed field of the ratio of precipitation of a day to the daily climatology of that day was then computed over Japan for each day by interpolating the ratios of surrounding stations. Finally, daily gridded analysis fields of precipitation were calculated by multiplying the climatology and the ratio. This daily precipitation is the final product and is called "Product 3." The series of methods is referred to as "Method 3"; the numbering of the method and product are described below.
When calculating the climatology, we excluded AMeDAS data from 1992 to 1994, when considerably fewer observation stations were operating compared to the rest of the period. Consequently, data from 787 stations for 17 years (1981 2000, except 1992, 1993, and 1994) were used to create the climatology fields. Because the time series of daily precipitation at each station was highly variable throughout the year, even after 17-year averages were calculated, we applied a moving average with a 61-day window, which is simpler than the Fourier truncation performed by Xie et al. (2007).
The final product was "Product 3." Products 1 and 2, from Methods 1 and 2, were also created. For Product 1, daily precipitation at each station before undercatch adjustment was interpolated onto all of the grid points. A gridded daily precipitation product was then constructed. This gridded daily precipitation product and the calculation method without undercatch adjustment were called "Product 1" and "Method 1," respectively. Similarly, the gridded daily precipitation after adjustment, but without incorporating orographic effects, was called "Product 2." Thus, "Product 3" corresponds to the gridded precipitation product after adjustment and with incorporation of orographic effects. Because orographic effects do not clearly appear in daily time-scale but do appear in climatological time-scale, two steps are carried out: calculating climatology and calculating daily value.

Spatial interpolation of climatology with the incorporation of orographic effects
Higher elevations tend to receive larger amounts of precipitation. As part of the process of interpolation to all grids, we incorporated elevation effects using linear regression following the methods of Daly et al. (1994Daly et al. ( , 2002: where X and Y are the elevation and the amount of precipitation, respectively, of the corresponding grid. The coefficients of the linear regression equation were derived from least-squares fitting to observed precipitation and elevation at surrounding stations of the corresponding grid. The observation stations had unequal weights in the least-square calculations. An observation station having similarity to the corresponding grid was highly weighted; for example, a station that was located near the corresponding grid, or a station that was located in the same "facet" of the corresponding grid, was highly weighted. Specifically, the weight (= W0i ) was given by where W(d), W(z), W(f), and W(e) are distance weight, elevation weight, facet weight, and effective terrain height weight, respectively. This equation is almost the same as that used by Daly et al. (1994Daly et al. ( , 2002, and detailed descriptions can be found in their reports. Terrain height weight and facet weight are also explained in detail by Daly et al. (2002). Briefly, the terrain height weight reflects whether a station is located on relatively flat terrain or in narrow, steep terrain. The facet is explained later. Daly et al. (2002) appeared to use additional weight parameters such as cluster weight, vertical-layer weight, and coastal-proximity weight. However, these weight parameters were difficult to incorporate based only on the information provided in Daly et al.'s (1994Daly et al.'s ( , 2002 reports and available datasets. Therefore, we ignored these weight parameters. It corresponds to those weight parameters = 1. A topographic facet is a contiguous terrain slope that has a common orientation (Daly et al., 1994). Figure  2 shows the topography of Japan and the delineated facets. The facet weight can represent the difference in precipitation between the leeward and windward sides of a mountain chain. Following the methods of Gibson et al. (1997), we delineated facets from the elevation data on a 0.01°grid. Eight directions were assigned for each delineated facet (e.g., north, northwest, west, southwest, etc.).
After all of the weights were calculated and the linear regression coefficients were determined at each grid, the Y (= precipitation) at each grid could be computed depending on the elevation of the grid. Thus, the gridded daily precipitation climatology considering the orographic effects was constructed from the station climatology for each day of a year.

Spatial interpolation of the ratio
The ratio of daily precipitation to climatology was calculated at each observation station for each day in the target period. The ratio was then interpolated onto the grid points using the inverse-distance weighting interpolation method.

Final product
The final product, a new gridded gauge-based analysis of daily precipitation over Japan (Product 3) was computed by multiplying the climatology by the ratio for all of the grid points for each day of the target period. As an example of the results, Figure 3 shows the distribution of mean annual precipitation according to this product.

Cross-validation of precipitation
Cross-validation tests were carried out to examine the validity of the methods. In general, cross-validation testing involves withdrawing one station from a list of stations and then estimating the amount of precipitation at that station by interpolating the amounts of precipitation at the surrounding stations. The estimated value is then compared with the actual value at the excluded station. For our test, we randomly selected 10% of the stations and did not use precipitation data from those stations for the interpolation; the remaining 90% of the data were used to estimate the precipitation at the excluded stations. We repeated this procedure 10 times, each time changing the stations that were excluded.
Relative root mean square errors (REs) of the crossvalidation tests, based on Chen et al. (2002), were calculated for Methods 2 and 3. The calculated REs were around 130% which is acceptable if compared with the value of REs in Chen et al. (2002) which is around 60%. Here, please note that REs for monthly data like Chen et al. (2002) should be smaller than REs of daily data like our data. It suggests that our methods and products are not out of range; there is no huge mistake. Of course, it was impossible to directly compare the REs with corresponding values from Chen et al. (2002) because of the many differences, e.g., in the methodology, target area, resolution, and density of rain gauges. Data after adjustment were used for this test, so there were no results for Method 1; however, the RE for Method 1 should be quite similar to that for Method 2 because the only difference was the adjustment.

Validation with river discharge
Products 1, 2, and 3 were also validated against observed river discharge data on an annual mean basis. Because precipitation cannot be directly compared with river discharge, a simple water budget equation, P E = R, was used, where P is precipitation, E is evaporation, and R is the runoff (discharge) of a catchment. We defined discharge as the amount of flow observed at a certain point of a river and runoff as the area-mean runoff of a catchment (i.e., discharge divided by area). Evaporation was estimated using a method proposed by Komatsu et al. (2008). Komatsu et al. (2008) proposed an empirical equation with which to estimate evaporation based only on the surface air temperature. The equation was developed from their investigations of water budgets at many experimental forest watersheds in Japan. When we took evaporation, which was calculated by a land surface model in Yoshimura et al. (2008), we obtained a quite similar conclusion introduced in the following.
The annual mean runoff for each catchment was estimated by subtracting evaporation from Products 1, 2, and 3, using R calculated as P E. We then compared estimated and observed runoff for each catchment ( Figure  4); it should be noted that observed runoff also contains uncertainty. In general, Products 2 and 3 were better than Product 1. Product 3 was only slightly better than Product 2. This indicates that the impact of undercatch adjustment is more effective than the impact of interpolation using orographic effects. This argument was derived only from Products 1, 2, and 3 and Methods 1, 2, and 3. Another production method may produce another product. However, the good correspondence between estimated and observed runoff (Figures 4b and  c) implies that this argument may be generally applied to any gauge-based analysis of precipitation over Japan. Orographic effects and the sparse observation network at higher elevations have commonly been considered the main causes of the insufficient area-mean observed amounts of precipitation found in many basins in Japan. Surprisingly, undercatch adjustment seems to be more influential. Additional careful investigation is needed to determine whether this argument can be applied to other regions of the world that have different observation station densities, climate, and topographic characteristics from those of Japan.
Although Products 2 and 3 improved the overall underestimation of precipitation, considerable scatter remained. We tried to identify the regionality of the scatter as a clue to its source, but no systematic characteristic was found.
For reference, we also used Radar-AMeDAS (R-A) data published by the Japan Meteorological Agency ( Figure 5), which are the only precipitation data that have temporal and spatial coverage similar to those of Products 1, 2, and 3. The scatter was larger for R-A ( Figure 5) than for our products (Figure 4). In general, radar observations can have more uncertainties than ground-based observations; thus, the greater scatter in R-A data is not surprising. Thus, the source of scatter is not easy to explain, even using R-A data. Still, the reduction of the scatter in our products is an important topic for future research and will likely require careful regional-scale and river basin-scale investigation.

SUMMARY
We constructed a gauge-based analysis of daily precipitation, a gridded product, that covers all of Japan for the period of 1981 2000. Adjustment for wind-induced undercatch and spatial interpolation considering orography were included in the construction. Validation against observed runoff indicates good confidence in the product. Unexpectedly, wind-induced undercatch had a more significant effect than did orographic effects.
The product has a spatial resolution of 0.01°. However, the spatial resolution of precipitation observations was approximately 17 km (nearly 0.2°), with 0.01°resolution playing a role only in areas of steep topography. Thus, the "effective" resolution may be~0.2°.
The official or commonly known value of average precipitation over Japan (e.g., as reported in the Japanese government white paper on water resources) is 1700 1800 mm/year, whereas our product shows an average of > 1900 (nearly 2000, specifically 1956 by Product 2 and 2009 by Product 3) mm/year. Thus, our products have nearly 10% difference from the 50 Figure 3. Distribution of mean annual precipitation (mm/year) over Japan.
commonly thought value. Our product should be useful for nationwide water resources assessment, climate change studies, and hydrological prediction. Finally, it should be noted that no data sets are without some weakness, and further improvement through additional validations and applications will make the product better. More sophisticated interpolation schemes and/or the addition of more observed precipitation data (e.g., those observed by MLIT for reservoir management) may increase accuracy of the product.   Figure 5. Scatter diagram for the validation of analyzed precipitation against observed runoff at each catchment for Radar AMeDAS precipitation data. Analyzed precipitation was converted into runoff by subtracting evaporation. Horizontal axis is observed runoff, and vertical axis is estimated runoff.