{'fraction_non_alphanumeric': 0.03374354902739182, 'fraction_numerical': 0.02515879317189361, 'mean_word_length': 4.608961870303368, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Stalled Improvement in a Numerical Weather Prediction Model as Horizontal Resolution Increases to the Sub-Kilometer Scale

This study evaluated the performance of a regional weather prediction model. The horizontal resolution is increased to the sub-kilometer scale in a series of experiments over areas of Japan through the summer or winter seasons of 2015−2016. The performance improves less when increasing the horizontal resolution from 2 km to 1 km or 500 m than it does from 5 km to 2 km, especially when topography and ice microphysics are less relevant. Although the velocity and magnitude of updrafts, cloud size, and convection in the boundary layer indeed change with the horizontal resolution, these differences turn out to have little impact on the model performance. (Citation: Ito, J., S. Hayashi, A. Hashimoto, H. Ohtake, F. Uno, H. Yoshimura, T. Kato, and Y. Yamada, 2017: Stalled improvement in a numerical weather prediction model as horizontal resolution increases to the sub-kilometer scale. SOLA, 13, 151− 156, doi:10.2151/sola.2017-028.)


Introduction
Recent advances in computing enable numerical weather predictions (NWPs) to be conducted with higher and higher resolution. For example, the Japan Meteorological Agency (JMA) has launched the "Local Forecast Model" (LFM) with a horizontal grid spacing of 2 km for 9-hour forecasts around areas of Japan (Hirahara et al. 2011). Such resolution is so fine that the parameterizations of cumulus convection that have been used for NWPs are thought to be no longer required. These NWPs, which are referred to as convection-allowing models, explicitly resolve cumulus convection on numerical grids, and are believed to be more faithful than those with the cumulus parameterizations. Certain improvements in some aspects have been observed when the horizontal grid spacing is increased from about ten to several kilometers (e.g., Done et al. 2004;Kain et al. 2006;Weisman et al. 2008;Duc et al. 2013;Yashiro et al. 2016). NWPs with even finer horizontal grid spacing (~1 km) are now often used for various research purposes including regional climatology (e.g., Uno et al. 2014). Forecasts of fog in real cases have been shown to benefit from the finer horizontal grid spacing (333 m) (Boutle et al. 2016).
It might be expected that any increase in the horizontal resolution of an NWP employing the finite discretization should result in better performance. With finer resolution, however, a new issue may arise in conventional parameterization schemes. Wyngaard (2004) has pointed out that moderately fine resolution may have difficulty dealing with planetary boundary layer (PBL) parameterizations: conventional PBL schemes do not assume that cases exist in which some turbulence eddies are resolved while many other smaller scale eddies have to be parameterized. Such a resolution range is referred to as a "gray zone" or "Terra Incognita" for which a suitable parameterization is still under discussion for PBL convection (e.g., Honnert et al. 2011;Zhou et al. 2014;Ito et al. 2015;Shin and Hong 2015;Efstathiou et al. 2016;Kitamura 2016). Another study suggested a horizontal grid spacing of ~100 m is required to faithfully reproduce cumulus clouds (Bryan et al. 2003).
A higher resolution NWP must consume more computer resources, so one should carefully think about its costs and benefits. A few studies (e.g., Barthlott and Hoose 2015;Schwartz et al. 2017) have evaluated the performance from the perspective of application to daily weather forecasting. Envisioning such a future operational use of an NWP over areas of Japan, the present study statistically evaluated the performance of a regional NWP, the JMA's Non-Hydrostatic Model (JMA-NHM; Saito et al. 2006), over areas of Japan when the horizontal resolution was increased. No special treatment was made for the "gray zone" issue. This paper reports the consequences of such a "naïve increase" in the horizontal resolution.
The sensitivity to the horizontal resolution was investigated through two series of experiments in contrasting seasons (summer and winter). The prediction of local heavy rainfall in the summer in the Kanto region (Tokyo metropolitan area), which has the smoothest topographies in Japan, is known to be difficult (e.g., Kawabata et al. 2007). Whether an NWP with the finer horizontal resolution realistically reproduces rainfalls over the region is particularly of interest. This paper is organized as follows. The next section describes the experimental configurations, target areas and periods, and the observational datasets used for validation. Section 3 evaluates the performance compared with observations. Section 4 discusses the results based on vertical profiles of the NWP. Section 5 discusses the dependence on the horizontal resolution, and Section 6 presents the conclusions.

Experimental method and targets
The regional NWP model JMA-NHM has been used for operational forecasts by the JMA and other researchers for mesoscale weather events and local climatology. Those experiments, referred to as Exp-Summer and Exp-Winter, were conducted in the summer of 2015 and winter of 2016, respectively. For the experiments in Exp-Summer, 18 hours of time integrations were performed starting from 0300 Japan Standard Time (JST: +9 hours UTC) every day between 1 July and 31 August 2015. For those in Exp-Winter, 12 hours of time integrations were performed starting from 0900 JST every day between 12 January and 18 February 2016. Their computational domains covered regions of 1100 km by 900 km as shown in Fig. 1. Two-(three-) dimensional outputs of the NWP were stored every 2.5 (5) minutes only for the target areas ( Fig. 1; Kanto and Kansai in Exp-Summer and Hokuriku in Exp-Winter) due to limited data storage capacity.
The horizontal grid spacing Δx was varied among 5 km, 2 km, 1 km, and 500 m by using horizontal grid size of 220 by 180, 550 by 450, 1100 by 900, and 2200 by 1800, respectively. Even the coarsest experiments (Δx = 5 km) did not employ the cumulus parameterization (e.g., Kain and Fritsch 1993), although it is used for operational forecasts by JMA-NHM. The cloud microphysics considered five kinds of hydrometeors (cloud, ice cloud, rain, snow, and graupel). The Mellor-Yamada-Nakanishi-Niino level 2.5 scheme was used to parameterize the PBL process (Nakanishi

Stalled Improvement in a Numerical Weather Prediction Model as Horizontal Resolution Increases to the Sub-Kilometer Scale
Junshi Ito 1 , Syugo Hayashi 1 , Akihiro Hashimoto 1 , Hideaki Ohtake 1, 2 , Fumichika Uno 1, 2 , Hiromasa Yoshimura 1 , Teruyuki Kato 1, * , and Yoshinori Yamada 1 1 Meteorological Research Institute, Tsukuba, Japan 2 National Institute of Advanced Industrial Science and Technology, Tsukuba, Japan tionary meteorological satellite, made every 2.5 minutes with a horizontal resolution of 2 km (Bessho et al. 2016) and precipitation analyses from the JMA's Radar-AMeDAS system (Nagata 2011), which estimates precipitation over Japan every 30 minutes with a horizontal resolution of 1 km, correcting composite radar echoes with rain gauge observations. Figure 2 compares the probability density functions (PDF) of IR brightness temperature observed by Himawari-8 with those of the simulation results for each value of Δx. The latter is computed through a satellite simulator implemented in JMA-NHM. The probability for a lower brightness temperature below 220 K increases as Δx decreases only for Hokuriku in Exp-Winter, but such a systematic change is not seen for either Kanto or Kansai in Exp-Summer.

IR brightness temperature
The simulation results exhibit large discrepancies compared with the observed IR: differences of several orders of magnitude between the model outputs and satellite observations are seen for brightness temperatures below 237 K. This temperature coincides with the homogeneous freezing of cloud water in the model, which implies an issue with the cloud microphysics modeling. This study does not go further into this subject, but at least we found that the horizontal resolution has little effect on the PDF.

Surface precipitation
This study employed the fractions skill score (FSS) that has been suggested as suitable for evaluating the precipitation in high-resolution models (Roberts and Lean 2008;Duc et al. 2013) instead of classical point-by-point validation. This is because experiments with finer horizontal resolution are intrinsically more prone to position errors if point-by-point validation is used. In FSS, position errors are allowed for a fixed neighborhood size.
All of the Radar-AMeDAS data and simulation results with different Δx were projected on a verification grid with horizontal resolution of Δx = 1 km. In this projection, each grid value for the verification is derived by an original grid value at the nearest grid point, except for that of the finer simulation results for Δx = 500 m which is given as an average of original grid values in 2 by 2 boxes. The fraction of surface precipitation occurrence larger than a threshold r in square neighborhoods of size L by L (km) are obtained for both the model M(n) and the observation O(n) at a verification pixel at (i, j) for a time slice t as ) and Niino 2009). The slab surface model (Saito et al. 2006) was employed at the bottom. Topography and land use in the model with each Δx were given by smoothed finer resolution gridded data (National Land Numerical Information and GTOPO30 distributed by the Geospatial Information Authority of Japan and United States Geological Survey, respectively).
The two target areas for Exp-Summer, Kanto and Kansai, have the largest and the second largest populations in Japan, respectively. Thus, heavy precipitation in these areas may have a significant civil impact. Furthermore, the environmental factors of Kanto in the summer tend to be among the most unstable in Japan (Chuda and Niino 2005), and showers occasionally occur in the afternoon (Nomura and Takemi 2011). The Kansai area is at similar latitude as the Kanto region, but has a more complicated topography (Fig. 1). Precipitation over the area is more likely to be associated with the topography.
For Exp-Winter, in contrast, the Kanto and Kansai areas have less precipitation. However, Hokuriku, which is on the other (northern) side of the mountain ranges and faces the Sea of Japan, has a greater chance of snowfall due to outbreaks of cold air from the Eurasian continent and succeeding air-mass transformation over the warm Sea of Japan. The sea beneath the atmosphere is so warm that vigorous cumulus convection accompanied by lightning frequently occurs (e.g., Ishii et al. 2014).
To verify the numerical results, we used infrared observations (IR band #13, wavelength 10.4 μm) by the Himawari-8 geosta-  where I M and I o are binary values that depend on whether the surface precipitation is larger than the threshold r (1) or not (0). Then the FSS is given by summing over a target area and time slices as The FSS for each target area was evaluated by the sums of the hourly-accumulated precipitation for every hour between 1200 and 1900 JST on every day through the series of the experiments. Figures 3a1, 3a2, and 3a3 present the FSSs for r = 0.1 mm/h. To emphasis the horizontal resolution dependences whose qualitative features are found to be varied little by L, Figs. 3b1, 3b2, and 3b3 show the differences of the FSSs (L = 20 km) from those in the experiments with Δx = 2 km for various r.
The FSS of experiments with Δx = 5 km is mostly worse than those of the others. Comparing the difference between Δx = 5 and 2 km, there seems little benefit in increasing the grid spacing further from Δx = 2 km in Kanto of Exp-Summer (Fig. 3b1). Whereas, certain advantages are seen for finer Δx in Kansai and Hoku riku (Figs. 3b2 and 3b3). For some results of the heavy rainfall (large r), FSSs of Δx = 1 km and 500 m are inferior to that of Δx = 2 km and even Δx = 5 km (Fig. 3b1).

Resolution dependence in the 3-dimensional model output
From an examination of the 3-dimensional model outputs, certain dependencies on resolution are seen for Δx finer than 2 km. The present study employs a simple procedure that represents the average "width" of an updraft by assuming that all updraft regions are shaped like a square with a certain width Φ w . The width is estimated as the square of all the updraft regions divided by the length of their boundaries. We obtain the number of updraft grid points where w > 0, n up , and the number of grids at boundaries, n bd . The estimated values of Φ w ≡ n up /(2 × n bd ) for each target area and Δx are shown as solid lines in Figs. 4b1, 4b2, and 4b3. Despite the crude assumption regarding the shape of the updraft regions, the systematic vertical profiles imply that the use of the simple procedure is reasonable. The width of the updraft regions shows an almost linear dependence on Δx. Figures 4c1, 4c2, and 4c3 show similar "widths", but of cloudy regions for which the non-zero mixing ratios of the hydrometeors are presented. The cloudy region width, Φ qw , is much larger than Φ w , and also depends significantly on Δx. However, the sensitivity to Δx for Φ qw is not as strong as for Φ w .
The averaged mixing ratio of the hydrometeors, q w , depends little on Δx for all the analysis regions (Figs. 4d1, 4d2, and 4d3). By examining the breakdown of the hydrometeors, a significant dependence on Δx is found only for the mixing ratio of graupel in Hokuriku in Exp-Winter.

Discussion
The increase in Δx up to the sub-kilometer scale does not result in significantly better performance in Kanto in Exp-Summer (Fig. 3b1). In fact, both the timing and amount of precipitation among Δx = 2 km, 1 km, and 500 m vary little regardless of whether they are consistent with observations.
The benefit of an increased Δx up to 500 m (but not 1 km) is relatively apparent in the Kansai area (Fig. 3b2) where the topography is more complicated. In Kansai, the updraft intensities and the amount of hydrometeors are similar to Kanto (Figs. 4a1 and 4a2,Figs. 4b1 and 4b2,and Figs. 4d1 and 4d2), although the widths of the cloudy regions are smaller (Figs. 4c1 and 4c2). These smaller scale clouds may be associated with the small-scale topography in the area (Fig. 1). The following two points regarding the Kansai results are intriguing, but we have not found solid reasons: the PDF of the brightness temperature below 220 K is higher particularly for Δx = 1 km (Fig. 2b), and the FSSs of Δx = 1 km and 500 m are worse particularly for moderately weak thresholds (i.e., 0.5 or 1 mm/h).
For Δx = 1 km and 500 m, the updraft intensities in Exp-  Summer have maxima below the altitude of 1 km (Figs. 4a1 and 4a2), implying the appearance of daytime convection in the PBL. With Δx = 1 km or 500 m, the PBL convection is still incompletely resolved (e.g., Ito et al. 2015), and convective transports in the PBL would be carried out by both the resolved updraft and subgrid PBL model. Since the prevailing altitude of the cloud layer (~5 km) is far above the PBL, whether the PBL convection is resolved has little impact on cloud properties (Figs. 4d1 and 4d2) and precipitation (Fig. 3b1).
In Exp-Winter, the averaged resolved updraft intensities are much stronger than those in 4a2,and 4a3). This is perhaps due to the cumulus convection prevailing in the stationary unstable environment of Sea of Japan in winter. The altitude of the cloud base on average is much lower than it is in summer (Fig. 4d3), so PBL convection is likely to be coupled with cumulus convection above the PBL. Unlike in Exp-Summer, the difference in the updraft intensity certainly affects ice particle distributions: the mixing ratio of graupel, q g , depends strongly on Δx (Fig. 4d3).
As mentioned above, improving Δx finer than 2 km in Kanto in Exp-Summer turned out to be of little benefit, but there was an exceptional day (30 July) on which finer Δx performed notably better. On this day, convective cells caused heavy rainfall in the afternoon in Kanto, and the largest count of cloud-to-ground lightning during the period of Exp-Summer (345 times in the Kanto area between 1200 and 1900 JST) was observed by the Lightning Detection Network (LIDEN; see Ishii et al. 2014) system operated by the JMA. The FSS obtained for this day (Fig. 5a) exhibits much larger differences between Δx = 2 km and 500 m (~0.1) than the average over the experiments (~0.01; Fig. 3a).
The heavy rainfall was seen in the center of the Kanto area in the observation (Radar-AMeDAS) at 1600 JST (Fig. 5b). The rainfall was indeed simulated in both of the experiments with Δx = 2 km and 500 m, although positions of areas with the heavy rainfall were deviated by several tens of kilometers to the west (Figs. 5c and 5d). These simulation results look equally poor.
Nevertheless, a slight improvement by increasing the horizontal resolution is found: the simulated convective cells moved little in the experiment with Δx = 2 km, whereas the system propagated to the east in the experiment with Δx = 500 m as indicated by the red arrow in Fig. 5d, although the propagation was still slower than that in reality. This difference in the positions of the convective cells appears to contribute to the better FSS of this day for Δx = 500 m.
The surface winds in the periphery of the convective cells (i.e., the vectors inside the boxes in Figs. 5c and 5d, southwesterly in the south and southerly in the north) were larger for Δx = 500 m, suggesting that stronger outflows associated with cold pools were present and affected the propagation of the cells. Similar to the results of Exp-Winter, the mixing ratio of graupel is notably larger for Δx = 500 m (not shown). The larger concentration of graupel may induce the stronger outflow. The larger amount of observed lightning implies that formation of abundant graupel in the convective cells is a reality. It is noted that Schwartz et al. (2017) has evaluated regional NWPs over areas of the United States and found simulated cold pools tend to be stronger for those with finer horizontal resolution.

Concluding remarks
Increasing horizontal resolution from 5 to 2 km yielded better performances in NWPs, but further increase in the horizontal resolution produce less improvements, especially in the results from the Kanto area where topography and cloud microphysics processes are of less concern. Nevertheless, the intensity and size of updrafts and the size of clouds strongly depend on the horizontal resolution.
The PBL convection appears to be resolved if the horizontal resolution is better than 1 km. However, whether PBL convection is resolved seems to have little impact. The cloud microphysics processes associated with graupel seem to be more strongly affected by the horizontal resolution due to the differences in the intensities of resolved updrafts in clouds. Tuning of processes associated with clouds is likely to result in larger differences.
As with surface precipitation, we did not find notable systematic differences in surface wind speed or temperature between experiments with horizontal grid spacing better than 2 km in the results from the Kanto area, except that their variations are definitely larger with finer horizontal resolution. Systematic changes are likely to be caused by the passage of mesoscale disturbances. It might be expected that triggers of cumulus convection emerge at finer resolutions and the growth of convective systems is realistically simulated, but the present results suggest that these are not achieved by merely increasing the horizontal resolution to the sub-kilometer scale.
The resolution dependences associated with the simulated updrafts and clouds are significant even for the sub-kilometer scale. Further increases in the horizontal resolution (Δx~100 m) might bring better performance. However, holding the same domain at such a resolution is beyond current computing capabilities. If a smaller numerical domain is used, the effects of confining the domain should be somehow evaluated, and comparison may not be as straightforward as in the present study.