Guidance on Using Climate Data in Decision-Making

The dynamically downscaled WRF dataset and the hybrid downscaled LOCA2-Hybrid dataset were developed to emphasize different strengths of the two downscaling methods and provide users with distinct dataset options. However, the two datasets differ in several important ways. Therefore, before deciding which dataset to use, users should review the hyperlinked data memos above and this data catalog, which provide fundamental information about the datasets, as well as the downscaling methods, uncertainties, and credibility of the data. To provide one illustrative example of how these datasets can differ – although both datasets provide downscaled climate projections over California at 3-km grid spacing, WRF has hourly data and more variables but for a much smaller set of GCMs, only one SSP, and only one ensemble member. LOCA2-Hybrid data is available for fewer variables at daily resolution but provides projections from many more GCMs, multiple ensemble members, and often for three SSPs. Additional information on WRF and LOCA2-Hybrid downscaling approaches can be found in journal articles linked in the References page.

In general, choices between these two datasets may be based on:

1. Data fit - i.e. whether the datasets have the variables, temporal and spatial scales, and SSPs that fit the user’s application context
2. Sampling needs - i.e. whether the user needs to sample widely across models and ensemble members
3. Consistency - i.e whether there user needs dynamical or physical consistency among different climate variables
4. Bias correction - i.e. whether the user needs a specific type of bias corrected data

Broadly, users should first map the key characteristics of both datasets, and determine if one or both have the specific types of information (scales, variables, models, SSPs, etc.) that are essential for a given context or application. For most applications that need daily data within California, the LOCA2-Hybrid dataset provides a wider range of models, ensemble members, and SSPs. The WRF dataset is useful in cases where users need hourly data, WECC (Western Electricity Coordinating Council) wide data, a larger set of variables than the LOCA2-Hybrid dataset provides, or dynamically consistent variables. For example, for a WECC-wide analysis that requires hourly wind data the WRF data might be the only viable option, because WRF offers hourly wind data at a 9-km resolution that covers the WECC region. For examining the return period of daily temperature or precipitation extremes (i.e. 1-in-X- events) in California, the LOCA2-Hybrid dataset might be more appropriate since it enables a wider sampling of possible outcomes. In this case, users might also choose to simultaneously examine the WRF dataset, to see if and how results look across the two datasets (see next question).

Both WRF and LOCA2-Hybrid are credible datasets, however they differ in how they were developed. For example, the datasets differ in how they were bias-corrected and how some of the physical variables such as wind or precipitation were computed. Therefore, there is not an easy apples to apples comparison between the two datasets, and they cannot be readily combined together. However, it may be beneficial (and appropriate) to look at both WRF and LOCA2-Hybrid data in tandem but as separate datasets. For instance, if a user wants to understand how certain extreme events will change in the future, they can examine the detailed dynamics of a single extreme event within the hourly WRF data while also separately examining the distribution of similar events within the much larger set of LOCA2-Hybrid projections.

Because of the differences in the two datasets, it is not advised to ‘combine’ WRF and LOCA2-Hybrid into a super ensemble dataset without additional data processing. If users do indeed find it necessary to combine WRF and LOCA2-Hybrid data, they should first consult with a climate science expert on the type of additional data processing that might be essential to make the two datasets comparable with one another. For example, users may need to ensure that the bias correction approaches that have been applied and the computation of physical variables such as wind or precipitation are comparable across the datasets. Also, since some of the same model runs have been downscaled using both WRF and LOCA2-Hybrid methods, the issue of double counting the same run would need to be addressed.

If users determine that the WRF dynamically downscaled dataset is most appropriate for their application, it is recommended that they choose the a-priori bias corrected models [i.e. MIROC6 r11i1p1f1, TaiESM1 r1i1p1f1, EC-Earth3 r1i1p1f1, MPI-ESM1-2-HR r3i1p1f1, EC-Earth3-Veg r1i1p1f1]. These models will likely be the most appropriate option for energy sector applications due to their ability to produce more realistic representations of regional climate than the non-bias corrected WRF models.

Detailed analyses of the non-bias corrected WRF data have shown that they generate results for precipitation, temperature and snow that do not seem to be physically plausible. These non-bias corrected models might still be useful for some advanced applications where users only want to examine the degree of change in the projected data (i.e. change between modeled future and modeled historical conditions) without using absolute values. In this case, users should consult with a climate science expert to carefully factor the implications of the biases in the data into their analyses and results. Users can also review this bias correction memo and this manuscript that describe the benefits of bias-correction in WRF data. Additional documents on bias correction are available on the Analytics Engine References page.

The Analytics Engine data catalog shows the total number of models and ensemble members from both WRF and LOCA2-Hybrid datasets that are available to users. Broadly, the WRF and LOCA2-Hybrid datasets include several models, and LOCA2-Hybrid has multiple ensemble members for a select few models. Both the WRF and LOCA2-Hybrid datasets come from GCMs that have been pre-selected based on a rigorous skill evaluation of how well they capture key characteristics of California’s climate (Krantz et al., 2021).

Once users have selected a dataset, it is generally recommended that they examine as many models and ensemble members as possible to develop the most complete picture of the range of plausible future outcomes. The Analytics Engine has tools and functionalities that allow users to more easily work with a larger number of models than was previously feasible for the energy sector.

However, there may be certain instances where selecting a smaller number of model runs is necessary (e.g. time series data applications). How a user undertakes this type of sub-selection will depend on the type of climate data application (including the variable, location, and scale of analysis) and the user’s ability to process large amounts of data. In cases where it is necessary to select a smaller set of model runs, users must first perform a preliminary exploratory analysis on the full suite of available data within the Analytics Engine to determine the appropriate set of models and ensemble members for their specific context. During the exploratory analysis, it might be useful for users to determine if, for their specific application, they care more about (1) sampling across a range of potential outcomes, or (2) gaining a statistical understanding of (changing) extreme weather events. While choosing between these two different options is not ideal and is always limiting, there are cases where such difficult trade-offs might need to be made based on practical considerations.

1. For cases where it is determined that a range of outcomes are needed, the users’ preliminary analysis could entail a ranking exercise to characterize all the available model runs for the range of outcomes for the region, timescale, and primary variable(s) of interest (e.g. characterizing model runs from hottest to coolest, or wettest to driest, or based on the change signal in the models). Users can then sub-select a set of model runs to represent the median, mean, and/or either tail of the distribution, depending on which aspect of the distribution is most relevant to their application. If the goal is to select a plausible range of outcomes, then it may not be necessary to distinguish among models and ensembles. If calculating a mean, median, or some other statistical property of the model distribution, though, users should be mindful of how models with different numbers of ensembles are weighted.
2. For cases where it is determined that a statistically robust understanding of changing extreme weather events is required, it is recommended that users avoid sub-sampling to the extent possible. Extreme event analyses require large ensembles of climate data because these events are often poorly sampled within a single climate model run (a 1-in-100 year event may only occur once in a given ensemble run). Properly understanding the statistics of such extremes requires large ensembles so that there are enough instances of rare events to understand their properties and frequencies. A user’s preliminary analysis should then entail an assessment of both the internal variability and model uncertainty of the entire dataset. This is because both these types of variability are important for extreme event analyses. Internal variability can be examined in the Analytics Engine as the differences among ensemble members from each individual model (i.e. as intra-model variability), and model uncertainty can be examined as the differences between various models in the dataset (i.e. as model-to-model variability or intra-model variability). The availability of multiple ensemble members for some models hosted in the Analytics Engine makes it possible to conduct extreme event analyses for each individual model, and to distinguish how different models represent changing characteristics of extreme events. After comparing across models to see how similarly or differently they represent the internal variability, users can then choose models and ensemble members accordingly. If a user still finds it necessary to sub-sample the models, they should perform the aforementioned analyses of the two types of variability for all the Analytics Engine models that have multiple ensemble members. Users should then cross-check the inter-model spread of their chosen sub-sample with the inter-model spread of the broader dataset to check if the full range of outcomes is captured in their sub-sample.

The default setting in many Analytics Engine notebooks is to provide outputs that span all models and ensembles. Users can use the tools in the Analytics Engine to examine both the internal variability and model uncertainty (see example applications here). Model sub-selections can also be done through the Cal-Adapt Data Download Tool or the API.

While it is always recommended for users to examine as many models and ensemble members as possible, Analytics Engine also hosts a set of “General Use Projections”. These five LOCA2-Hybrid model runs [ACCESS-CM2 r1i1p1f1, MPI-ESM1-2-HR r3i1p1f1, EC-Earth r1i1p1f1, FGOALS-g3 r1i1p1f1, MIROC6 r1i1p1f1] have been pre-selected to reasonably capture a range of future outcomes for a limited number of commonly used climate variables, scales, and SSPs. If users are unable to perform context-specific preliminary analyses as suggested in the previous question and/or are less comfortable working with large amounts of data, the GUPs can provide a minimum set of model runs to start with before more sophisticated analyses can be done.

Users should review this short memo [Link upcoming] for details on how the GUPs were identified and what ranges of outcomes they span (and do not span). Broadly, the GUPs were evaluated for SSP 370 and 2045-2074 (mid-century) changes relative to the 1950-2014 historical period. This was done for selected variables including temperature (change in statewide average annual daily Tmax, degree F), precipitation (change in statewide average annual precipitation, % change), and wind (change in statewide average annual wind speed, % change). In the assessments, the five selected GUPs were able to reasonably capture the range of projected changes for the above-mentioned precipitation and temperature metrics. Future changes in wind speed projected by these GCMs varied quite a lot by location in the state, therefore users who require changes in wind at specific locations should evaluate a broader set of models for their specific location of interest rather than relying solely on the reduced set of general use projections.

Limitations of GUPs: Since GUPs only represent a minimum set of starter projections, they are limited in their scope, and users should exercise caution while drawing conclusions from this sub-selection. While the GUPs attempt to cover a range of variability for a limited set of pre-determined metrics, it cannot be guaranteed that these model runs will necessarily cover the appropriate range for every user-relevant outcome, i.e. for different time periods, SSPs, regions, and/or variables/metrics. Particularly for applications which involve examining extreme events, the GUPs may not be appropriate, as they may not be able to reasonably capture the full range of extremes. Therefore to the extent possible, it is recommended that users rely upon a larger number of models and ensembles, or conduct their own preliminary analyses to determine which models are needed to capture the appropriate range of plausible outcomes for their specific contexts (as recommended in the model selection question). It is worthwhile to reiterate that existing and forthcoming tools of the Analytics Engine will make working with larger amounts of model runs much easier for users, and therefore users will be able to make use of the full suite of data rather than relying on limited sets of projections.

Temporal scale considerations for users often include four key choices (1) temporal resolution of the data (hourly or daily), (2) sampling window (number of years of analyses) (3) temporal aggregations (whether and how to average across years of analyses) (4) timeframe (historical, earlier-21st, mid-century, end-century). The data and tools in the Analytics Engine allow a user to work with different options for these four choices. Note that these choices have big implications for the size of the data and computational time required to perform analyses. Therefore choices need to be made thoughtfully based on the type of application and potential tradeoffs, i.e. choosing the finest temporal resolution of data might limit the ability to work with longer sampling windows or a larger number of models (if data size is an issue). A user can also conduct preliminary analysis within the Engine’s server to examine the implications of these different choices before a decision is made on the temporal scale of the data to be downloaded for further use.

(1) Temporal Resolution

In the AE, depending on the type of data and variable of interest, a user can select between hourly or daily resolution data or pre-aggregated data at daily or monthly scales (refer to this Analytics Engine data table for more details). Users’ choice about the temporal resolution of the data should be informed by the temporal nature of the climate hazard of interest and how it interacts with the user’s system and planning process. Overall, the broad recommendation is for users to identify specific temporal aspects of their application, start coarsely with some preliminary analyses, and then work towards larger sets of data with finer resolutions. For instance, if users are working with broader trend analysis, changes in central tendency, or seasonal questions, where very specific temporal characteristics are less relevant, they may be able to work with data at daily scales or even pre-aggregated monthly, seasonal, or annual scale data. However, if the application or process involves a daily or hourly model, or if an 8760 analysis is needed, then the user will likely need to work with the higher resolution data. Generating a Typical Meteorological Year (TMY) is another example where hourly data may be required for a long climatological period. See the TMY methodology notebook on AE.

Additionally, if a user is looking at applications or hazards where a certain temporal characteristic (e.g. time of day or the diurnal pattern of data) is of critical importance, then the choice of daily versus hourly data can make a big difference to the results. For instance, examining wildfire risk in the afternoons, hourly exceedances for parking lot design, storm surges, or Santa Ana winds may need hourly or sub-hourly data, whereas examining the recurrence of long-term droughts might not require as fine resolution data (i.e. it may be okay to trade off finer resolution data in exchange for the ability to include a larger number of years in the sampling window).

(2) Sampling Window

Often the choice of an appropriate sampling window is determined by the type of planning or modeling processes that a user is interested in. It is important to note that longer time windows (i.e. 30+ years) are better able to sample key modes of variability that affect California, e.g. a 30 year sampling window is needed to capture a reasonable number of El Niño or La Niña cycles, as well as positive and negative Pacific Decadal Oscillations. In addition, some applications like Extreme Value Analyses require at least 30 years of simulation to appropriately pull event maxima. While it is possible to select and examine shorter sampling windows within the AE, users need to be aware that these windows might not adequately represent the modes of variability, and therefore may not be fully representative of the various hazards that might impact their systems (i.e. there is a possibility of missing out on key hazards). If users have to choose a sampling window that is less than 30 years, it is recommended that they use as many models and ensemble members as possible in their analysis. Using data with many different initial conditions and perturbations enables a better representation of different forms of natural variation over a shorter sampling window, than a single model and ensemble member.

(3) Temporal Aggregations

When analyzing data at certain resolutions and sampling windows, some applications may require further temporal aggregations (e.g., for input into other models, or for developing summary statistics or graphs for reports). The tools in the Analytics Engine allow users to analyze events or metrics at a finer scale within the engine’s servers, and then aggregate and download (or further analyze) the data at a coarser scale. For instance, users can evaluate certain heat wave metrics at a fine scale within the engine (e.g. use hourly data to examine total hours of heat index above 90 degrees F). Users can then choose to temporally aggregate the resultant derived heat wave data more coarsely before downloading (i.e., total daily/weekly hours of heat index above 90 degrees F, or daily max heat index) without losing the quality of information. In this manner, the Analytics Engine enables users to examine heat waves at a fine scale using hourly data, without having to download the entire suite of hourly data from every model. Some fine-scale metrics (such as heat index) are already pre-aggregated in the Analytics Engine (some of these pre-aggregated metrics can be found here).

Broadly, it is recommended that the temporal averaging (if necessary) be done at the end, once the temporal variability and distribution have been examined and considered within the Analytics Engine server. For example, upon examining hourly versus more temporally aggregated data, it was found that for energy demand forecasts, it is more appropriate to calculate demand for each hour and then average or summarize the demand outputs across several years. This was found to be more appropriate than calculating energy demand from the average over a day or over several years, since energy demand varies non-linearly with temperature and other factors. Examples of how to work with temporal aggregations of hourly data depending on desired metric are demonstrated in AE’s Heat Index notebook and the Annual Consumption notebook.

(4) Timeframe

How to choose a timeframe for analysis (whether to analyze climate impacts for mid-century, 2070, end-of-century, or any other period) is one of the most common questions that users have. Users often need to align their timeframe choices with their planning horizon and/or with the lifetime of the asset/infrastructure under consideration. However, climatic changes, as well as the impacts and risks that stem from these changes, could be non-linear and increase at a more rapid rate towards the end of the century. Hence, near-term timeframes might not always provide a holistic picture of long-term risk and impacts, particularly for the higher SSPs. For instance, if peak load forecasting is analyzed only until 2050, it might not provide a good understanding of the risk that might ensue post 2060. Therefore, when feasible, users should consider the evolution of risk over longer periods of time (e.g. 2070, end of century, etc), and then break up the analysis into shorter time windows that better align with their planning horizons, asset lifetimes, or other application-specific considerations. Such longer-term analyses may be essential to better understand how decisions or actions may need to scale over time, particularly for climate impacts that are likely to change at a much faster rate after mid-century (e.g. sea level rise). It can be noted here that using Global Warming Levels (GWL) as an alternate framework can help to overcome some of these challenges (see question on GWLs below), as this framework is more suited to adaptive planning.

Spatial scale considerations for users often include three key choices (1) spatial resolution of data (the size of grid cell or data at a point location), (2) spatial extent (the area over which data is analyzed) (3) spatial aggregations (whether and how to aggregate over a larger geographic area, such as counties or hydrological units, HUCs). Similar to temporal scales (see above question), the data and tools in the Analytics Engine offer different options for these choices. Given that these choices have big implications for the size of the data and computational time required to perform analyses, it is recommended that users conduct preliminary analyses within the Analytics Engine server to examine these implications before deciding on the spatial scale of the data to be downloaded or synthesized for further use.

(1) Spatial Resolution

The first spatial scale choice that climate data users often need to make is whether they need data at a point location or if they are able to work with gridded data. In general, if users want to examine a hyper-local issue (e.g. a point-based asset such as a substation) or if they are working with a model or workflow that is parameterised or dependent on a particular weather station, then they may need to use point location data. However, if users are looking at regional climate responses, then gridded data might be most appropriate.

Point location data: Climate models do not automatically output data at point locations. However, the Analytics Engine has developed a Weather Station Localization notebook that allows users to generate data at particular locations for which the user has historical data. Hourly temperature data is also available on the Analytics Engine at 32 select weather stations that were identified as important for the energy sector for the LOCA2-Hybrid dataset. If users are looking to examine a point location that does not have available historical data, then they would need to conduct a qualitative assessment using the finest resolution gridded data for the nearest gridcells to the point location to determine how representative the grid cell average is of the specific point location.

Gridded data: If users choose the LOCA2-Hybrid dataset, this is offered only at 3km x 3km resolution. However if users choose the WRF dataset, this data is available at 9km x 9km as well as 3km x 3km resolution. When choosing a spatial resolution, the broad recommendation is that users should first identify the specific spatial aspects of their application, and then conduct preliminary analyses in the Analytics Engine servers to compare between different resolutions. Users should evaluate the tradeoffs (information that is gained or lost) by choosing different resolutions. For instance, if users have data size or computational constraints, choosing very fine resolution data may restrict their ability to examine a greater number of model runs or larger spatial extents. It can also be noted that higher resolution spatial data is not always better or necessary for every analysis. Many informed decisions and investments can be made by using lower resolution data that considers more models and plausible future outcomes. When conducting preliminary analyses, users must also be aware of the spatial heterogeneity/variability (topographic, climatic, or other) within the region that they want to assess. Finer resolution data could provide greater benefits for regions that are spatially heterogeneous (e.g. areas of complex terrain or regions that transition quickly from water to land) as compared to regions that are more spatially homogeneous (e.g. the Central Valley). It can be noted that many areas in California are characterized by a high degree of spatial variability in weather, topography, and the human dimensions of infrastructure and demographics. Therefore, depending on the region and climate variable under consideration, the spatial heterogeneity within a grid cell (and also across adjacent/nearby grid cells) might be large. Overall, the goal should be to select the most appropriate spatial resolution for the context, that also allows for a broad range of model runs to be included.

(2) Spatial Extent

The choice of spatial extent is often dependent on the risk or climate hazard that users want to analyze, and/or the question they are trying to answer. The spatial extent needs to be large enough to properly analyze the climate issue under consideration. For instance, to analyze the impacts of a heat event on system-level planning, a user may need to examine the entire regional extent where the impacts of the heat events may occur. For other specialized analyses, such as evaluation of streamflow, sector-specific domains (or spatial extents) such as watersheds can be applied to gridded climate data. Here, care must be taken to create a rigorous workflow which carefully considers edge cases where a gridcell may be only partially included in a polygon area (In the AE, functions such as rioxarray clip have been used to better examine some of these issues). Users should also note that conducting hyper-local spatial analyses (e.g. using census tracts in urban areas) may require specialized approaches that are highly dependent upon a given question or application, and therefore are beyond the scope of this general guidance document.

(3) Spatial Aggregation

Some applications may require aggregating data across a larger spatial area (such as a county or a watershed) before it is downloaded or examined for further use (e.g. for developing synthesis graphs or summary numbers for reports). However, users should note that some of the quality of the information may be lost when spatially aggregating across different locations or gridcells. For instance some aggregations (such as averaging over large spatial areas) could lead to missing out on specific distributional impacts for different communities, which might have equity implications. Therefore, users need to be aware of the types of impacts that they might miss out on when aggregating across gridcells. If aggregation is necessary, it is recommended that users do not default to spatial averaging. Instead, users should first examine climate impacts without aggregation, and then aggregate at the end, once the spatial variability and distribution have been examined within the Analytics Engine server. For example, finer scale analyses of population-weighted heat exposure can be first assessed at the gridcell level and then aggregated to a county level for reporting, after the variability has been examined. In such cases, users should consider reporting other distributional statistics over the aggregated areas, such as the spatial variability within the county.

The homogeneity of the climate hazard over the spatial area that a user is aggregating over is also very important. For instance, aggregating heat metrics over a largely homogenous plain area might be less challenging than aggregating precipitation over a topographically diverse area. In addition, users must exercise further caution when aggregating over gridcells for assessments of extremes, such as for extreme value analysis. In extreme value analyses, if multiple gridcells from a climatologically diverse area are averaged, there is a chance of accidentally biasing the results to the most extreme value in the area and/or missing an extreme of interest. As in several of the previous questions, users will need to make an informed decision about the right balance between performing a good quality and rigorous analysis versus one that is computationally light and technically feasible within their context.

[Upcoming]

Until the last few years, it was most common to assess climate projections at a particular time period, such as mid-century or end-of-century, and for different emission scenarios (RCPs or SSPs). However, it is becoming increasingly common to assess climate projections at different levels of warming, for example how regional climate change might look like in a world that is 2˚C or 3˚C warmer on average than pre-industrial levels. Key national and international climate assessments such as the IPCC AR6 and the NCA5, are using the warming level framework for several reasons. This blog and slides 11-17 of this presentation explain in detail why the warming level framework has been preferred.

Broadly, one of the main reasons for using GWLs rather than a time-based approach, is that there is less certainty around the rate (timing) of climate change than the range of possible temperature outcomes at a given level of warming (see this Analytics Engine blog for a visual representation). When using a time-based approach to plan for a specific target year(s), users are often forced to pick projections from a particular emissions trajectory. This limits the overall sample size of model runs and therefore reduces the ability to holistically characterize future outcomes, specifically extremes and rare events. On the other hand, the GWL framework allows users to avoid choosing among the different emissions trajectories and hence increases the sample size of model runs that they can use. That is, to evaluate climate impacts in a 2˚C world one must combine the regional projections from all available simulations, regardless of the emissions scenario and regardless of when they reach 2˚C of global average warming. This larger sample size provides users with a greater ability to holistically examine risks and, in particular, to better characterize risks from extremes. The approach also allows for isolation of the different sources of uncertainty in regional climate responses, which lends itself to a more adaptive management approach (see this Analytics Engine blog for more details). Overall, the GWL framework allows more robust climate change analyses, as summarized in the IPCC AR6 “…robust projected geographical patterns of many variables can be identified at a given level of global warming, common to all scenarios considered and independent of timing when the global warming level is reached.”

The AE’s Warming Level Notebooks allow users to easily examine regional climate impacts for a specific global warming level. In these notebooks, users can first identify target warming levels (e.g. 2˚C or 3˚C global warming), and then calculate the regional climate response of interest for those GWLs with the largest possible set of model runs. These notebooks also have functionalities that allow users to examine the median time period when a given warming level might be reached (i.e. go from GWL to time estimates), thereby also providing a time-based estimate for planning purposes. This example application can help users get familiar with the functions of the notebook. Similar to the recommendations on choice of timeframes, it is also recommended that users do not fix-in on any one warming level, but also undertake exploratory analyses at additional warming levels. In particular, including a higher-end warming level may be beneficial for capturing potential nonlinear impacts of climate change, which can enable adaptive planning.

All of the downscaled data available in the Analytics Engine comes from GCMs that have been pre-selected based on a rigorous skill evaluation of how well the models capture several relevant characteristics of California’s climate. These GCMs were shown to perform well for both process-based as well as local climate metrics. The models were able to simulate the key physical processes and patterns that strongly influence the hydrological cycle and extreme weather in California, and they were also able to capture local climatic patterns such as annual and seasonal patterns of temperature and precipitation. This sort of state-level GCM assessment is unique, and lends broad credibility to the data that is being hosted in the AE. More details on how the GCMs were evaluated and selected are available in Krantz et al., 2021.

Although all the models in the Analytics Engine have been shown to perform well at capturing California’s climate broadly, this does not necessarily mean that all models perform equally well for any specific sub-region within California or for any specific metric (particularly ones that were not evaluated within the GCM selection process). In such cases, users may want to additionally understand whether and to what extent the data being used is credible for their specific region, metric, or application. Here, an additional level of credibility analysis could be of interest to a user. However, the approach for such additional user-driven GCM evaluations is still an evolving area of research and experimentation that scientists and practitioners are collectively trying to figure out. If users are interested in undertaking additional credibility evaluations, it is recommended that they thoroughly review the Guiding Principles section [Upcoming], and consult with a climate expert who is well-versed in GCM evaluation approaches.

For additional credibility analyses, users may first want to qualitatively assess whether and to what extent the GCMs represent the key underlying physical processes that are relevant to the phenomenon or region of interest. This can be done using existing studies or literature on the GCMs. Then they may want to quantitatively examine the historical performance of the models or dataset, for the metric that is of interest. Upon such analyses, users might consider three key questions: (1) Are the class of models available suitable/appropriate for the specific use-case, climate phenomenon, or variable of interest? (2) Is there a need to differentiate between the various models based on their historical performance?, and (3) Should some models be avoided (or culled) for specific regions,variables, or applications? However, these are nuanced questions that require advanced analyses that cannot be predicted or easily identified.

One instance where an additional credibility evaluation might be useful to users, is for cases where specific wind variables are needed. Wind is often a tricky variable to capture well in climate models due to its extremely local characteristics, which can be affected by local buildings, trees, hills, etc. A user may want to validate that the data they are using reasonably captures the distribution of wind extremes in a given location of interest before proceeding to analyze changes in that metric in the future. (It can be noted that such a validation analysis might need access to specific observed data sets to compare to, which can be difficult to obtain for variables like wind). When such fine-scale features are being examined, it is possible that only a subset or even none of the models represent the phenomenon of interest very well. In this case users may want to rule out certain models or might determine that the available data is not appropriate for their application. Ruling out models should be balanced with considerations about sampling (see the model selection question), so that users ensure that they do not overscreen models and miss out on plausible and realistic outcomes.

Overall, it is worthwhile to reiterate that user-centric credibility evaluations are still an evolving matter in both science and practice. Sub-selections or culling of models based on very specific credibility assessments need to be done carefully and in consultation with climate science experts on the best course of action.

The goal of the climate projections in the Analytics Engine is not to provide a singular prediction of the future but rather to allow users to examine different plausible future outcomes. The data in Analytics Engine allows users to explore a range of potential future climate projections using different datasets, models, and emission scenarios. There is inherent variability within this range due to the natural fluctuations within the climate system (internal variability), variation in climate responses of different models (model uncertainty), as well as uncertainty in future emissions of greenhouse gases (scenario uncertainty). More details on types of uncertainty can be found in AE’s About Climate Projections and Models page, the Glossary page, and also in the "Choosing Climate Projections" section of the EPRI Climate Data Users Guide. Overall, users should be aware that there are uncertainties both in the amount of future greenhouse gas emissions as well as how the climate system will respond to these emissions. However, these uncertainties do not invalidate the usefulness of climate data for adaptation, but rather represent a necessary part of planning for climate hazards and risks. Further, climate modeling processes are continually improving to provide better representations of the climate system.

Although all three sources of uncertainties are important to understand, the relative value of each type of uncertainty depends on the type of application, context, and time-period of decision-making. Therefore, disentangling the different sources of uncertainty and examining how each of them evolve over time, can help energy sector users better characterize and work with the specific uncertainties that are most useful for their context. For example, internal variability might be more important for planning processes that focus on the near-term (e.g. the Integrated Energy Policy Reports, IEPR assessments), whereas scenario uncertainty might be of greater value to consider for end-of-century planning (e.g. long-term distribution planning).

In the AE, users can use the internal variability notebook, the model uncertainty notebook, and the forthcoming emissions uncertainty notebook to perform some of these uncertainty analyses (Example Applications). In addition, specific recommendations related to appropriate consideration of uncertainty are also included within some of the previous answers, such as model selection, spatial scale selection, and temporal scale selection.

These guidelines have been developed as part of the CEC-funded project EPC-20-007 with support from Amazon Sustainability, titled “A Co-Produced Climate Data and Analytics Platform to Support California's Electricity Resilience Investments”. The main aim of this project is to provide customized data, advanced analytics, and powerful cloud computing resources to support climate-informed planning and research to improve resilience in the electricity sector. The project uses a “co-production” approach, where the data, tools, analytics, and guidance documents (including these guidelines) are collaboratively developed alongside key energy sector climate data users and a diverse group of scientists.

The initial set of questions that have been answered here were developed based on interviews and focus groups with energy sector climate data users. In these interviews, users identified the most common challenges that they encounter while using climate data for planning and adaptation purposes. The initial answers to these questions were drafted by three climate scientists and two social scientists who are part of the Analytics Engine project team. These preliminary answers were informed by the latest climate science literature as well as first-hand knowledge of the issues faced by climate data users. The answers were then edited and refined through a rigorous and iterative process that spanned several months, and incorporated the perspectives of a wide-range of data users (such as energy utilities, state agencies, energy consultants), scientists (including climate scientists, climate data producers, social scientists, energy sector specialists), as well as software tool developers. This process included gathering both written and verbal feedback through deliberative focus group discussions. In particular, the project team from CEC’s EPC-20-006 project “Development of Climate Projections for California and Identification of Priority Projections” provided extensive feedback on several versions of the document.

• These guidelines provide users with a scientifically-grounded starting point for how to use climate data to answer real world questions. However, the onus is ultimately on the user to identify whether, and to what extent, these broad guidelines are sufficient for their specific context. There may be complex or nuanced cases where these guidelines either do not apply or are insufficient.
• These guidelines are only applicable to the climate projections data that are hosted in the AE. While some of the key takeaways could also be more broadly relevant, their applicability to other regions or planning contexts has not been examined.
• Some of the recommendations in the guidelines require access to computing resources, tools, and notebooks, such as those in the AE.
• These guidelines were developed based on the perspectives of a diverse but limited set of experts and their understanding of the usability of LOCA2-Hybrid and WRF datasets. As the credibility and actionability of this data is further examined by other scientists and practitioners, some of the guidelines might need to be re-considered.
• These guidelines have not yet been empirically tested/examined in terms of where (and for which applications) they work and where they do not. As users work through these guidelines, if there are suggestions for improvements please reach out to analytics@cal-adapt.org.
• Finally, these guidelines are not static. They are meant to be updated continuously over time as practical, scientific, and regulatory understanding of climate change evolves and as new data emerges.