Guiding Principles on the Use of Climate Data

Preamble

The purpose of this section is to provide users of climate data with broad principles and a process that can be followed when working with climate data for different applications. This process goes through the various steps in climate data analyses – from identifying data needs and choosing the right products, to analysing data and reporting on climate-related results. The principles were developed through 2.5 years of iterative deliberations among different scientists of the Cal-Adapt Analytics Engine Project, with regular feedback from key climate data users (including energy utilities) at various steps of the process. The principles try to balance scientific rigor and the practical constraints of data use to afford users different options to work with climate data. These principles can be useful for anyone who is currently integrating or looking to integrate climate data into different applications or decision-contexts.

How This Section is Organized

Principles

The main text of each principle describes what a user should do at that stage of the process.

Important Considerations

Each principle is followed by an Important Considerations callout, which provides background, rationale, and do’s and don’ts. This text has more technical depth and may require a basic understanding of some key climate-related concepts.

Boxes

Five cross-cutting boxes cover topics that arise across multiple principles. These are separated out for readability and are referenced from the relevant principles.

Principles

Click on each box for additional details on the background, rationale, and to find a step by step process for each principle.

Principle 1: Scope and identify data needs

• Identify climate data applications
• Identify audience for the analysis and results
• Involve relevant people
• Determine types of data analyses needed
• Develop a list of metrics, variables, temporal and spatial scales, scenarios, downscaling, and unknowns
• Determine risk and uncertainty tolerance
• Evaluate technical capabilities
• Evaluate computational capacity

Principle 2: Catalog available data

• Identify all datasets covering the spatial domain
• Catalog data details for each dataset
• Review metadata
• Identify available user guidance
• Identify data-related regulations or policies

Principle 3: Screen available datasets against requirements

• Conduct initial screening for fit of datasets
• Revisit important data considerations/criteria
• Assess tradeoffs between datasets
• If data size is a constraint, revisit spatial and temporal scale choices
• Select datasets that fulfil the most important data considerations/criteria
• Consider a preliminary comparative analysis between multiple datasets

Principle 4: Assess credibility of data

• Identify existing credibility assessments
• Critically review assessments to understand approach, known biases, and credibility at multiple scales
• Identify the most pertinent credibility-related considerations and determine if elimination of models/datasets is necessary
• Document reasons for elimination of models/datasets
• Cross-check to ensure that data spans a range of outcomes
• Determine if a context-specific credibility assessment is necessary

Principle 5: Sub-select data

• Determine whether sub-selection of models is necessary
• Develop a principled, step-by-step plan for choosing a well-balanced sample
• Conduct an exploratory analysis to characterize the distribution of model runs
• Use the distribution plots to choose an appropriate sample based on context
• Cross-check that the sample spans a range of outcomes
• Document and communicate the sub-selection approach

Principle 6: Analyze data and summarize results

• Revisit the purpose and audience for the data analysis
• Identify metrics, statistical analyses, and aggregations needed to analyze data
• Develop a step-by-step method to compute metrics, aggregate across scales, and synthesize across models and scenarios
• Consider model weighting, presenting results as change signals versus absolute values, and framing impact relative to natural variability
• Evaluate uncertainty
• Document and communicate the analysis approach and limitations

Principle 7: Evaluate and document the analysis

• Evaluate if the analysis answered the original questions
• If questions were not answered, revisit previous principles or reframe the original question
• Finalize documentation of the workflow
• Consider a plan and timeline for updating the analysis
• Determine how the analysis and results should be presented and communicated
• Archive the most important data and outputs

Additional Notes

The principles are presented sequentially and in a step-by-step manner, however a user may follow a different order based on their context. There are also many offramps for this process, and some users may not need to go through all the steps based on their specific contextual needs. This process can be highly iterative and may require certain assumptions and decisions (e.g. metrics, scales, or types of analyses) to be re-visited several times. Although the document presents broad principles that can apply to many different contexts and uses, specific applications will likely require more complex and nuanced examinations within one or more principles. Each principle or step of this process might also require specific technical skills in climate data analysis such as coding, statistics, data visualisations, understanding of basics of climate models, etc.

Additionally, this section does not include principles on how to conduct specific analyses downstream of the climate data, i.e. how to do climate risk assessments or a health impact assessment. It is meant mainly to help users understand how to pick and choose the right data and analysis approaches for analysing the climate hazards of interest.

Acknowledgments

We thank our many collaborators who contributed to the development of these guiding principles, including California Investor Owned Utilities, state agencies, consulting groups, and researchers. We also specifically thank Grace Di Cecco and Naomi Goldenson for their input on early drafts of the principles.