Guidance for Health Outcome Data Review and Analysis Relating to NYSDEC Environmental Justice and Permitting


The Environmental Justice (EJ) Advisory Group of the New York State Department of Environmental Conservation (NYSDEC) issued a report in January 2002 to the NYSDEC Commissioner. The report focused on the environmental permit process and included recommendations for creating an environmental justice program. It was followed by NYSDEC Commissioner Policy-29, Environmental Justice and Permitting (CP-29), issued March 19, 2003. This policy established two work groups to assist NYSDEC in developing and incorporating critical environmental justice information into the NYSDEC environmental review process: the Disproportionate Adverse Environmental Impact (DAEI) Work Group and the Health Outcome Data (HOD) Work Group.

In its report, the EJ Advisory Group recommended that closer scrutiny be given to environmental decisions in minority and low-income communities and that NYSDEC expand the types of information used in the permit approval process to address environmental justice concerns. The charge of the HOD Work Group as specified in CP-29 was to "identify reliable sources of existing human health data and recommend means to incorporate such data into the environmental review process." Members of the work group had technical knowledge and experience in the areas of environmental health, toxicology, epidemiology, risk assessment, or environmental sciences. In its report, the HOD Work Group identified available health outcome data, developed a method to describe and present health outcome data for use in NYSDEC's permit review process, and provide recommendations to DEC. The report is available on NYSDEC's website. Examples that have been updated with more recent health data are in Appendix A of this document.

How do I determine if a review and analysis of health outcome data in a community is necessary to the permit review process?

If a proposed action receives a positive declaration under SEQRA, an assessment of the impacts of the proposed action on the health of the community will be conducted as part of the environmental review. If a proposed action receives a positive declaration under SEQRA and may impact a minority or low-income community, the assessment of the impacts of the proposed action on the health of the community must include a review and analysis of health outcome data, which will provide a context for assessing the estimated impact and potential risk of the proposed action. If the population of a community has a higher rate of health-related events than comparison areas, it may be more vulnerable to the effects of environmental exposures. Where a positive declaration is not issued or the proposed action will not impact a minority or low-income community, a review and analysis of health outcome data may be requested by DEC staff if such an analysis could provide information needed to make a determination on the application.

How will the Health Outcome Data Work Group Report assist the development of an Environmental Assessment or Environmental Impact Statement?

A review of health outcome data can provide information on the current health status of the COC. The COC identified by NYSDEC staff for purposes of the health outcome displays and comparisons is defined as an area whose population is likely to be affected by at least one potentially significant adverse environmental and/or human health impact related to a proposed action, for which a permit application has been submitted. The process described in the Report provides a method for comparing the health status of the COC with that of other communities.

What are Health Outcome Data? What are Demographic Data?

Health outcome data are counts and rates of health-related events in a population. Examples of health-related events are hospitalizations for diseases such as asthma or cardiovascular disease, births of infants with low birth weight, and new diagnoses of cancer. Another health-related event is mortality from all causes or specific causes. A count is the number of events in a specific area over a given period. For example, in 2003-2005 the number of deaths from diseases of the heart was 4,887 in Monroe County and 2,077 in Rockland County. A rate takes into account the size of the population in those specific areas and is expressed as the number of events in a population of a certain size for a specific time period (usually a year). The age-adjusted mortality rate during this period from diseases of the heart was 188.9 deaths per 100,000 population for Monroe County and 253.0 deaths per 100,000 population for Rockland County. Rates allow us to compare one area with another and one area with a larger entity such as New York State. Age-adjustment of rates allows comparison of rates between communities with different age structures. (More information on age adjustment).

Although a display of health outcome data can provide information on the number and percentage of people in a community who have a variety of diseases or health outcomes, and on how this community compares in disease rates to other areas, it cannot tell us what is causing the disease in the community. Many factors influence the risk of disease including heredity, age, lifestyle factors such as smoking and diet, adequacy of nutrition, housing, health care, and exposures to chemicals in the air or water. However, if the population of a community has low health status, it may be more vulnerable to the effects of environmental exposures.

Demographic data, which are available from the U.S. Census Bureau, are data on the population in terms of the number of people and characteristics such as age, gender, race/ethnicity, income, etc. The U.S. Census Bureau collects this information every 10 years; the most recent information available is from the 2000 Census (select "Decennial Census"). The HOD Work Group recommends focusing on age, gender, race/ethnicity, and a measure of income such as median household income or percent of persons under poverty. Gender is important when considering health outcomes because certain health outcomes affect only one gender (e.g., prostate cancer) or are more common in one gender than the other (e.g., breast cancer, cardiovascular disease).

Focusing on specific age groups can also be important because some health outcomes are more common in certain age groups; for example, asthma hospitalization rates are highest in young children. The U.S. Census Bureau displays information on age by 5-year or smaller age groups, but these age groups are frequently combined so that fewer groups are displayed in published reports. Income level or the proportion of the population living in poverty is relevant because certain factors that can affect rates of disease are higher or lower in poor populations. These factors include smoking and exposure to second-hand smoke, quality of housing, adequacy of nutrition, and access to and source of medical care. Rates of some health outcomes vary among racial and ethnic groups, for example, infant mortality rate and mortality rates due to cardiovascular disease, breast cancer in females, and prostate cancer.

Where Do Health Outcome Data Come From and Which Data are Relevant?

There are many health-related events and many sources of data. Population-based disease registries, which contain records for people diagnosed with a specific type of disease who reside within a defined geographic region, are an important source of data. The New York State Department of Health (NYSDOH) maintains several population-based disease registries, including the Cancer Registry and the Congenital Malformations Registry. Another important source is the New York State Vital Statistics system, which includes information from birth and death certificates. Mortality rates and rates of birth outcomes such as low birth weight and infant mortality are obtained from Vital Statistics. Hospitalization data are obtained from the Statewide Planning and Research Cooperative System (SPARCS).

The HOD Work Group developed the following criteria to provide guidance for the applicant in choosing health outcomes to be displayed under NYSDEC environmental justice policy CP-29.

  1. The health outcome should be one that has some plausible relation to environmental exposure. The relation may exist because the exposure (1) is a potential cause of the health outcome, or (2) increases vulnerability to the effects of other causes of the health outcome. (This criterion is intended to distinguish a health outcome such as asthma that may be related to environmental exposures from a health outcome such as head injuries that is unlikely to be related to environmental exposures as defined above.)
  2. The data for the health outcome should be readily available to the public at the ZIP Code level or smaller geographic level. As of March 2008, some health data are available on the NYSDOH public web site at the ZIP Code level or at a smaller geographical level. However, we anticipate that more data will be available by ZIP Code in the future.
  3. The data for the health outcome should be able to characterize both the population of interest and a comparison area population, e.g., data should be collected on a countywide or statewide basis. Sufficient information should be available to describe the quality of the data. The quality of the data for timeliness and completeness should be rated low, medium, or high. Accuracy and consistency of data collection across the state or county are difficult to rate on this type of scale. Available information on how accuracy and consistency of data collection are assessed should be summarized and provided. A more complete discussion of data quality can be found in the HOD Work Group Report available from NYSDEC.

Based on the criteria in the report, the HOD Work Group developed a list of health outcomes recommended for consideration whenever health outcome data are displayed under NYSDEC's environmental justice policy. The most recent data available should be included in the health outcome data display. This list of potential outcomes could be revised in the future based on new information. Note that only health outcome data that are publicly available at the ZIP Code or smaller geographic level are being recommended for inclusion. Currently, data are available by ZIP Code at the NYSDOH web site for asthma hospitalizations and four types of cancer (female breast, lung, prostate, colorectal). It is anticipated that additional information for health outcomes such as low birth weight, infant mortality, and childhood blood lead screening results will become available in the future.

  1. Hospitalization for asthma (ICD-9-CM code 493). Zip Code data from SPARCS.
  2. Cancer incidence for female breast, lung, prostate, colorectal cancer. Zip Code data from the New York State Cancer Registry (1999-2003). .

If appropriate, an applicant may wish to include additional health outcomes based on the specific application or geographic area. A community may be concerned about a particular health outcome for which a reliable source of data meeting the criteria mentioned above does not exist. There is a discussion in Section II.A of the HOD Work Group Report about issues to keep in mind when other sources of data are considered for display.

For What Areas Should Health Outcome Data Be Displayed?

Because the use of comparison areas provides a context for evaluating the data in the COC, health outcome data should be displayed for the COC and for a number of comparison areas. Some comparison areas are relatively large areas, such as a county, and represent some "average" health status. Some comparison areas are smaller and located in fairly close proximity to the COC.

Since the smallest geographic area for which health outcome data are provided is the ZIP Code, for health outcome data displays, the COC must be defined by selecting ZIP Codes that best approximate the COC as delineated by NYSDEC. This can be done by using mapping software to overlay a ZIP Code map on the map of the COC as delineated by NYSDEC. The comparison areas must be selected as ZIP Code areas as well, or as a county or group of counties.

The HOD Work Group Report includes a lengthy discussion of types of comparison areas and includes the following list of criteria for choosing comparison areas. For more information, refer to the report.

  1. Use multiple comparison areas (at least three) to have enough information to evaluate the disease rates in the COC in the context of a number of different settings. Include numbers 2, 3, and 4 below. If the way in which the data are provided has an effect on the comparison areas that can be used, as with ZIP Code level cancer data, show indirect comparisons if possible (see Examples I and II in Section III.B of the Report).
  2. At least one of the comparison areas should be a local area (e.g., within the same city or county) since the community will be more likely to have some knowledge of a local area chosen as a comparison area as opposed to a distant area, including type of land use (e.g., residential, commercial/office, industrial/manufacturing, etc.).
  3. At least one of the comparison areas should include a population within the same general geographic area (e.g., county or contiguous counties) that is similar in population density to the COC. (Population density is discussed in section III.A.4 of the Report.) This comparison would provide information about whether the COC is more severely or less severely affected in terms of the health outcomes displayed than a community that is similar in urban/rural characteristics.
  4. The county in which the COC is located should be one comparison area. (If the COC is located in more than one county, all of the counties should be included.) Another area should be New York State exclusive of New York City or New York City depending on the location of the COC. These are larger areas that will provide a wider perspective.
  5. If specific alternative locations for the facility are being considered, these specific alternative locations should be included as comparison areas.
  6. Other comparison areas may also be included with explanations for why they were selected.

How Should the Health Outcome Data Be Displayed?

This guidance document contains examples of tables displaying health outcome data in Appendix A. In addition, the HOD Work Group Report also contains examples, but with older data. Health-related events are frequently displayed in categories, such as gender, race/ethnicity, and age groups. The appropriate categories for display of a particular outcome depend on the characteristics of that outcome, i.e., whether the outcome is more common in a particular age group, ethnic group, etc. There is not a single set of display categories that is appropriate for all health outcomes. For example, Lyme disease is not generally displayed separately for males and females, but lung cancer is displayed in this way because of interest in gender differences in lung cancer rates.

Tables of health outcome data generally show numbers of health-related events, which can be births, deaths, or cases of a health outcome, and rates based on these numbers. When data are divided into many different categories, the number of events in each cell of the table becomes smaller. In highly populated areas or when the health outcome is relatively common, this may not present a problem. However, if the outcome is rare or the population is small, the number of events in some cells of the table may become so small that rates based on these events may be unstable and fluctuate dramatically from year to year. There is a good discussion of the issue at the Data Source and Tools - Chronic Disease and Conditions on the NYSDOH website (under "Teaching Tools," select "Rates Based on Small Numbers"). A graph is included which shows that with 20 cases, the relative standard error (a measure of statistical uncertainty or random variability around the rate estimate) is 20% and with 10 cases it is 30%. One should be cautious about drawing conclusions from rates based on small numbers of events because, due to the influence of random variability, they may not reliably represent the underlying risk of disease.

These general guidelines should be followed when displaying data:

  • Do not display information in a way that could identify an individual and constitute a breach of confidentiality. A count of 1 or 2 events in a small population could unintentionally disclose confidential information. For example, if there is one person in a small community who is frequently hospitalized, a table that shows one case of a serious illness such as cancer, AIDS/HIV infection, etc., in this community could inadvertently reveal confidential information.
  • Avoid the presentation of rates for cells with 10 or fewer events. Cells with 10 or fewer events should be displayed, but rates based on 10 or fewer cases may be unstable and should be interpreted cautiously. When possible, combine the number of cases or deaths over several years so that the rates are based on a larger number of cases. For example, use three-year or five-year average annual rates instead of single-year rates. It may also be possible to combine the number of cases across geographic areas to obtain a larger number of cases, for example, by combining ZIP Codes or using the rate for a county; however, in the process the ability to characterize the health status of the specific geographic area of interest may be lost. If rates based on 10 or fewer cases are displayed, highlight these rates with a footnote stating that these rates may be unstable.
  • If there are enough cases or deaths that small numbers are not an issue, the data can be displayed in sub-categories as appropriate:
    • If the outcome has differences in rates between males and females, the data should be displayed separately for males and females, e.g., cancer, mortality.
    • If there are known differences in rates by specific age groups, these age groups should be used when displaying the data, e.g., asthma rates are higher in children 0-4 years of age.
    • If reliable data are available by race/ethnicity and there are known differences in rates by race/ethnicity, the data should be displayed by race/ethnicity.
  • The categories used to display data for specific outcomes at the New York State Department of Health web site or the U.S. Centers for Disease Control and Prevention web site may be used as a guide.

What Method Should Be Used to Compare the Data?

A discussion of a number of different ways of comparing health outcome data in the COC with data from the comparison areas is included in the HOD Work Group Report. The report concludes that, for the purpose of comparing disease rates between one area and another, the calculation of rate ratios and confidence intervals is the most informative.

As stated previously, the rate of a health-related event is usually expressed as the number of events in a specified population (e.g., per 10,000 or 100,000 population) during a given time period. When possible, annual rates should be calculated. In order to assess the difference between the rate of the health-related event in the COC and the rate in the comparison area, a rate ratio can be calculated and a confidence interval can be found. The rate ratio is the ratio of the rate in the COC to the rate in the comparison area. When the two rates being compared are the same, then the ratio is equal to 1. When the rate is higher in the COC than in the comparison area, the ratio is greater than 1, and when the rate is lower in the COC than in the comparison area, the ratio is less than 1.

When the two rates are compared, a confidence interval can be calculated to gain a sense of certainty about the estimated difference between the rates. The confidence interval is the range around the ratio in which the true measurement lies with a certain degree of confidence. The confidence interval is a measure of the variability in the data; in this case variability is contributed by the two rates in the ratio. One reason for variability is because there are random fluctuations in the number of cases in an area over time or between different communities. If there were little variability (i.e., the rates were relatively stable), the value of the ratio would be close to the same if the measurement were repeated.

Confidence intervals can be calculated using commercial statistical software or spreadsheet software. Appendix B contains the formulas for calculating confidence intervals and a link to preprogrammed spreadsheets for calculating confidence intervals when working with the asthma hospitalization and cancer incidence data provided at the NYSDOH web site.

A confidence interval that does not include the number 1 provides an indication that the difference between the rates being compared is not likely to be due to the random-like variability mentioned above. If the confidence interval includes the number 1, then the difference in the two rates is likely to be due to random variability. (More information on confidence intervals can be found at the web site Washington State Department of Health - Health Data and NYSDOH's Confidence Intervals - Statistics Teaching Tools website and in Appendix F of the HOD Work Group Report).

The use of rate ratios and confidence intervals is best shown in examples of health outcome data display. Two examples were provided in the HOD Work Group Report, one using a COC in an urban area (Example 1) and the other using a COC in a rural area (Example 2). In the examples, comparison areas for the COC are selected, tables for asthma hospitalizations and two types of cancer in the COC and the comparison areas are developed, and rates of these health outcomes in the COC are compared to those in the comparison areas. Since the Report was issued in 2004, the asthma hospitalization data and cancer data have been updated. In addition, there have been some changes in how the data are displayed on the NYSDOH web site. The examples using the updated data are attached to this guidance document in Appendix A.

What Can We Conclude from the Comparisons? How Can the Results Be Used in the Permitting Process?

In the examples attached to this document, the rates of several health outcomes (asthma hospitalization, female breast cancer, and colorectal cancer) in a COC have been compared to the rates of the same outcomes in a number of comparison areas. When evaluating the results of the analyses, we focus on the rate ratios (greater than 1 or less than 1), the number of comparisons for which rate ratios are greater than 1, and the confidence intervals around the rate ratios (whether or not 1 is excluded). We also look at the number of health outcomes for which we see higher rates in the COC and consider the specific types of health outcomes that show elevations.

The HOD Work Group Report indicates that the health outcome data are to be considered as part of the permitting process, recognizing that the data provide no information about the causes of any increase or decrease in rates between the COC and comparison area populations. The more often the observations fall into the same pattern, the greater the likelihood that the observations suggest a real difference in health status between the COC and comparison area populations. The report states that, if any of the following conditions are met, consideration of additional options for the permitting conditions should be reviewed as part of the permitting process because of the health outcome data displays and comparisons. The greater the number of conditions that are met, the greater the likelihood is that the health status of the COC is actually lower than that found in other areas.

  1. A disease rate is higher in the COC than in any comparison area population for any health outcome;
  2. A disease rate is higher in the COC than in multiple comparison area populations for any health outcome;
  3. The confidence intervals are greater than 1 (A greater elevation in health outcome in the COC compared to the comparison area population and a larger number of events will increase the likelihood that the confidence interval will not include 1.);
  4. There is a pattern of higher rates of multiple health outcomes in the COC; and
  5. Health outcomes that result from an acute exposure (e.g., asthma exacerbations) are elevated rather than those that result from a chronic exposure (e.g., cancer). Health outcomes resulting from an acute exposure may be more relevant to current place of residence than those that result from a chronic exposure. For a chronic effect such as cancer, a crucial exposure or risk factor may have occurred decades earlier when the individual resided at a different location.

The asthma hospitalization data in Example 1 show a case where the pattern of observations gives greater confidence in the results. The last row in each part of Table II provides information for the total population (0-65+ years); each rate ratio is greater than 1 and all of the confidence intervals exclude 1 and are in bold type, indicating that for every comparison the asthma hospitalization rate is higher in the COC and the difference in rates is not likely due to random variation. A similar result is seen for almost every age group comparison. Because of similar results with multiple comparisons, there is considerable confidence that the asthma hospitalization rate in the COC is elevated.

The cancer results in Table III for the same COC show a different picture. The number of breast cancer cases in females in the COC was lower than expected based on the rate in New York State, and the number of colorectal cancer cases in males and females in the COC was similar to that expected.

Example 1 in this document meets four of the conditions listed above: number 1 is true because the asthma hospitalization results in Table II show at least one rate ratio that is greater than 1; number 2 is true because the rate ratios in Table II are greater than 1 for multiple comparisons; number 3 is true because multiple confidence intervals in Table II exclude 1 and are in bold type; number 4 is not true because only one outcome (asthma hospitalization) shows elevations; number 5 is true because asthma hospitalization is an outcome that results from acute, rather than chronic, exposure. In the case of the COC in Example 1, additional options for permitting conditions would be considered.

The COC in Example 2 is in a sparsely populated rural area. There were only 12 hospitalizations for asthma in three years for this study area. Because of the sparse population and the resulting small numbers of health outcomes, health outcome rates are likely to fluctuate considerably from year to year. The asthma hospitalization rate in the COC is compared with that in three comparison areas in Table V. The rate ratios are all less than 1, indicating that the asthma hospitalization rate is lower than that in each of the comparison areas. Table VI shows the results for breast cancer in females and colorectal cancer in males and females. The number of breast cancer cases observed in females was lower than expected based on the rate in New York State. The observed and expected numbers of colorectal cancer in males and in females are very small (fewer than 10). Rates based on such small numbers of cases can change dramatically from year to year with a change of only one or two cases; therefore, the rates are considered too unstable for analysis. We cannot conclude that the rates of female breast and male and female colorectal cancer are different from those in New York State. None of the conditions listed above have been met, and the results of the health outcome data displays would not result in a recommendation for the evaluation of more stringent permit conditions, voluntary pollution reduction, or other corrective measures for this COC.

The information on the health status of the community provides a more complete picture of the area under study and may suggest the need for action, including more stringent permit conditions, voluntary pollution reduction, or other corrective measures. Therefore, the health outcome data displays and comparisons should be used in making a permitting decision along with other considerations such as regulatory standards, environmental impacts, mitigation, benefits, needs, and costs. The significance of the difference between the COC and the comparison area populations should be considered in determining which action is appropriate. Possible actions may include, but are not limited to:

  • Considering alternative siting locations, especially outside of the COC;
  • Applicant evaluation and implementation of pollution prevention options, such as chemical substitution; changes in work practice standards, such as evaluating ways to reduce fugitive emissions; emission reductions by the installation of best available control technology to achieve the lowest achievable emission rates (LAER) possible; the implementation of holistic environmental management system; or the purchase of emission reduction credits.
  • Providing assistance beyond the permit review process by using agency regulatory authority to evaluate the feasibility of reducing existing exposures from other sources that may be contributing to a health outcome of concern, such as obtaining offsets from other sources of air pollution in the COC;
  • Providing assistance that may be beyond the permit review process by using agency regulatory authority to prevent, diagnose, monitor or treat the health outcome of concern or improve health status of the COC; and
  • Taking no action based on the results of the health outcome comparisons.

Outside of these recommendations, there are state and Federal environmental justice grant programs for community groups that could enter into partnerships with government, business, and the academic sector to work on projects to address local environmental and/or public health concerns (see NYSDEC environmental justice grants and EPA environmental justice grants). Regardless of which options are considered, the local and state health departments may be able to identify services that might improve the health status of the COC.