Over the last three decades, several authorities made recommendations for improving transparency in the use of science[1] in the administrative process.[2] Partially in response to these recommendations, the executive branch and Congress have made a number of reforms to the scientific process undergirding agency decisionmaking. In 2009, President Obama issued a memorandum directing that, “[t]o the extent permitted by law, there should be transparency in the preparation, identification, and use of scientific and technological information in policymaking.”[3] “Each agency should [also] have appropriate rules and procedures to ensure the integrity of the scientific process within the agency.”[4] The Office of Science and Technology Policy (OSTP) elaborated upon this memorandum in 2010, instructing agencies to “communicate scientific and technological findings by including a clear explication of underlying assumptions; accurate contextualization of uncertainties; and a description of the probabilities associated with both optimistic and pessimistic projections.”[5]
At base, these initiatives demand heightened transparency of agencies’ use of science as a central means of ensuring the basic accountability of agency regulation. If an agency identifies the role that scientific information plays in its ultimate decision and explains how it ensured that its scientific analysis was rigorous, then the public has a basis against which it can evaluate both the scientific and policy judgments underlying the agency’s decision. This transparency allows those outside the agency to assess whether the agency’s policy decision comports with the authorizing law and the scientific record. A transparent decisionmaking process also advances other institutional and scientific goals, such as identifying promising areas for future research and serving as a bulwark against misuse of science for political ends.[6]
Despite these important initiatives, a study commissioned by the Administrative Conference[7] (and public meetings that considered questions it raised) revealed that agency decisionmaking processes would benefit from further improvements. Drawing on this learning, the recommendation offers several proposals for enhancing the transparency of agencies’ use of science. At the same time, the Conference recognizes that agencies’ abilities to implement this recommendation may be affected by resource limitations.
First, the recommendation highlights a number of innovative practices undertaken by different federal agencies to enhance the transparency of their scientific decisionmaking processes. As a general matter, agencies should articulate the specific questions to be informed by scientific information, specify study designs for new research, and establish criteria for weighing existing studies.[8] Agencies should identify scientific reports or data upon which they relied and material literature that they considered, but upon which they did not rely, to the extent practicable and permitted by law.[9] Agencies should establish checkpoints (i.e., times for closing off consideration of additional research or debate prior to making a final regulatory decision) and policies for reopening that consideration. Agencies should also consider extending attribution to individual staff who participate in the preparation of scientific reports and taking other steps to promote robust debate among agency scientists.[10] In addition, agencies should share best practices with other agencies and should recommend the removal of any legal impediments to promoting transparency in decisions in which science is an important element.[11]
Second, the recommendation offers a series of proposals to bring greater congruity to the treatment of publicly and privately funded scientific research. Specifically, it encourages the disclosure of data underlying scientific research, including both privately funded and federally funded research, that an agency is considering (to the extent practicable and permitted by law).[12] Similarly, it recommends extending conflict of interest disclosure norms to private parties who submit studies used by an agency.
RECOMMENDATION
Suggested Agency Practices Regarding the Use of Science in the Administrative Process
1. Explaining Agency Scientific Decisionmaking. Agencies should explain in proposed and final decision documents how they ensured rigorous review of the scientific information underlying each science-intensive regulatory project. This includes a statement of how each agency evaluated the scientific information used in its analysis; how the agency made that information available to reviewers and the public; how the analysis was reviewed by experts and interested parties; and how the agency ensured that the final decision was supported by the scientific record.
2. Assuring Transparent Assessments. At an early stage in their decisionmaking processes, agencies should identify the specific policy questions that may be informed by science; describe the design of the assessments needed to characterize risks and inform policy decisions; and describe the criteria to be used in reviewing and weighing existing studies. When completed, assessments should: identify other appropriate analytical choices and explain why they were not chosen; provide a synthesis of the available evidence and relevant literature guided by the assessment design or criteria; identify significant assumptions and choices of analytical techniques; provide a statement of remaining uncertainties; and discuss how different plausible choices might change the results of the assessment. Where possible, agencies should also explain the relationship between their scientific findings and the final policy choice. Agencies should strive to communicate this information in a manner that is clear to the general public.
3. Disclosing Underlying Studies and Data. To the extent practicable and permitted by law and applicable policies, each agency should identify and make publicly available (on the agency website or some other widely available forum) references to the scientific literature, underlying data, models, and research results that it considered. In so doing, the agency should list all information upon which it relied in reaching its conclusions, as well as any information material to the scientific analysis that it considered but upon which it ultimately did not rely. Consistent with the limitations in the Information Quality Act (IQA) guidelines issued by the Office of Management and Budget and its own IQA guidelines, each agency should ensure that members of the public have access to the information necessary to reproduce or assess the agency’s technical or scientific conclusions.
4. Checkpoints and Explanations. Agencies should consider establishing explicit checkpoints for regulatory projects, defining both the conditions under which they intend to close their consideration of research or debate in order to reach a decision and when they might reopen that consideration, particularly in cases when they are not bound by judicially enforceable deadlines. In any case, agencies should explain their decisions to initiate, stop, or reopen consideration of research or debate. Such explanations should reference significant relevant ongoing research or other relevant factors.
5. Identifying Future Projects. For science-intensive projects, agencies should identify specific types of future research that may be needed to reduce significant uncertainties in order to advance understanding of the issues.
6. Attribution for Agency Personnel. Agency personnel play an important role in producing their respective agencies’ scientific analyses. Agencies should consider providing their personnel with some form of consensual attribution for reports or analyses to which they contribute in a significant way. If appropriate, such attributions should be made for personnel who contributed in a significant way to a technical or scientific report, including not only scientists but also economists, lawyers, and other contributors. Reviewers and other contributors could be identified by name and general contribution.
7. Encouraging Debate. Agencies should encourage vigorous debate among agency scientists and should explore ways of incorporating the diversity of that debate in any resulting work product. Agency employees should be encouraged to publish their scientific work in the peer reviewed literature, provided that they follow applicable agency procedures and that confidential governmental deliberations are not compromised. Dissenting staff members should be protected from reprisals.
8. Sharing of Agency Best Practices. Agencies should identify and publicize the innovations they have developed for transparently incorporating science into their regulatory decisions. OSTP, an interagency group headed by OSTP, or another body should consider occasionally convening agency representatives to discuss and share best practices.
9. Addressing Legal Obstacles to Transparent Decisionmaking. Agencies should identify legal obstacles that may impede otherwise appropriate public access to the scientific information underlying agency analyses or that may prevent the agencies’ development of scientifically robust decisionmaking processes. Agencies should recommend appropriate actions to eliminate such impediments, including revisions in existing law, to the Executive Office of the President.
Agency Disclosures to Enhance the Transparency of Research
10. Data Disclosure. To the extent practicable and in compliance with applicable legal restrictions, privileges, protections, and authorities, agencies should seek to provide disclosure of data underlying scientific research, including both privately and federally funded research being considered by the agencies. Where practicable, such information should be disclosed in machine-readable format. Where such data are not subject to legal or other protections, and the data’s owners nonetheless will not provide such access, agencies should note that fact and explain why they used the results if they chose to do so. Agencies should review their confidential business information policies to ensure that they include appropriate mechanisms to prevent over-claiming.
11. Conflict of Interest Disclosure. Agencies should require conflict of interest disclosures on all scientific research submitted to inform an agency’s licensing, regulatory, or other decisionmaking processes. This disclosure should be similar to the conflict of interest disclosure required by some scientific journals, such as that used by the International Committee of Medical Journal Editors. The regulatory conflict of interest disclosure should also, where permitted by law, identify whether the experimenter or author had the legal right without approval of the sponsor of the research to: design the research; collect the data; interpret the data; and author, publish or otherwise disseminate the resulting report or full dataset. To the extent that a party other than the principal investigator (e.g., the study sponsor or funder) had control over the design or publication of the study, agencies should disclose this fact and specify the nature of the control such an entity exercised.
[1] The scope of this recommendation is limited to the “natural sciences” (e.g., chemistry, physics, medical science, geology, etc.), mathematics, statistics, computer science, and other allied fields. It is based upon a report that deals with agency research and decisionmaking related to the natural sciences. Wendy Wagner, Science in Regulation: A Study of Agency Decisionmaking Approaches (Feb. 18, 2013), available at http://www.acus.gov/sites/default/files/documents/Science%20in%20Regulation_Final%20Report_2_18_13_0.pdf.
[2] See e.g. Nat’l Research Council, Review of the Environmental Protection Agency’s Draft IRIS Assessment of Formaldehyde (2011); Comm. on Risk Assessment of Hazardous Air Pollutants, Nat’l Research Council, Science and Judgment in Risk Assessment (1994); Nat’l Research Council, Risk Assessment in the Federal Government: Managing the Process (1983); Bipartisan Policy Ctr., Improving the Use of Science in Regulatory Policy 16, 41-42 (2009) [hereinafter “BPC Report”]; see also Ctr. for Effective Gov’t, Advancing the Public Interest through Regulatory Reform: Recommendations for President-Elect Obama and the 111th Congress 26, 34, 47 (2008).
[3] Memorandum from the Admin. of Barack H. Obama for the Heads of Executive Departments and Agencies on Scientific Integrity, Daily Comp. Pres. Docs., 2009 DCPD No. 00137 (Mar. 9, 2009) [hereinafter “Obama Scientific Integrity Memo”], available at http://www.gpo.gov/fdsys/pkg/DCPD-200900137/pdf/DCPD-200900137.pdf.
[4] Id.
[5] Memorandum from John P. Holdren, Director of the Office of Science and Technology Policy, to the Heads of Executive Departments and Agencies on Scientific Integrity (Dec. 17, 2010), available at http://www.whitehouse. gov/sites/default/files/microsites/ostp/scientific-integrity-memo-12172010.pdf. To effectuate this and a number of other responsibilities, agencies were asked to report back to OSTP on the actions taken to develop and implement their scientific integrity policies by April 2011.
[6] BPC Report, supra note 2, at 3.
[8] In so doing, agencies should endeavor to explain the relationship between scientific research and the policy decisions the research is intended to inform. Nat’l Research Council, Comm. on the Institutional Means for Assessment of Risks to Public Health, Risk Assessment in the Federal Government: Managing the Process 7 (1983).
[9] See Administrative Conference of the United States, Recommendation 2011-1, Legal Considerations in E-Rulemaking, ¶ 4, 76 Fed. Reg. 48,789, 48,789 (Aug. 9, 2011); see also Exec. Order. No. 13,642, Making Open and Machine Readable the New Default for Government Information, 78 Fed. Reg. 28,111 (May 14, 2013); Memorandum from John P. Holdren, Director of the Office of Science and Technology Policy, to the Heads of Executive Departments and Agencies on Increasing Access to the Results of Federally Funded Research (Feb. 22, 2013) (calling for agency plans to permit public access to research papers funded in whole or in part with federal monies). As a general matter, the agency should make publicly available any scientific literature it considered, including literature it reviewed but upon which it ultimately did not rely. For purposes of the recommendation, literature that an agency “considered” includes not only any study an agency official relied upon but also any study an agency official reviewed but ultimately determined not to rely upon (because it was deemed to be outside the scope of the scientific study at hand, was not considered sufficiently reliable, or was otherwise rejected by the agency official). Cf. Administrative Conference of the United States, Recommendation 2013-4, The Administrative Record in Informal Rulemaking, __ Fed. Reg. __ (providing a similar definition of “consider” in the context of the administrative record in informal rulemaking). If an agency official merely had access to a study but did not specifically analyze it to determine its relevance, that study has not been “considered” within the meaning of the recommendation for purposes of making such literature publicly available.
[10] In response to President Obama’s call for agencies to develop “appropriate rules and procedures to ensure the integrity of the scientific process,” Obama Scientific Integrity Memo, supra note 3, a number of agencies have promulgated integrity policies to promote open debate among agency scientists. See, e.g., Envtl. Prot. Agency, Scientific Integrity Policy (Feb. 2012), available at http://epa.gov/osa/pdfs/ epa_scientific_integrity_policy_20120115.pdf; Food and Drug Admin., Scientific Integrity at FDA, FDA Staff Manual Guides, Volume IV-Agency Program Directives 2 (2012), available at http://www.fda.gov/ ScienceResearch/AboutScienceResearchatFDA/ucm306446.htm; Nat’l Oceanic and Atmospheric Admin., Scientific Integrity (Dec. 7, 2011), available at http://www.corporateservices.noaa.gov/ames/administrative_orders/ chapter_202/202-735-D.pdf; Nuclear Regulatory Comm’n, Collaborative Work Environment Program, http://www.nrc.gov/about-nrc/values.html#open (last updated May 4, 2012); see also Francesca T. Grifo, Federal Agency Scientific Integrity Policies: A Comparative Analysis (Mar. 2013), http://www.ucsusa.org/ assets/documents/scientific_integrity/SI-policies-comparative-analysis.pdf.
[11] See Wagner, supra note 1, at 135–38 (identifying a number of external legal impediments to promoting transparency, including short statutory deadlines, limits on dissemination of scientific studies, resource limitations, and caps on the number of discretionary advisory committees agencies can constitute).
[12] Legal restrictions that may limit agencies’ ability to provide such disclosures include, among other things, protections for personal privacy, trade secrets, and confidential business information.