| Database Support for Public Policy Paper Location | |
| Qiana Johnson | |
| Qiana Johnson (q-johnson@northwestern.edu) is Schaffner and Distance Learning Librarian, Schaffner Library, Northwestern University Libraries, Chicago | |
| Abstract | This article analyzes the inclusion of public policy papers in three resources—Policy File, Policy Archives, and PAIS International. The databases were analyzed by searching for known policy papers across the three resources, analyzing the subject coverage of the policy papers contained in the databases, and the inclusion of particular policy institutes. PAIS International contained more policy papers based on subject and included more of the selected policy institutes. Policy File contained more of the preselected policy papers. |
Of the myriad resource types needed by public policy students and researchers, policy papers can be the most difficult to locate because of the various sources in which they can be found. Policy papers or policy research can be beneficial to government officials as they create new laws or those looking at the impact of a policy. Jay M. Shafritz defines a policy paper as “a formal written argument in favor of (or opposing) a particular public policy.”1 Gambhir Bhatta defines policy research as “a term that refers to work done on the relationships and interactions among several variables that reflect social problems and more importantly that can be manipulated by public policy.”2 Access to policy papers is often provided through licensed databases such as PAIS International and Policy File as well as through freely available web sources such as Google and Policy Archive. As researchers look to select tools to locate this resource type, analysis needs to be done to determine the best tools. This article will analyze the content of three databases for known policy papers, policy papers on particular topics, and the presence of particular think tanks’ policy papers.
Much of the research about the information needs in public policy focuses on researchers’ information-seeking behavior and resources for locating articles. James Church noted the need for libraries to collect materials published by non-governmental organizations.3 While library government documents departments routinely collect materials from US and international governments, the same collection development effort is often not happening for non-governmental organization documents. While noting the need to collect these types of materials, Church also recognizes the challenge in doing so. The question then arises that if these materials are difficult to collect, they are probably equally difficult for users to locate. In her article, Elizabeth Tompkins presented a “selected, annotated bibliography” on the topic of think tanks and public policy research institutes.4 The books and articles highlighted focus on the role of think tanks in society. Research has also been conducted on the most useful periodical databases in this area. James Cory Tucker analyzed the database support for public administration related areas, including public policy.5 Tucker analyzed the journal coverage in six databases—ABI/INFORM, Business Source Premier, General BusinessFile ASAP, International Academic Research Library, Academic Search Premier, and Expanded Academic Index ASAP. In the area of public policy, Tucker found that Expanded Academic Index, Academic Research Library, Academic Search Premier, and Business Source Premier had similar levels of journal coverage.
Many of the annotated bibliographies on public policy and the policy sciences are older and date to the late 1970s and early 1980s. Only a few of these specifically address sources for locating policy papers; regardless, the tools available for locating and accessing these materials have changed in the intervening years. A search in WorldCat for the subject heading “policy sciences bibliography” returns 248 items. From that list, the most recent bibliographies broadly covering public policy were published in the mid-1990s: Pamela Newell Paonessa’s 1994 Public & Social Policy: Subject Analysis with Reference Bibliography and Antony E. Simpson’s 1993 Information-Finding and the Research Process: A Guide to Sources and Methods for Public Administration and the Policy Sciences. Other bibliographies were published based on the holdings of a particular library or think tank or a narrow area within public policy. In their guide, Murin, Greenfield, and Buenker included citations to resources in a number of different policy areas, including urban policy, education policy, and housing, as well as citations to some individual policy papers.6 Given the nature of the bibliography, it was not meant to be a comprehensive list of resources—articles, books, or policy papers. This guide also does not list tools for locating policy papers. In his annotated bibliography, Simpson does list some sources for locating policy papers and the work of individual think tanks such as Selected Rand Abstracts and the NORC Report.7 The author also lists PAIS International in Print, which is now available electronically.
Faye Miller conducted a small study of the information-seeking behavior of public-policy oriented researchers at Charles Sturt University. Miller interviewed three researchers working in various public policy arenas. She identified five information needs of this group:
- They need to understand an issue from the multiple perspectives of many stakeholders in academic public policy research and to be sufficiently informed to communicate and debate in academic and nonacademic context. Thus, information that is critical, balanced, and nonpartisan is essential.
- They need access to electronic current awareness information services to ensure the continuing relevance of their research because research questions change over time in response to current changes, issues and priorities in public policy development.
- They need to know where to look for research funding opportunities that match their research questions and interests.
- They need personal interaction (face-to-face interviews, informal discussion) with academic colleagues, policy makers, other stakeholders, and users of research, to develop research questions and increase the potential affect of their research.
- Once they have formulated a broad question, they need to define the research problem and specific aspects of interest in relation to public policy development.8
The first two identified needs lend themselves to assistance from libraries, especially the need for “critical, balanced and non-partisan” information. Periodical literature can help a great deal in this area as well as policy papers from a number of sources. Finally, in their article, Stinson and Stewart analyzed public policy reference questions at two universities in North Carolina.9 The authors examined the number of questions received in this area as well as the resources used to answer the questions. Librarians used periodical and newspaper indexes to answer many of the questions. Librarians also referred to reference titles, especially those relating to the US government. No mention was given to assisting students in locating policy papers or resources that might have been used.
Why is it important to know where researchers can locate policy research? What is the value of think tanks and by extension their research? James G. McGann writes that think tanks “provide public policy research, analysis, advice, and operate independently from governments and political parties.”10 Think tanks also have roles in the following:
- Mediating between the government and the public
- Building confidence in public institutions
- Serving as an informed and independent voice in policy debates
- Identifying, articulating, and evaluating policy issues, proposals, and programs
- Transforming ideas and emerging problems into policy issues
- Interpreting issues, events, and policies for the electronic and print media, thus facilitating public understanding of domestic and international policy issues
- Providing a constructive forum for the exchange of ideas and information between key stakeholders in the policy formulation process
- Facilitating the construction of “issues networks”
- Providing a supply of informed personnel for the legislative and executive branches of government
- Challenging the conventional wisdom, standard operating procedures, and “business as usual” of bureaucrats and elected officials11
There are a number of methods for evaluating the influence of a think tank and its research. In some cases policy institutes were created in response to politics. Bertelli and Wenger studied the creation of think tanks and their relationship to legislative debates.12 By analyzing the year of formation, house polarization, S&P 5002-year return rate, top marginal estate tax rate, estate tax exemptions, percentage of freshman in the House, whether there were mid-term elections, years after the Legislative Reform Act, years after the House Committee Reforms, and the gross receipts of think tanks, the authors found that the number of think tanks formed increased with the polarization in the House. This increase in the number of think tanks is due to the increased need for research-based information. In their article, Rich and Weaver analyzed the media visibility of think tanks. The authors analyzed fifty-one policy institutes as related to their coverage in the Christian Science Monitor, the Wall Street Journal, the Washington Post, the Washington Times, and USA Today. Rich and Weaver also noted what percentage of all publicity was received by each think tank as well as the percentage of think tanks cited being nationally focused Washington, DC-based institutions, nationally focused and non-DC based institutions, and state/regionally focused institutions. The authors noted a strong connection between a think tank’s funding and its media visibility. Personal networks and editorial decision-making also influence media visibility:
This is most apparent in the geographic biases associated with media visibility: Washington-based think tanks remain the overwhelmingly dominant players relative to think tanks based outside the Beltway, not just as a function of their size, but also as a function of proximity. Their location promotes the development of personal relationships and networks among social scientists, journalists, and policymakers’ staff, relationships perhaps more regularly and aggressively sought by think tanks based in Washington than those based elsewhere.13
This visibility can have direct or indirect effect on the policy-making process. In his dissertation, Andrew Rich examined how the expertise of think tanks is perceived based on the number of newspaper citations and appearances before Congress. Rich found that the use of think tank expertise and the number of newspaper citations was affected by the think tanks’ marketing strategy and the think tanks’ goals. He wrote, “The greater frequency with which marketing-oriented think tanks testify with other marketing-oriented think tanks and with interest groups suggests that members of Congress and their staffs may recognize differences in the contributions of marketing-oriented think tanks in comparison to other types of think tanks.” He goes on to contrast nonmarketing oriented think tanks saying, “The high frequency with which non-marketing oriented think tanks are paired on congressional panels with university and government officials begins to suggest that the expertise of these organizations may be viewed as more compatible with that of these officials and generally more balanced, authoritative, and credible than that of marketing-oriented think tanks.”14 Nonmarketing oriented think tanks are also more frequently cited in newspapers. Think tanks and the research they produce have a strong influence on the policy-making process. However, as noted in the research, nationally focused, Washington, DC-based institutes have a greater influence than other institutes. In the absence of building relationships, policy makers and journalists can learn about think tank research through search tools.
Research began by compiling a list of 360 policy papers from 18 public policy think tanks—20 policy papers from each of the 18 think tanks (table 1). According to the Think Tank & Civil Societies Program 2011 listing of “The Leading Public Policy Research Organizations in the World,” there are 1,815 US-based think tanks. The chosen think tanks represented a small sampling (1 percent) of those organizations. The Think Tank and Civil Societies Program also identified 4,730 non-US-based think tanks. Because of the number and diversity of types of non-US-based think tanks, this paper focuses only on US think tanks to narrow the research scope. To address issues of media visibility discussed by Rich and Weaver, the majority of think tanks selected by the researcher were nationally focused, Washington, DC-based organizations. Policy papers produced by these think tanks are more likely to be seen by journalists and the public. Because of this, they might also have a better chance of being included in databases. To include a variety of types of think tanks, slightly more than 20 percent of the selected think tanks were university based. The researcher selected another 22 percent to be based outside of Washington, DC. The organizations ranged from large groups such as the Brookings Institute and the Pew Charitable Trusts to smaller university-based policy groups such as the Center for Public Policy and Administration and the Public Policy Research Center. All of the chosen centers were based in the United States and included a mixture of neutral and ideological organizations. Chosen papers came from across the research areas of each think tank. Papers also included some of the earliest research provided on the organizations’ websites. The centers’ websites cited or provided full text to the chosen policy papers. The earliest paper chosen for this project came from 1977 and the latest paper came from December 2011. The selected papers came from a number of subjects commonly researched in public policy as well as some newer topics and included employment, economic development, Internet policy, housing, education, welfare, and immigration. Papers fit the definition of a policy paper because they sought to analyze the effect of a particular public policy or sought to make recommendations for policy. The researcher searched for these 360 papers by title in Policy File; Policy Archive, a free search tool produced by the Center for Government Studies and Indiana University-Purdue University Indianapolis; and PAIS International. Google can be used to locate policy papers, and as with Policy Archive, has the benefit of being freely available. However, search difficulties arise because of the volume of web pages Google indexes and the lack of controlled vocabulary. For this reason, Google was not included as a studied search tool.
The researcher also searched the three resources for coverage of specific topics to analyze the depth of their content. Coverage of particular subjects was ascertained by searching for the topics using the subject terms or other controlled vocabulary where possible. Not all topics in every resource had a corresponding subject term. In these cases, the researcher conducted keyword searches. No date or other restrictions were put into place for these searches. Finally, building a list based on think tanks included on the National Institute for Research Advancement’s (NIRA) World Directory of Think Tanks and the Think Tank and Civil Societies Program 2011 listing of “The Leading Public Policy Research Organizations in the World,” the researcher analyzed the databases for their inclusion of a number of major US policy institutes.
The researcher analyzed the number of prominent policy institutes included in each of the three resources by putting together a list of policy institutes based on their presence on two lists—the National Institute for Research Advancement’s (NIRA) World Directory of Think Tanks and the Think Tanks and Civil Societies Program 2011 listing of “The Leading Public Policy Research Organizations in the World” (Go-To). Only US think tanks were chosen and needed to be included on at least one of the lists. Using these criteria, the list came to 112 policy institutes. Of that list, 21 (19 percent) were only on the Go-To list, 63 (56 percent) were only on the NIRA list, and 28 (25 percent) were on both lists (table 2).
To determine whether a policy institute’s papers were included, the researcher searched the list of included organizations provided by the databases, where possible, or searched for the name of the think tank as a publication title, author, or publisher. This last method of searching was conducted to locate materials in PAIS International.
Here, PAIS International had the most comprehensive coverage. Of the 112 think tanks searched for, PAIS included papers from 88 of them. Policy File included 68 of the think tanks and Policy Archive included 36. Of the policy institutes that were not included in PAIS, they were often the institutes that focused on US-Asian relations such as the Maureen and Mike Mansfield Foundation and more regional concerns such as the Cascade Policy Institute. While PAIS contained papers from the largest number of institutes, there were 11 institutes (12.5 percent of the 88 institutes included) for which PAIS had fewer than ten papers. Policy File included fewer institutes overall, and the areas focused on by those excluded institutes were broader than PAIS and included general policy issues and economic issues. Of the policy institutes included in Policy File, 6 organizations (9 percent of the 68 institutes included) had ten or fewer papers included. For Policy Archive, of the included think tanks, 13 institutes (36 percent of the 36 institutes included) had ten or fewer papers included. PAIS International most closely matched the percentage of where the institutes were listed. For PAIS, 18 (21 percent) were listed on the Go-To list; 42 (48 percent) of the think tanks were listed on the NIRA list, and 28 (32 percent) were included on both lists. Policy File included 11 (16 percent) on the Go-To list, 32 (47 percent) think tanks listed on the NIRA list, and 25 (37 percent) included on both. For the think tanks included in Policy Archive, 9 (25 percent) were listed on Go-To list; 10 (28 percent) were included in the NIRA list, and 17 (47 percent) were included on both lists (table 3).
PAIS International was also stronger when searching for papers based on subjects covered in the chosen papers. In going through the 360 papers, the researcher identified 17 subjects that were covered (table 4).
The researcher examined the three databases for policy papers on these 17 subjects using controlled vocabulary where possible. PAIS International had the most papers at 69,376, followed by Policy File at 44,385, and Policy Archive at 9,268. However, these numbers are potentially misleading. Because policy papers in PAIS were identified by limiting to the resource type “books,” some of the items returned as policy papers were citations to books and not policy papers. By examining a small percentage of the most relevant results returned, as determined by the database, the percentage of books versus policy papers ranges from 6 to 51 percent with an average of 27 percent. The sample included the first 1 percent of relevant articles for searches that returned more than 1,000 results. For searches that returned fewer than 1,000 results, the first 10 percent of results were examined. Even factoring in this ambiguity, PAIS International still led in its inclusion of policy papers. Some of PAIS International’s strength may be explained by its international focus. However, as explained earlier, it was also stronger in coverage of US-based think tanks. Policy File may have included more policy papers, but the interface capped results at 5,000 per search. All of the resources were strong in the more common areas of public policy such as employment, housing, and welfare. While there were fewer materials, each of the three resources also had papers on emerging topics such as Internet policy and environmental policy.
Using the 360 selected policy papers, Policy File had the largest number of papers at 151, and Policy Archive had the fewest at 22. Of the think tanks chosen, both Policy File and PAIS International included an equal number of think tanks at 15. Policy Archive included just 5 of the 18 selected think tanks. Of the 15 think tanks included, there was heavy overlap between Policy File and PAIS: 14 of the 15 included think tanks were in both Policy File and PAIS. In both cases, the policy papers from the Center for Public Policy and Administration and the Public Policy Research Center—two of the four public policy institutions based at universities—were not included. The difference between the two databases was that Policy File included some papers from the Heritage Foundation while PAIS did not. PAIS included papers from the Milken Institute whereas Policy File did not. While PAIS included the same number of think tanks as Policy File, the number of papers included from those institutions was fewer in the case of PAIS, as evidenced by the final totals. Only in the cases of the Pew Charitable Trusts, the Henry L. Stimson Center, and Human Rights Watch did PAIS cite more of the chosen papers (table 5).
None of the databases examined included all the known papers or all the think tanks. However, each resource had strengths. Of the preselected papers, Policy File contained more of the papers and 15 of the 18 policy institutes the papers came from. PAIS International had fewer of the chosen papers, but an equal number of included policy institutes at 15. Policy Archive had the least number of chosen papers and included the fewest think tanks from the chosen list. However, in searches based on topic, PAIS had more citations to policy papers with a total of 69,376 for the subjects searched. For eight of the subjects, Policy File included more citations to policy papers than did PAIS International. Another strength of PAIS is the depth of its controlled vocabulary. Only three of the topics searched did not have corresponding subject terms. Policy File had fewer papers than PAIS at 44,385. Also, six of the 17 topics did not have corresponding subject terms. Policy Archive only had 13 percent of the number of papers included in PAIS. However, only five of the topics chosen did not have corresponding subject terms. The number of policy papers included in PAIS International may be explained by the number of the think thanks included from the NIRA and Go-To lists. Because Policy File had stronger subject coverage in eight areas, it makes for a strong supplement. Of these three resources examined, PAIS International had a broad reach and the most content.
Each of these resources has room to grow. The coverage of university-based think tanks was spotty. Because these institutes often focus on local and regional issues, excellent materials are not as easily located using these resources. As a freely available resource, Policy Archive has great potential to bring policy research to those who do not have access to the paid databases. Policy Archive has a wide range of policy papers, but not the number that is included in either PAIS or Policy File. An area for future research is the inclusion of policy papers from think tanks outside of the United States in these or other resources.
References
| 1. | Jay M Shafritz, Dictionary of Public Policy and Administration (Boulder, CO: Westview, 2004): 222. |
| 2. | Gambhir Bhatta, International Dictionary of Public Management and Governance (Armonk, NY: M.E. Sharpe, 2005): 465. |
| 3. | James Church, "“International Documents Round Up: The NGO Documents Task Force, ”," DttP: A Quarterly Journal of Government Information Practice & Perspective (2007) 35, no. 3: 10–13. |
| 4. | Elizabeth K Tomkins, "“Think Tanks and Public Policy Research Institutes: An Annotated Bibliography, ”," Behavioral & Social Sciences Librarian (2007) 26, no. 2: 11–27. |
| 5. | James Cory Tucker, "“Database Support for Research in Public Administration, ”," Behavioral & Social Sciences Librarian (2005) 24, no. 1: 47–60. |
| 6. | Murin W J, "“Public policy: A Guide to Information Sources, ” in American Government and History Information Guide Series," (Detroit: Gale, 1981): . |
| 7. | Antony E Simpson, "Information-Finding and the Research Process: A Guide to Sources and Methods for Public Administration and the Policy Sciences," (New York: Center for Productive Public Management, 1993): . |
| 8. | Faye Miller, "“Research Information Needs of Public Policy Oriented Researchers at a Regional University: Issues Emerging from a Pilot Study, ”," Australian Academic & Research Libraries (2008) 39, no. 4: 257. |
| 9. | Stinson M BStinsonM B , Stewart A W, "“A Comparison of Public Policy Reference Inquiries in Two Academic Libraries, ”," Reference Librarian (1987) 8, no. 20: 171–85. |
| 10. | James G McGann, "Think Tanks and Policy Advice in the United States Academics, Advisors and Advocates, Routlege Research in American Politics 1," (Abingdon, England; New York: Routledge, 2007): 5. |
| 11. | Ibid, 5–6 |
| 12. | Anthony M. Bertelli and Jeffrey B Wenger, "“Demanding Information: Think Tanks and the US Congress, ”," British Journal of Political Science (2009) 39, no. 2: 225–42. |
| 13. | Andrew Rich and R Kent Weaver, "“Think Tanks in the U.S. Media, ”," Harvard International Journal of Press/Politics (2000) 5, no. 4: 99. |
| 14. | Andrew Rich, "“Think Tanks, Public Policy, and the Politics of Expertise”," (PhD diss., Yale University, 1999): 161-62. |
Tables
URLs for Policy Institute Papers
Number of Think Tanks Appearing on the NIRA Directory, the Go-To List, or Both
| The NIRA Directory | The Go-To List | Both Lists | |
| Number of think tanks from: | 63 | 21 | 28 |
| Percentage of the total list from: | 56% | 19% | 25% |
Number of Compiled Think Tanks Appearing in Each Database
| Resources | # of Think Tanks from NIRA | # of Think Tanks from Go-To | # of Think Tanks on Both Lists | % from NIRA | % from Go-To | % on both |
| Policy File | 32 | 11 | 25 | 47 | 16 | 37 |
| Policy Archive | 10 | 9 | 17 | 28 | 25 | 47 |
| PAIS International | 42 | 18 | 28 | 48 | 21 | 32 |
Number of Policy Papers on a Given Subject Included in a Database
| Search Terms | # of Policy Papers in Policy File | # of Policy Papers in Policy Archive | # of Results in PAIS International | Estimated % of Results in PAIS International that are Books |
| Employment/unemployment | 5000 | 276 | 15,215 | 17% |
| E-government/government online/electronic government | 64* | 13* | 168* | 6.25%** |
| Economic development | 1,833* | 665 | 9,468 | 56% |
| Internet policy | 14* | 7* | 2* | 50%** |
| Health policy | 5,000 | 5,099 | 5,226 | 6% |
| Urban policy | 66* | 59 | 292 | 38%** |
| Corrections/policing | 3,596 | 80 | 685 | 15%** |
| Child welfare | 5000* | 103* | 1,685 | 31.25% |
| Housing | 3,061 | 304 | 7,633 | 21% |
| Education policy | 5,000 | 378 | 3,944* | 28% |
| Poverty/welfare | 5,000 | 1,278 | 3,463 | 17% |
| Environmental policy | 522* | 127 | 8,816 | 51% |
| Homeland security | 2,069 | 201 | 1,452 | 29% |
| Immigration | 2,159 | 171 | 2,090 | 40% |
| Privatization | 1,321 | 66* | 780 | 32%** |
| Transportation | 2,712 | 276 | 5,726 | 3.5% |
| Pensions | 1,968 | 115* | 2,731 | 22% |
| Total number of papers | 44,385 | 9,218 | 69,376 |
*Denotes no controlled vocabulary for this topic
**Denotes sampling done based on 10% of results
Number of Chosen Policy Papers Included in a Database from a Particular Policy Institute.
| Policy Institute | # of Chosen Articles in Policy File | # of Chosen Articles in Policy Archive | # of Chosen Articles in PAIS International |
| American Enterprise Institute for Public Policy Research | 8 | 0 | 7 |
| Brookings Institute | 12 | 0 | 5 |
| Cato Institute | 12 | 4 | 9 |
| Center for American Progress | 7 | 0 | 4 |
| Center for Public Policy and Administration | 0 | 0 | 0 |
| Center on Budget and Policy Priorities | 18 | 4 | 9 |
| Henry L. Stimson Center | 10 | 0 | 12 |
| Heritage Foundation | 10 | 6 | 0 |
| Human Rights Watch | 17 | 0 | 18 |
| Institute for Policy Research | 8 | 0 | 2 |
| Institute for Research on Poverty | 13 | 0 | 9 |
| Milken Institute | 0 | 0 | 7 |
| Nelson A. RockefellerInstitute of Government | 9 | 4 | 2 |
| Pew Charitable Trusts | 1 | 0 | 3 |
| Progressive Policy Institute | 7 | 0 | 2 |
| Public Policy Research Center | 0 | 0 | 0 |
| Reason Foundation | 6 | 4 | 2 |
| Urban Institute | 13 | 0 | 7 |
| Total numbers per database | 151 | 22 | 98 |
Article Categories:
|
|
Refbacks
- There are currently no refbacks.
© 2023 RUSA