Archives and data in United States

Archives and data

See Using public records

Archives and data

1. Federal Archives. The U. S. National Archives include material that dates from before the founding of the country. Some documents and other materials were, in fact, carefully set aside in their own time, because someone knew they were important. Others, however, just happened to be stored and forgotten, only to be found years later. Still others may have been purchased by the Archives or donated to them by collectors or the families of their original owners, who had carefully preserved them. The Archives estimates that about 1-3% of federal records and other materials are preserved because of their historic interest. The rest are thrown away or otherwise lost.

Preserving records means more than just storing them. Paper breaks down over time, and must be treated with chemicals and/or kept in a carefully controlled environment so that it won’t crumble into dust in a hundred years or so. Other materials – film, magnetic tape, glass photo negatives – also must be properly treated and stored, and handled with great care. Most archives of any size employ conservators – experts in restoring and preserving these materials – to keep their collections in good condition.

There is another element to preservation, which has only surfaced as computer use has become nearly universal in developed countries. Records may reside in an obsolete format, and will no longer be readable unless they’re constantly transferred to the latest medium. How many of us have the capacity to listen to Edison-era wax cylinder recordings, for instance? More to the point, how many of us have drives that can read 51/4 -inch floppies? 3.5-inch floppies will be in the same category before long, and CDs and DVDs will undoubtedly soon follow on the road to oblivion.

Not all archived materials are well known historical documents. Among those stored in the U.S. National Archives, for instance, are records, and sometimes original copies, of:
•Laws and regulations from the earliest days of the country to the present
•Proceedings of the Congress for much of the country’s history
•Treasury records
•Census data from the first U.S. Census to the present
•Immigration and naturalization records
•Indian affairs documents
•Customs and shipping
•History and historical documents (famous and not-so-famous)
•Military and veterans’ information
•150 years of photographs
•Survey and other maps
•Audio recordings of Presidents’ inaugural speeches
•Historically significant personal documents

In addition to the collections of the National Archives, some government agencies have archives of their own, also open to the public. An enormous source of archival material in the U.S. is the Library of Congress, which includes, among other treasures, recorded oral histories of thousands of people from the 1930’s and ’40’s on audiotape and vinyl, as well as in transcripts.

Most national governments have archives, although whether or not they are open to public examination depends on the government. Some that are public, and that have websites, include:

UK national Archives

Canadian National Archives

Australian National Archives

The National Archives of India

The National Archives of Ireland

2. State archives. Not only U.S. states, but the states and provinces of other countries as well, usually have their own archives. These are similar to the federal archives (and may even contain some of the same or similar material), but largely dedicated to the history of the state or province, rather than of the country as a whole. Holdings differ widely from state to state, partially because of the differences in their establishment dates, at least in the U.S. Some records – often dating back to the state’s beginnings – a state archive might contain include:
•Laws and regulations dating back to the founding of the state
•Legislative proceedings
•Election information
•Vital statistics (births, deaths, marriages, etc.)
•Property transactions
•Deeds
•Wills and other probate-related material
•Photographs, particularly of the early days of the state, or old photos of well-known state citizens and places
•Maps
•Archeological surveys
•Audio tape, film, and videotape of press conferences, speeches, and other historically or politically significant events
•Paintings or other artwork by state-based artists or referring to the state (Massachusetts, for example, has a series of paintings of “The Birds of Massachusetts.”)

As with public records, a quick search under “archives” and the name of the state should get you a website. Many state archives will also be part of or linked to their states’ websites.

3. County and municipal archives. These are likely to contain material similar to state and municipal archives, but relating instead to county or town history. In most cases, official archives exist only for large towns and cities (and not for all of those), which have the resources to support them.

The situation varies from town to town, and you should check to find out whether there’s a local archive or not. Smaller towns may store records, but they often aren’t catalogued, so that specific material is difficult, if not impossible, to find. In many cases, small-town records aren’t stored at all because there’s no place to put them, or are stored in cardboard boxes in someone’s attic or garage and forgotten, or lost to mildew and vermin. Without some sort of fundamental conservation, paper records seldom survive past 50 years or so.

In the small New England town where the author lives, town records – some nearly 200 years old – have been stored in two enormous safes in the Selectmen’s office and in the notoriously damp basement of the 19th-Century town hall. Recently, the town obtained a grant to catalogue them – a task largely carried out by volunteers – and to store them in better boxes. What’s been done, however, is far from real conservation, and the town still has no official place to store its records.

4. Other useful archives. Not all archives – even those containing public documents – are official and government-sponsored. Some private organizations and institutions keep their own, usually relating to the history of the entity; and some are unofficial, though generally related to the work of the entities that maintain them. Some of the most common:
•Newspaper archives. Most newspapers keep files – paper, microfilm, or electronic – of every paper they’ve published (or at least all they can retrieve) since their founding. These are often available through public or academic libraries as well, and some are even available on line for free.
•Magazines keep similar archives.
•Audio, video, and movie collections, some kept by studios and broadcast networks, some in private hands, and some maintained by some of the institutions listed below

Video collections are particularly fragile, especially early films and the kinescopes of early TV shows. The celluloid on which they’re recorded deteriorates over time, a process hastened by their exposure to various temperature extremes and particular substances. In many cases, playing them contributes to their destruction. Unless transferred to a longer-lasting medium – generally by digitizing, and storing them as DVD’s or computer files – classic examples of early movies, TV shows, and other performances can be lost forever.
•Private photo archives. The best known of these is probably the Bettmann Archive, keeper of millions of images, which sells rights to print individual images.
•Museum collections. Museums often buy whole sets of records, photographs, maps, and other material from collectors or others, and carefully catalogue them. Many make these materials available for research.
•Public libraries. Libraries are publicly funded, but are not archives in and of themselves. They almost always have large collections of archival material, however, often locally oriented.
•University libraries.
•Archive of organizations and institutions. Many organizations keep archives of their own history and proceedings in order to maintain an institutional memory. Among those who might have such records are health and human service organizations (including hospitals), colleges and universities, and private clubs or associations.

These archives may contain both historical material and objects – old surgical instruments, refectory benches from 18th century dining halls – and records and minutes of the business of the entity – membership lists, board meetings, grant applications, study results, etc.
•Items not specifically meant for historic or research purposes. A group looking for the original use of a piece of land it wants to preserve as open space, for example, might find that information in a painting on someone’s living room wall or a photo in a great-grandmother’s scrap book.

In addition to the governmental archives listed above, here are some archived survey data resources that may be helpful:

ICPSR – Inter-University Consortium for Political and Social Research
ICPSR offers more than 500,000 digital files containing social science research data. Disciplines represented include political science, sociology, demography, economics, history, gerontology, criminal justice, public health, foreign policy, terrorism, health and medical care, early education, education, racial and ethnic minorities, psychology, law, substance abuse and mental health, and more.

General Social Survey
The GSS contains a standard ‘core’ of demographic and attitudinal questions, plus topics of special interest. Many of the core questions have remain unchanged since 1972 to facilitate time trend studies as well as replication of earlier findings.The GSS takes the pulse of America, and is a unique and valuable resource. It is the only survey that has tracked the opinions of Americans over an extended period of time.

Public Use Data Files from the California Health Interview Survey

Centers for Disease Control and Prevention
Data from the CDC including statistics on various topics (such as aging, alcohol, births, genomics, HIV/AIDS, reproductive health, physical activity, weight and obesity, smoking, STDs and others).

Public-Use Microdata Files from Social Security

Public Use Files from the National Bureau of Economic Research

National Longitudinal Surveys from the Bureau of Labor Statistics

U.S. Energy Information Administration

Presidential Data Archive
The American Presidency Project contains the most comprehensive collection of resources pertaining to the study of the President of the United States.

Surveillance Epidemiology and End Results
SEER Data, 1973-2006
(Including July-December 2005 Hurricane Katrina Impacted Louisiana Cases)
The SEER limited-use data include SEER incidence and population data associated by age, sex, race, year of diagnosis, and geographic areas (including SEER registry and county). A signed Limited-Use Data Agreement is required to access these data. Use of these data for publication purposes should contain a citation which includes submission and release dates. See Citations for SEER Data and SEER*Stat Software for specific suggestions regarding the citation format.

National Practitioner Data Bank
Healthcare Integrity and Protection Data Bank
The NPDB Public Use Data File contains selected variables from medical malpractice payment and adverse licensure, clinical privileges, professional society membership, and Drug Enforcement Administration (DEA) reports (adverse actions) received by the NPDB concerning physicians, dentists, and other licensed health care professionals. It also includes reports of Medicare and Medicaid exclusion actions taken by the Department of HHS Office of Inspector General. Malpractice payers, state licensing agencies, hospitals, other entities, and professional societies are required to report this data to the National Practitioner Data Bank under the provisions of Title IV of P.L. 99-660, Health Care Quality Improvement Act of 1986, as amended. This file does not include any information that identifies individual practitioners or reporting entities. The file is designed to provide data for statistical analysis only.

HEDIS Public Use Files
Provides annual Medicare Health Plan Employer Data and Information Set (HEDIS) performance measures.

NCES Surveys and Data
The National Center for Education Statistics (NCES) is the primary federal entity for collecting and analyzing data related to education.

Add Health Data
The Add Health study collects data to use in exploring the influences of both the individual attributes of respondents and the attributes of their various environments on health and health-related behavior.

National Science Foundation

HINTS: Health Information National Trends Survey
HINTS collects nationally representative data routinely about the American public’s use of cancer-related information.

SIPP: Survey of Income and Program Participation
The main objective of SIPP is to provide accurate and comprehensive information about the income and program participation of individuals and households in the United States, and about the principal determinants of income and program participation. SIPP offers detailed information on cash and noncash income on a subannual basis. The survey also collects data on taxes, assets, liabilities, and participation in government transfer programs. SIPP data allow the government to evaluate the effectiveness of federal, state, and local programs.

Substance Abuse and Mental Health Services Administration (U.S. Department of Health and Human Services)
Office of Applied Studies Statistics
Office of Applied Studies Data Tables

U.S. Department of Health and Human Services: Analysis of the Effect of Regulation on the Quality of Care in Board and Care.

Cornell Institute for Social and Economic Research
A list of links to internet data for Social Scientists

C. Use public records and archives to gain information that will inform your assessment

In general, it’s better to know more rather than less, but there’s so much data available that you could find yourself lost in it. Try to narrow your search to material that’s truly relevant to your goals.
One way to increase the reliability of data where you are dealing with small data sets is to combine multi-year data (for instance, results of cancer deaths in a community for three years instead of one). A drawback to this option is that looking at multi-year data limits the ability to monitor program interventions and identify new trends. Rolling year averages (e.g., looking at data for 1997-2000 one year, and 1998-2001 the following year) may overcome this drawback and should is an option that should be considered.

Another way to decrease the possibility of statistical instability is to expand the geographic area you are investigating by looking at regional health assessments conducted by collaborating neighboring jurisdictions, or in the example above, expanding from county to state. A drawback to this option is that you may then be examining results for a geographical area that does not necessarily apply to your assessment. Analyzing data at the regional level may also mask interesting local variations in the data.

Resources

Internet resources:

U.S. National Archives and Records Administration.

The Centers for Disease Control and Prevention.

The website of the U.S. Census Bureau. Census data, maps, and reports.

Canadian National Archives.

The U.S. Department of Labor.

The U.S. Department of Education.

The U.S. Environmental Protection Agency.

Text of the Freedom of Information Act.

The Federal Election Commission (oversees the Campaign Finance Reform Act).

The Freedom of Information Center at the University of Missouri, providing more information on the Freedom of Information Act, as well as links to the state laws on freedom of information.

Links to the 50 official U.S. state websites.

The Congressional Record, which records the proceedings of the U.S. House of Representatives and Senate, with an index going back to 1983.

The U.S. Department of Health and Human Services

The U.S. Department of Housing and Urban Development.

Text of the U.S. Uniform Commercial Code (UCC), with links to state laws that follow the guidelines of the UCC.

Villanova Law School Library index of federal court records on the Web.

Australian National Archives

The National Archives of Ireland

UK national Archives

The National Archives of India

The (U.S.) National Institute of Health.

Open Secrets. Filings under the Campaign Finance Reform Act – records of corporate political contributions, and who got them.

The Occupational Safety and Health Administration of the Department of Labor.

PACER (Public Access to Court Electronic Records). Federal court records, available directly from the federal court system for a reasonable fee.

The website of the industry magazine Search Engine Watch. Tips on searching the web, explanations of how search engines work, and other information.

Search Systems, a collection of links to free (and some not-free) public records and other material, including federal court records (some free) and state records in a variety of areas.

The U.S. Securities and Exchange Commission, which oversees stock and bond trading and corporate activities.

The website of the U.S. Supreme Court: opinions, dissents, and other information from recent sessions (past three years) are available at no charge.

Project Vote Smart. Voting records of all members of the U.S. Congress.

Print resources:

Berg, Bruce L. (2007), Qualitative Research Methods for the Social Sciences (6th edn). Boston: Allyn and Bacon. Contains a chapter on using public and private archives.

Hock, Randolph (2004), The Extreme Searcher’s Internet Handbook: A Guide for the Serious Searcher. Medford, NJ: CyberAge Books.

Lane, Carole A. (2002), Naked in Cyberspace: How to Find Personal Information Online (2nd edn). Medford, NJ: CyberAge Books. Contains helpful general search information.

Main source: Community Tool Box: http://ctb.ku.edu/