Frequently Asked Questions - JFK Database Explorer
This page describes the JFK Database Explorer - what it is, the data it is based on, and how to use it.
QUICK LINKS TO TOPICS ON THIS PAGE
- What is the JFK Database Explorer?
- Where Does This Database Come From?
- What are RIF Records?
- Is the Database Accurate?
- Is the Database Complete and Up-to-Date?
- Is the Entire JFK Collection in This Database?
- What are the "2017" Records?
- Why Can't I View All the Documents?
- How To Use This Explorer - A Quick Tutorial
- How Does the Explorer Relate to the RIF Search Page?
- Why Do I Sometimes See: "JFK Database Explorer Records Limit Reached"?
What is the JFK Database Explorer?
The JFK Database Explorer is a tool for browsing and searching a copy of the JFK Records Collection electronic "index", or database. This database, created by the National Archives and Records Administration (NARA), contains a record for each unique document processed under the 1992 JFK Records Collection Act. There are over 300,000 documents in the database.
Each document record contains a set of fields such as title, agency, date, subjects, and several others. The Explorer lets you pick one such field and see all possible values for that field, and then see a list of records which match any particular value. Then, you may apply search filters to further refine your results.
This Explorer is meant to complement rather than replace the search page provided by the National Archives at www.archives.gov/research/jfk/search.html. That page is the authoritative source of information on these records, and the NARA website is also a handy source of further information about the JFK Collection.
Where Does This Database Come From?
This copy of the JFK Records Collection electronic index was supplied to the MFF by Ramon Herrera, a private researcher who used software to "scrape" the records data off the public pages of the National Archives search system. This work was done over the summer of 2015.
Ramon has also made this copy of the database available through his "Not the National Archives and Records Administration" search page at: www.jfknumbers.org/nara-collection.
The MFF has added the additional programming to create a sophisticated browse-and-search interface to this database, and entitled the result the JFK Database Explorer.
What are RIF Records?
RIF stands for Record Identification Form. Each document in the JFK electronic index, aka database, has a unique 13-digit record number,along with various other form fields of "metadata" which describe the document. The fields are:
- Record Number - 13-digit unique record number assigned by NARA (XXX-XXXXX-XXXXX)
- Record Series - Groupings of documents into series
- Agency - Name of agency releasing the document
- Agency File Number - Original file number or name assigned to the document
- Originator - Organization or person who created the document
- From - Person or organization from whom the document was sent
- To - Person or organization to whom the document was sent
- Title - NARA-assigned document title
- Date - Date when the document was created
- Pages - Number of pages in the document
- Document Type - Type of document (Letter, Report, etc.)
- Subjects - List of subjects discussed in the document, separated by semi-colons
- Classification - Document's security classification
- Restrictions - Restriction codes identifying types of redactions applied to this document
- Current Status - Current release status (Open, Released with Deletions, Postponed in Full)
- Date of Last Review - Date when last reviewed for release or change in redaction
- Opening Criteria - Criteria to be applied to decision-making about release
- Comments - Comments attached to this document
RIF data is shown in this Explorer in a card-like form. The title is the topmost line; the other fields are labeled in the card.
Is the Database Accurate?
This is really two questions in one:
1. Is the National Archives electronic index accurate?
2. Is this a faithful copy of that index?
As to the first question, a quick perusal of the records easily shows a substantial number of typos, mis-spellings, and multiple variations of the same term. As just one example, browsing by Document Type shows that "AFFIDAVIT" has these variants: "AFFADAVIT", "AFFADIVIT", "AFFIDAAVIT", "AFFIDAVID.", "AFFIDAVIT .", "AFFIDAVIT.", AND "AFFIDAVIT.T".
To some extent this is inevitable in such a large database. One value of this Explorer is that the browse feature allows you to see these variations firsthand, whereas a purely search-based system would often leave them hidden from view.
So the answer to question #1 is: not particularly.
As to the second question, the accuracy of this copy, the answer is "so far it appears to be a faithful copy, with one exception." The web-scraping process utilized did not always properly handle long lines which wrapped onto a second line. So on rare occasions long titles are truncated.
Spot-checking various records and counts has as yet revealed no differences beyond this issue, and in general the methodology used to create it seems sound.
While it is unfortunate that the copy has this slight imperfection, the occasional truncation of long titles should not impede general use of this Explorer. Diligent researchers should check any record they are especially interested in against the NARA website.
It is certainly possible that other kinds of errors or deviations from the Archives' database have occurred. You can spot-check any record yourself by using the National Archives search page at www.archives.gov/research/jfk/search.html, entering a record number, and comparing the resulting record to this Explorer's version. If you find an inaccuracy while using the JFK Database Explorer, please let us know at email@example.com. Indicate the 13-digit record number.
Standard Search form at NARA can be used to look up by record number
Is the Database Complete and Up-to-Date?
The method of collecting this data from the National Archives electronic index involved running searches across every represented date range. The result should be complete, though there is no practical way to verify this.
The total number of records in this copy is 319,106. The number of records in the Archives' index was stated by Special Access/FOIA Staff Manager Martha Murphy to be 318,866. This is actually 240 documents fewer than the count of records in the copy; the reason for this discrepancy is as yet unknown.
IMPORTANT. It is important to make clear that this copy is a snapshot of the official JFK database, created in the summer of 2015. Any updates to the National Archives' version will not be reflected here, at least not without additional effort.
Is the Entire JFK Collection in This Database?
No. The entire collection of documents residing in the JFK Collection at the National Archives is estimated to be roughly 5 million pages of documents. By adding up the Pages form field for the ~319,000 records in the database, the total page count for these documents is just under 2 million.
Why the 3-million page discrepancy? Remember that the documents in this database are only those processed under the 1992 JFK Records Collection Act. Documents released prior to that time, in particular voluminous Warren Commission records, are not in this database. Also, the staff files of the Assassination Records Review Board, the body which oversaw the declassification of these records, are themselves outside the database.
Browsing the JFK Records Collection Register on the National Archives' website provides some insight into the nature of the full collection, though the listing is fairly abbreviated and page counts are not provided.
What are the "2017" Records?
In response to a Freedom of Information Act request by FOIA specialist Michael Ravnitzky, in early 2016 the National Archives provided a 146-page listing of RIF records of documents. These are records currently "postponed in full", but scheduled to be released in full in 2017.
As described on the NARA website, the 1992 JFK Records Collection Act provided for postponement of release of documents or portions of documents, with the stipulation that all such withholding would end on October 26, 2017, unless overridden by the President of the United States.
NARA has set up a team and is currently processing what are said to be 3,603 document currently postponed in full (later revised to 3,598), as well as many more currently redacted (portions blacked out).
There are a couple of discrepancies in the numbers, which the MFF is attempting to resolve:
- The 146-page PDF file lists only 3,571 documents, not the 3,598 said to be postponed in full.
- The JFK Database shows 9,718 documents currently postponed in full, not 3,598 or 3,571. Possibly the other 6,000+ documents are not really postponed in full and the database is out of date, but this has not been verified.
According the Database, the number of documents classified RELEASED WITH DELETIONS, and therefore also scheduled for full release in 2017, is 34,927.
In this Explorer, the records listed in the 146-page PDF are marked with (2017) on their record cards:
A record card marked as scheduled for 2017 release
You can also hone in on just the 2017 postponed-in-full records in two ways:
1. use the filter panel available to refine any search result to refine the search to include only 2017 option (see How to Use This Explorer section subsequently)
2. use a special link on the main page of the Explorer to view all such records (see below):
Special link near bottom of main JFK Database Explorer page
Why Can't I View All the Documents?
The National Archives website allows you to search the JFK Collection and view the metadata records, but not the actual document pages themselves. Here at the MFF, often you can take that next step.
But only a subset of the full JFK Collection is available.
The Mary Ferrell Foundation website contains over 1.3 million pages of searchable and viewable documents. A little over 1 million of those pages are JFK Collection documents; others relate to the assassinations of Martin Luther King Jr. or Robert Kennedy, or other topics such as Watergate.
So given that the complete JFK Collection holds approximately 5 million pages, then only about 20% of those pages are viewable online.
In terms of just the documents processed under the JFK Records Act, the percentage is slightly higher. Of the roughly 319,000 documents in the database, over 100,000 are available on MFF. That's a little over 30% of the total.
Why not 100% of the documents? The Mary Ferrell Foundation is not the National Archives, and we must obtain paper copies of these documents, scan them, and then process and upload the results. This is very time-consuming, especially the first step of acquiring paper copies. The MFF's relationship with the Assassination Archives and Research Center (AARC) has been critical to this effort.
The documents on this website come from a variety of sources, but most particularly:
- The holdings of the AARC. The tireless efforts of AARC President Jim Lesar and others at this organization have provided tremendous benefit to all of us. A single lawsuit brought by Mark Zaid and litigated by Jim Lesar - for CIA and FBI records seen by the HSCA - generated three-quarters of a million pages, all of them online here at MFF.
- Documents scanned for History Matters, including the 26 volumes of the Warren Report, the HSCA volumes, and more
- Documents scanned or photocopied in person at the National Archives
Documents which you can view online are marked specially by a red "View Document" button on the record card:
A record card with View Document button for online viewing here at MFF
Note that sometimes the MFF copy of a document may not be the most up-to-date version available, in terms of redactions. In these cases, the JFK Database Explorer information will be current (as of 2015), but the first page of the document, the scan of a printed RIF form page, will have older information.
There are also cases where the MFF has more than one copy of a given record. In these cases, the View Document button will provide a menu and allow you to pick which to view. If the document is of particular interest, you may want to view all copies. See the blue menu below the button:
How To Use This Explorer - A Quick Tutorial
The JFK Database Explorer has a powerful search engine wrapped inside a browse interface. The main page at www.maryferrell.org/php/jfkdb.php presents three ways of getting started:
- Pick a field to browse by
- See all records marked for release in 2017
- See all records in the entire database
Start with one of the first options. Pick a field such as Agency, Subjects, or Date. When you click on one of these links, you will be taken to a browse page. For example, if you picked "Record Series" to browse by, you will see this page:
Browsing by Record Series
Each row shows you a value for this field to the right, and to the left of it are three numbers::
- Total Count - the number of documents whose field matches this row's value column
- on MFF - the number of those which are viewable on MFF
- in 2017 - the number which are in the 2017 release list
Note the light blue sort bar, which lets you sort the resulting list alphabetically, or by document count (entries with higher documents are listed at the top). Below that, if there are more than 100 entries, a paginator lets you flip to different pages of 100 rows.
Some fields, such as Subjects, From, and To, have so many entries that a second level of "drill down" browsing is used. So for example if you select Subjects from the main page, the next page simply lists the first letter A-Z, and then you drill down into the rows of values starting with A (or B, or C, ...).
Click on a given value (in red), and you will be taken to a search results page, which shows the actual RIF records for matching documents:
Search results, in this case records whose Record Series matches the value
"08: NUMBERED COMMISSION DOCUMENTS"
Note that the sort bar has changed - these results pages can be sorted by record number, title, agency file number, or date. Instead of a paginator, there is a Next>> link for the next page of 20 results (and <<Prev if you are not on the first page).
The records include a top stripe with the following:
- Document title
- Record number
- (optional) If on MFF, red View Document button
- (optional) If in 2017 set, a (2017) tag
The larger bottom part of the card shows the remaining RIF form fields and their values. The field being browsed is boldfaced.
But wait! These cards are not static. Click on any field's value, and you will instantly be taken to a search page showing all records with a matching field:value pair.
A last powerful feature is the Filters link at the right side of the sort bar. Click it to drop down a filters panel. Here, you may refine the result set, filtering it by only documents within a date range, or only documents viewable on MFF, or in other ways.
Filter panel, used to winnow down the set of matching documents
The "click-on-field" feature, and the Filters panel shown above, combine to make a very powerful search engine inside the overall browse structure. They allow for a huge range of exploration of the JFk Collection Database. Explore away!
How Does the Explorer Relate to the RIF Search Page?
The MFF's RIF Search page is unchanged, and is only used for searching the RIF-based documents whose pages are available for viewing on this website.
The JFK Database Explorer, by contrast, lets you browse and search/filter the entire set of RIF-based documents in the JFK Collection. You are still obviously limited in terms of which ones you can view fully.
The two tools are complementary, and both useful. The JFK Database Explorer is in some ways encompasses the functionality of RIF Search and adds more. On the other hand, the RIF search tool can be more quickly used to look up a document by record number or in certain other ways, if you are only interested in the subset of documents actually viewable on the MFF.
Why Do I Sometimes See: "JFK Database Explorer Records Limit Reached"?
If you are not a member, or not logged in, you may use the JFK Database Explorer for a bit and then see this page:
While the JFK Database Explorer's drill-down "browse by field" mechanism is fully open, the search results pages which show cards of records are not. Just like the MFF's regular and RIF search features, only a limited number of free searches are afforded non-members.
The MFF is as dedicated as always to providing access to these valuable materials, and has never locked out any particular documents or pages for members-only treatment. But there are bills to be paid. Our best compromise approach to that reality has always been this: to limit usage of some of our tools to members-only, but not limit the viewability of documents, essays, or other materials per se. The JFK Database Explorer follows this model.
Supporting memberships, which provide unlimited searches of all search types, including this tool, are available for only $34.95 per year. Pro memberships, at $79.95/year, add the ability to download PDF copies of documents for offline reading, printing, and other uses.
Please support the work we do, providing not only the largest online collection of documents on these topics, but also sophisticated tools like the Mary Ferrell Database, Stewart Galanor's Dealey Plaza Witness Database, the CIA Cryptonym Project, and now the JFK Database Explorer.
Learn more about a Mary Ferrell Foundation Membership