Protocol

Overview

This page describes the protocol for collecting, capturing, and cleaning data associated with the project.

The text here is adapted from a working draft document written in Google Docs that is hosted here:

https://docs.google.com/document/d/1-ZKrNflBO7UIq5I0pJ8U5K-rtoiads12Zf_OQq_S65Q/edit

Warning

On April 1, no joke!, we changed the protocol substantially based on our discovery that Paperpile can export references with useful tags directly to our Github repo.

Literature search

Goals

1. Capture the published literature that reports data on using Teller Acuity Cards.

2. Create a database of that literature.

Phase I: Search

Note

Here are some preparatory steps that need to be completed prior to doing literature search tasks.

1. Open Google Chrome from a lab computer.
2. Log into psubrainlab’s google account (psubrainlab@gmail.com ask Dr. Gilmore or Andrea for password)
3. Search “paperpile chrome extension” and install the paperpile chrome extension
4. On the paperpile website click ’log in”. Choose the “log in with google” option

Search terms

teller acuity cards
visual acuity cards
teller cards

Google Scholar

1. Visit Google Scholar: https://scholar.google.com/
2. Enter the search term
3. Make sure to select ‘Sort by relevance’, ‘Any type’, and do not check ‘include patents’ or ‘include citations’

For example, this URL uses teller acuity cards:

https://scholar.google.com/scholar?hl=en&as_sdt=0,39&as_vis=1&q=%22teller+acuity+cards%22

When looking for sources on Google Scholar, look for the Paperpile box and ensure it is gray. Gray sources are sources we do not currently have in Paperpile.

Reference not yet in our Paperpile

Green sources indicate we already have collected them in Paperpile.

Reference already in Paperpile

Add the references not already in Paperpile by pressing the grey button.

4. After you have retrieved a page of entries, switch to the Paperpile tab to add the TAC tag to these entries. Use shift-click to select multiple entries in Paperpile.

Papers recently added to Paperpile. After clicking on the tags button and entering ‘TAC’ in the search window, Paperpile lists all of the paper with the . ‘TAC’ tag. To add ‘TAC’ to the new, selected papers, click on the checkbox.

Phase II: Data entry

Goals

1. Create a database of references found in Phase I.

Option A: Automated extraction

Note

It is possible to export the Paperpile references with the TAC tag directly to Github. Rick tested this workflow on 2024-02-02. What is yet unclear is whether the reference file automatically updates.

There appear to be some R packages that can read *.bib files and convert them to data frames, see https://rdrr.io/cran/revtools/ and https://docs.ropensci.org/bib2df/.

We should assign someone the task of determining how to parse a *.bib file into a data frame.

Option B: Manual entry

In the meantime, we fall back on manual data entry.

1. Open the Google Sheet “Legacy Project Acuity Data: By Paper” and open the paper_data tab:

https://docs.google.com/spreadsheets/d/1UFZkbh9oU4JHpYsrkDQcNmDyqD4J-qB74dhyMzIkqKs/edit#gid=2144658778

Figure 2.1: Paper reference database

2. In a second Chrome window, click on the Paperpile icon and open Paperpile.

3. In the left panel of the Paperpile window, search for the TAC tag and select it.

Selecting the ‘TAC’ (Teller Acuity Cards) tag in Paperpile

4. In the right panel of the Paperpile window, select sort by author.

Sort Paperpile entries by author

5. Shrink the Paperpile window and push it flush right, then shrink the Google Sheets window and push it flush left.

Window placement for data entry

6. Sort Google Sheet data by Author

7. Copy citation (in APA format) from Paperpile (using the command+c keyboard shortcut) and paste into ‘citation’ field in Google Sheet.

8. Reformat Google Sheet ‘citation’ column.

1. Select column
2. Select Format | Wrapping | Wrap

20. Sort Google Sheet data by author_first and pub_year

Data | Sort Range | Advanced

Sorting a Google Sheet

Sort settings for author_first and pub_year

Data evaluation & extraction

Goals

1. To evaluate each paper to determine whether it contains extractable data.
2. To extract group-level data from each paper identified as having extractable data.
3. Enter group-level data into a common database.

Data evaluation

Phase I

For each paper in the paper_data tab in the Google Sheet

https://docs.google.com/spreadsheets/d/1UFZkbh9oU4JHpYsrkDQcNmDyqD4J-qB74dhyMzIkqKs/edit#gid=2144658778

1. Open the paper via its URL.

Add “yes” to the url_openable data column in the Google Sheet to report that you could open a PDF or web version of the complete paper.

Enter “paywall” if the URL resolves, but you would need to pay for access to the paper.

If the paper can’t be found because the URL doesn’t resolve, enter “not_found”.

Phase II

Select the papers that could be opened (url_openable == “yes”)

1. Scan the paper to find any data tables that contain acuity card summary data.

Data extraction

Quality Assurance