Active Projects

The Databrary Project aims to increase scientific transparency and accelerate discovery in developmental science by building infrastructure for researchers to share video data and related meta-data. The project has five specific aims:

Databrary is an open-source software project. Penn State is one of the major “nodes”, with a large number of authorized users.

Publications

Gilmore, R. O., Kennedy, J. L., & Adolph, K. E. (2018). Practical Solutions for Sharing Data and Materials From Psychological Research. Advances in Methods and Practices in Psychological Science, SAGE Publications Inc. Retrieved from https://doi.org/10.1177/2515245917746500. OSF preprint.

Adolph, K.E., Gilmore, R.O., & Kennedy, J.L. (2017, October). Video data and documentation will improve psychological science. Psychological Science Agenda. http://www.apa.org/science/about/psa/2017/10/video-data.aspx

Gilmore, R. O., Kennedy, J. L., & Adolph, K. E. (2017, September 7). Practical solutions for the ethical sharing of identifiable research data. Retrieved from http://psyarxiv.com/kew8u.

Gilmore, R.O. & Adolph, K.E. (2017). Video can make behavioural science more reproducible. Nature Human Behaviour. doi:10.1038/s41562-017-0128.

Gilmore, R.O., Diaz, M.T., Wyble, B.A., & Yarkoni, T. (2017). Progress toward openness, transparency, and reproducibility in cognitive neuroscience. Annals of the New York Academy of Sciences, 1396, 5–18. doi: 10.1111/nyas.13325.

Gilmore, R.O., & Adolph, K.E. (2017, February 6). Video can make science more open, transparent, robust, and reproducible. Retrieved from http://osf.io/3kvp7.

Gilmore, R.O., & Adolph, K.E. (in press). Open sharing of research video: Breaking the boundaries of the research team, in Advancing Social and Behavioral Health Research through Cross-disciplinary Team Science: Principles for Success. Hall, Kara, Croyle, R., & Vogel, A. (Eds.). Springer.

Gilmore, R.O., Adolph, K.E., Millman, D.S. (2016). Curating identifiable data for sharing: The Databrary project. In Proceedings of the 2016 New York Scientific Data Summit. PDF of paper.

Gilmore, R.O., Adolph, K.E., Millman, D.S., & Gordon, A. (2016). Transforming education research through open video data sharing. Advances in Engineering Education, 5(2). HTML.

Gilmore, R.O. (2016). From big data to deep insight in developmental science. Wiley Interdisciplinary Reviews Cognitive Science. DOI: 10.1002/wcs.1379.

Gordon, A., Millman, D.S., Steiger, L., Adolph, K.E., & Gilmore, R.O. (2015). Researcher-library collaborations: Data repositories as a service for researchers. Journal of Librarianship and Scholarly Communication. doi:10.7710/2162-3309.1238.

Adolph, K.E., Gilmore, R.O., Freeman, C., Sanderson, P., & Millman, D. (2012). Toward Open Behavioral Science, Psychological Inquiry: An International Journal for the Advancement of Psychological Theory, 23(3), 244-247. doi:10.1080/1047840X.2012.705133.

Presentations

Gilmore, R.O. (2018, May). Open, says me: Practical solutions for sharing data and materials. Invited talk at the Association for Psychological Science 2018 meeting. San Francisco, CA. HTML slides

Gilmore, R.O. (2018, April 25). Video as data and documentation. Talk in the Department of Communication Arts & Sciences, Penn State. HTML slides

Gilmore, R.O. (2018, February 14). The future of quantitative developmental science. Talk given at the Quantitative Developmental Methodology brown bag series. HTML slides.

Gilmore, R.O. (2018, January 31). Introducing Databrary. Talk given at the Penn State Software in the Humanities & Social Sciences workshop. HTML slides.

Gilmore, R.O. (2018, January 26). The Video DatAbservatory: A platform for behavioral discovery. Talk at the Pathways to Competence initiative meeting of the Penn State Child Study Center. HTML slides.

Gilmore, R.O. (2017, September 29). Data sharing, research ethics, & scientific reproducibility. Talk at the Scholarship and Research Integrity (SARI) workshop series, Penn State. HTML slides.

Gilmore, R.O. (2017, September 7). Reproducibility in computationally intensive behavioral research. Talk at the ACI-ICS seminar series, Penn State. HTML slides.

Gilmore, R.O. (2017, August 1). Video can improve the reproducibility of psychological science. Lightning talk at the Society for Improving Psychological Science meeting, Charlottesville, VA. HTML slides

Gilmore, R.O. (2017, July 31). Beyond physics envy: Toward a databservatory for human behavior. Lightning talk at the Society for Improving Psychological Science meeting, Charlottesville, VA. HTML slides.

Gilmore, R.O. & Nilsonne, G. (2017, July 30). IRBs and the ethical sharing of research data. Talk given at the Society for Improving Psychological Science meeting, Charlottesville, VA. HTML slides. OSF project.

Gilmore, R.O. (2017, July 26). Yes, we can. Invited panelist at the AERA Workshop on Data Sharing and Research Transparency at the Article Publishing Stage. HTML slides.

Gilmore, R.O. (2017, July 10). The reproducibility crisis in psychology & neuroscience. Talk given at the Penn State Data Reproducibility Bootcamp. HTML slides.

Adolph, K.E., Binion, G. Gilmore, R.O., Oakes, L., & Vazire, S. (2017, April 6). Openness, replication, & data reuse in developmental science – unique challenges, existing resources, & what is still needed. Invited roundtable at the Society for Research in Child Development meeting, Austin, TX.

Gilmore, R.O. (2017, February 22). A Databservatory for human behavior. Talk given at the Cognitive Area Brown Bag. HTML slides.

Gilmore, R.O. (2017, January 31). An -ome of our own: Toward a more reproducible, robust, and insightful science of human behavior. Talk given to the Social Data Analytics (SoDA) 501 students. Penn State University. HTML slides

Gilmore, R.O. (2016, October). The future of big data in developmental science. Talk given at a meeting of the Penn State Child Study Center (CSC) faculty. HTML slides.

Gilmore, R.O. (2016, September). Donald Rumsfeld and the promise of a ‘big data’ science of human behavior. Talk given at a meeting of the Stochastic Modeling and Computational Statistics (SMACS) group, Department of Statistics. HTML slides.

Gilmore, R.O., Adolph, K.E., & Millman, D.S. (2016, August). Curating identifiable data for sharing: The Databrary project. In Proceedings of the 2016 New York Scientific Data Summit. HTML slides. PDF of paper.

Gilmore, R.O., Adolph, K.E., & Millman, D. (2016, May). Video doesn’t lie: Reproducible workflows with Databrary. Talk given at the NYU Data Science Center Symposium on Reproducibility. HTML slides

Gilmore, R.O., Adolph, K.E., Millman, D.S., Steiger, L., & Simon, D.A. (2015, May). Sharing displays and data from vision science research with Databrary. Poster presented at the Vision Sciences Society meeting, St. Pete Beach, FL. PDF.

Simon, D.A., Gordon, A.S., Steiger, L., & Gilmore, R.O. (2015, June). Databrary: Enabling sharing and reuse of research video. Proceedings of the 15th ACM/IEEE-CS Joint Conference on Digital Libraries, Knoxville, TN. doi:10.1145/2756406.2756951

Collaborators

Support

This project is supported by the U.S. National Science Foundation (NSF) Grant No. BCS-1238599, the Eunice Kennedy Shriver National Institute of Child Health and Human Development under Cooperative Agreement 1-U01-HD-076595-01, the Society for Research in Child Development, the LEGO Foundation, and the Alfred P. Sloan Foundation.

Play & Learning Across a Year (PLAY)

The Play & Learning Across a Year (PLAY) project serves as a model system for doing development science from a “big” data approach. Natural free play represents the foundation of infant learning, but we know little about how infants play, how play unfolds in real time and across development, and how individual and group differences promote infant learning and development through play. To answer these questions, the PLAY project will collect, code, and share 900 hours of video collected in the homes of children at 12, 18, and 24 months of age drawn from 30 sites across North America.

The aim of the project is to develop a new approach to developmental science that enables (1) “big” data science for researchers who would not otherwise have access; (2) a communal, low-cost means of collecting and coding data that retains the autonomy of individual labs; and (3) a plan for leveraging diverse expertise to address a common goal.

Presentations

Adolph, K.E., Tamis-LeMonda, C., & Gilmore, R.O. (2016, December 16). Video-based communal data collection & coding: Advancing the science of infant learning & development. A workshop held at the Eunice Kennedy Shriver National Institute of Child Health and Human Development. Video Cast. Materials & data.

Materials

Gilmore, R.O. (2017). PLAY-behaviorome. Github code repository. Retrieved December 19, 2017 from https://github.com/PLAY-behaviorome/.

Adolph, K., Tamis-LeMonda, C. & Gilmore, R.O. (2017). PLAY Pilot Data Collections. Databrary. Retrieved December 19, 2017 from https://nyu.databrary.org/volume/444.

Adolph, K., Tamis-LeMonda, C. & Gilmore, R.O. (2016). PLAY Project: Materials. Databrary. Retrieved December 19, 2017 from https://nyu.databrary.org/volume/254.

Adolph, K., Tamis-LeMonda, C. & Gilmore, R.O. (2016). PLAY Project: Webinar discussions on protocol and coding. Databrary. Retrieved December 19, 2017 from https://nyu.databrary.org/volume/232.

Collaborators

  • Karen Adolph, New York University, Co-Principal Investigator
  • Catherine Tamis-LeMonda, New York University, Co-Principal Investigator

Support

Support for the planning phase of this project has been provided by the Eunice Kennedy Shriver National Institute of Child Health and Human Development and the LEGO Foundation. A grant is under consideration by the National Institutes of Health.

Developmental Dynamics of Optic Flow Processing

Brain activity Optic flow

Visual motion provides humans and animals with information about their own movement through 3D space and about the structure of the environment – the objects, surfaces, and other animals that it may contain. How the human brain processes complex motion information poses an as-yet unanswered question. This project focuses on characterizing how sensitivity to visual motion emerges in the developing human brain: how brain (EEG) responses to patterns of ego- and object motion emerge, how they develop from infancy through childhood into adulthood, how specific changes in cortical circuitry might account for the observed patterns, and how behavioral sensitivity to motion corresponds to neural activation. The studies compare brain responses and behavioral discrimination patterns in infants, children, and adults to the same types of ego- and object motion. The studies also involve an effort to measure or simulate the statistics of optic flow experienced by infant, child, and adult observers in complex, natural environments using computer vision methods.

Publications

Gilmore, R.O., Thomas, A.L., & Fesi, J.D (2016). Children’s brain responses to optic flow vary by pattern type and motion speed. PLoS ONE. doi: 10.1371/journal.pone.0157911. Materials on Databrary at http://doi.org/10.17910/B7QG6W.

Gilmore, R.O., Raudies, F., & Jayaraman, S. (2015). What Accounts for Developmental Shifts in Optic Flow Sensitivity? Proceedings of the IEEE International Conference on Development and Learning and Epigenetic Robotics. doi:10.1109/DEVLRN.2015.7345450. Materials on Databrary at doi:10.17910/B7988V.

Fesi, J.F., Thomas, A.L., & Gilmore, R.O. (2014). Cortical responses to optic flow and motion contrast across patterns and speeds. Vision Research, 100, 56–71. doi:10.1016/j.visres.2014.04.004. Materials on Databrary.

Raudies, F. & Gilmore, R.O. (2014). Visual motion priors differ for infants and mothers. Neural Computation, 26(11), 2652-2668. doi:10.1162/NECO_a_00645.

Raudies, F., Gilmore, R.O., Kretch, K.S., Franchak, J.M, & Adolph, K.E. (2012). Understanding the development of motion processing by characterizing optic flow experienced by infants and their mothers. Proceedings of the IEEE International Conference on Development and Learning. doi:10.1109/DevLrn.2012.6400584.

Presentations

Gilmore, R.O. (2018, February 24). The development of perception for action. Data blitz talk at the Developmental Area graduate recruiting weekend. HTML slides.

Gilmore, R.O., Seisler, A., Shade, M., O’Neill, M. (2017, April). School-age children perceive fast radial optic flow in noise more accurately than slow linear flow. Poster presentation at the Society for Research in Child Development, Austin, TX. PDF. Databrary. GitHub.

Gilmore, R.O. (2017, February). Go with the flow: The development of behavioral sensitivity and brain responses to optic flow. Talk at Temple University. HTML slides. Markdown.

Gilmore, R.O., Fared, D.A., Dexheimer, M.G., & Seisler, A.R. (2016, November). The appearance and disappearance of visual forms defined by differential motion evokes distinctive EEG responses in school-age children. Presentation at the Society for Neuroscience meeting in San Diego, CA. PDF.

Gilmore, R.O. (2016, October). Go with the flow: The development of behavioral sensitivity and brain responses to optic flow. Talk at the Penn State Action club meeting. HTML slides.

Jayaraman, S., Gilmore, R.O., & Raudies, F. (2016, May). Changes in early optic flow experiences across development and culture. Talk at the International Congress on Infant Studies (ICIS) in New Orleans, LA. HTML slides.

Gilmore, R.O. (2016, September). Open science practices have made my work better. Talk at the Penn State Psychology Cognitive Area brown bag. HTML slides.

Adamiak, W., Thomas, A.L., Patel, S.M., & Gilmore. R.O. (2015, May). Adult observers’ sensitivity to optic flow varies by pattern and speed. Poster presented at the Vision Sciences Society meeting, St. Pete’s Beach, FL. doi:10.1167/15.12.1008. PDF. Materials on Databrary.

Raudies, F. & Gilmore, R.O. (2014, May). An analysis of optic flow experienced by infants during natural activities. Poster presented at the Vision Sciences Society meeting, St. Pete Beach, FL. Journal of Vision, 14(10). 226. doi:10.1167/14.10.226. PDF

Thomas, A.L., Fesi, J.D. & Gilmore, R.O. (2014, May). Temporal and speed tuning in brain responses to local and global motion patterns. Poster presented at the Vision Sciences Society meeting, St. Pete Beach, FL. Journal of Vision, 14(10). 482. doi:10.1167/14.10.482.

Fesi, J.D., Thomas, A.L., & Gilmore, R.O. (2012, October). Distinct space-time sampling thresholds of VEP responses to optic flow. Poster presented at the Society for Neuroscience meeting, New Orleans, LA. PDF

Gilmore, R.O., Raudies, F., Kretch, K.S., Franchak, J.M., & Adolph, K.E. (2012, June). Do you see what I see? Comparing optic flow experienced by infants and their mothers. Poster presented at the International Conference on Infant Studies, Minneapolis, MN. PDF.

Fesi, J.D., Stiffler, J.R., & Gilmore, R.O. (2012, May). Speed tuning of cortical responses to 2D figures defined by motion contrast is non-uniform across contrast types. Poster presented at the Vision Sciences Society meeting, Naples, FL.

Thomas, A.L., Mancino, A.C., Elnathan, H.C., Fesi, J.D., Hwang, K.R., & Gilmore, R.O. (2012, May). Children’s cortical responses to optic flow patterns show differential tuning by pattern type, speed, scalp location, and age group. Poster presented at the Vision Sciences Society meeting, Naples, FL. PDF.

Gilmore, R.O., Raudies, F., Kretch, K.S., Franchak, J.M., & Adolph, K.E. (2012, May). Patterns of optic flow experienced by infants and their mothers during locomotion. Poster presented at the Vision Sciences Society meeting, Naples, FL. PDF.

Raudies, F., Kretch, K.S., Franchak, J.M., Mingolla, E., Gilmore, R.O., & Adolph, K.E. (2012, May). Where do mothers point their head when they walk and where do babies point their head when they are carried? Poster presented at the Vision Sciences Society meeting, Naples, FL. PDF.

Materials

Collaborators

  • Florian Raudies, Hewlett-Packard Research
  • Swapnaa Jayaraman, Indiana University
  • Amanda Thomas, Swarthmore College
  • Jeremy Fesi, U.S. Marine Research

Support

This project was supported by the National Science Foundation under grant BCS-1147440.

The Proximal Emotional Environment Project (PEEP)

A 5-year-old overhears her parents arguing loudly in the next room. She may not understand why they are arguing, but she realizes something is wrong because she perceives anger in their voices. Exposure to interpersonal conflict is consistently associated with less skillful emotion regulation in children although the mechanisms remain to be explained. Because inter-personal conflict is a heterogeneous phenomenon, investigation of the specific features of conflict that contribute to developmental pathways to emotional dysfunction and symptoms requires a process-oriented approach. In this project, we focus on brain responses to angry prosody in natural speech. We are studying young children’s neural processing of angry prosody, spoken by mothers and strangers, as a first step toward a future longitudinal study investigating how the neurocognitive processing of angry prosody mediates relations between conflict exposure in children and the development of anxiety- and anger-related symptoms.

Publications

Moriarty, P. M., Vigeant, M. C., Liu, P., Gilmore, R., Wolf, R., & Cole, P. M. (2016). Low frequency analysis of acoustical parameters of emotional speech for use with functional magnetic resonance imaging. The Journal of the Acoustical Society of America, 140(4), 3237–3238. Acoustical Society of America. Retrieved from https://doi.org/10.1121/1.4970240

Maggi, M., Cole, P., Elbich, D., Gilmore, R.O., Pérez–Edgar, K., Scherf, K.S. (submitted). Hearing emotions: School-aged children’s neural processing of the human voice and affective prosody.

Liu, P., Cole, P., Gilmore, R.O., Pérez-Edgar, K., Vigeant, M., Moriarty, P., Scherf, K.S. (submitted). Mother’s voice uniquely modulates young children’s voice-sensitive neural network.

Presentations

Stoop, T.B., Moriarty, P., Vigeant, M., Gilmore, R.O., Liu, P., & Cole, P.M. (submitted). Children’s ratings of vocal emotion intensity depend on the emotion spoken and speaker familiarity but not acoustic parameters.

Moriarty, P., Vigeant, M., Liu, P. Gilmore, R.O, & Cole, P.M. (submitted) Comparing theory, consensus, and perception to the acoustics of emotional speech.

Moriarty, P.M., Vigeant, M., Wolf, R., Gilmore, R., & Cole, P. (2018). Creation and characterization of an emotional speech database. The Journal of the Acoustical Society of America, 143(3), 1869–1869. ASA. Retrieved from https://asa.scitation.org/doi/abs/10.1121/1.5036133

Collaborators

  • Pamela Cole, Penn State
  • Koraly Perez-Edgar, Penn State
  • Suzy Scherf, Penn State
  • Michelle Vigeant, Penn State
  • Pan Liu
  • Mirella Maggi
  • Peter Moriarty, Penn State
  • Tawni O’Dell, Penn State

Materials

Support

This project has received support from the Penn State Social Sciences Research Institute and the National Institute of Mental Health under R21-MH-104547.

Eye Tracking Technologies to Characterize and Optimize Visual Attending in Down Syndrome

Down Syndrome (DS) is the most common known genetic origin of intellectual disability and has an estimated incidence of 1 in every 1000 live births. Such children face unique challenges as they enter into the school years, because the speech that was previously adequate for communication with familiar partners in supportive settings is often not sufficient for academic communication with unfamiliar partners. Indeed, 95% of parents surveyed reported that their children with DS had difficulty being understood by persons outside their immediate social circle (Kumin, 1994). This has significant implications for academic, social, and vocational success; children with limited language skills are at risk of falling behind nondisabled peers academically and experiencing social isolation. Secondary issues often arise when children experience frustration in communication, commonly in the form of challenging behaviors. All aspects of development are further compromised when these behaviors involve aggression toward others, have significant health implications when they are self-injurious, and exacerbate service costs when they necessitate extensive behavior management plans. Children with DS are in desperate need of communication interventions that provide them with the tools to succeed throughout the school years. One form of intervention is called aided augmentative and alternative communication (AAC). In typical clinical applications, aided AAC systems employ picture books, tablet-style computers that present the user with graphic symbols, and sometimes text or synthesized voice output. Because AAC relies on vision rather than sound/speech for access to the communication messages, it is critical to map out how children with DS examine and extract information from visual AAC displays. Otherwise there is the risk of implementing systems that are poorly matched to children’s skills and needs, a practice that in turn results in limited use or abandonment of the system. Few current AAC designs consider the fit between the system and the visual processing skills of users, and most are uninformed by empirical knowledge about human visual information processing. Moreover, little is known about visual processing in persons with significant communication limitations associated with DS. This research aims to improve the design of AAC displays through characterization of visual attention patterns to different AAC displays and their effects on functional use. Eye tracking - rarely used in DS - will reveal attention patterns/processes that typically go unrecorded in behavioral research. Our three-phase program will begin with eye tracking studies of visual attention under largely non-social laboratory conditions. In the next phase, we will introduce social interactions and record gaze path using mobile eye tracking technology. In the final phase, we will translate the knowledge gained in the laboratory studies to optimize functional communication in individuals with DS in performing tasks that represent typical daily life activities.

Collaborators

  • Krista Wilkinson, Penn State, Principal Investigator

Materials

Gilmore, R.O. (2017). Wilkinson-lab. Code repository on GitHub. Retrieved December 19, 2017 from https://github.com/gilmore-lab/wilkinson-lab.

Gilmore, R.O. (2017). Eye-tracking-DS. Code repository on GitHub. Retrieved December 19, 2017 from https://github.com/gilmore-lab/eye-tracking-DS.

Support

The project is supported by NICHD under R01HD083381-02.

Finished projects

Computational Symmetry

Symmetric image Symmetric image

The ability to sense regular or near-regular patterns serves critical biological needs and is equally important for computer vision and machine intelligence. Despite wide variation in the types of regularity present in natural images, research on human and computer processing of pattern regularity has focused primarily on detecting bilateral reflection symmetry, using largely atheoretical approaches. The goals of this interdisciplinary research are to i) use principles of group theory to develop a conceptual framework for understanding regularity perception and brain activation in humans, and ii) to design general computer-based symmetry detection algorithms that can operate at a level of practical usability.

Presentations

Vedak, S.C., Gilmore, R.O., Kohler, P.J., Liu, Y., & Norcia, A.M. (2015, May). The salience of low-order visual features in highly self-similar wallpaper groups. Poster presented at the Vision Sciences Society meeting, St. Pete Beach, FL. PDF. Materials on Databrary.

Thomas, A.L., Gilmore, R.O., Norcia, A.M., Liu, Y., Fesi, J.D., Hwang, K.D., Stitt, J., & Liu, J. (2012, October). Visual patterns with rotational symmetry activate distinct cortical regions. Poster presented at the Society for Neuroscience meeting, New Orleans, LA.

Collaborators

  • Yanxi Liu, Penn State Computer Science & Engineering
  • Anthony Norcia, Stanford University, Department of Psychology

Support

This project was supported by the National Science Foundation under grant IIS-1248076.