Temple 2017-02-27

Rick Gilmore 2017-02-28 09:30:55

Go with the flow: The development of behavioral sensitivity and brain responses to optic flow

Rick O. Gilmore

Support: NSF BCS-1147440, NSF BCS-1238599, NICHD U01-HD-076595

Questions

  • What is optic flow?
  • Why is optic flow important?
  • How does optic flow sensitivity develop?
  • How do brain systems for processing optic flow develop?
  • What shapes these patterns of development?

Approach

  • EEG measures of brain responses to optic flow
  • Psychophysical measures of optic flow perception
  • Empirical measures of experienced optic flow across development from head-mounted video cameras
  • Computational simulations of optic flow experiences across developmental milestones

Claims

  • Brain and behavioral responses to optic flow develop throughout childhood
    • Still immature in 5-8 year-olds
  • Changes in the statistics of of optic flow children experience shape development in infancy, and likely beyond

A pitch and a prediction

  • Open, transparent, and reproducible research practices – including open data sharing – have changed my work.
  • For the better.
  • Within 10 years (maybe 5) it will be impossible to get funded or published if you have not adopted them.
  • Don’t worry; It will be good for us and for science.

What’s and why’s

What is Optic Flow?

  • Structured pattern of visual motion generated by observer movement

Cute kid playing hide and seek wearing GoPro camera.

Types of Optic Flow

(Yu et al. 2010)

Figure from Yu et al., 2010 of MSTd receptive fields. These parse the space of different types of optic flow. You can think of them as basic features of flow.

Why is optic flow important?

  • Geometry of environment
    • Surface layout, orientation
    • Object motion
  • Direction, speed of self-motion
    • Rotation, translation
    • Visual proprioception (eye vs. head vs. body)

Flow and self-motion

How Does Optic Flow Sensitivity Develop?

(R. Gilmore et al. 2007)

4-6 mo-old infants: Larger brain responses to linear patterns.

(Hou et al. 2009)

4-6 mo-old infants: Larger brain responses to faster speeds.

(Kiorpes and Movshon 2004)

Sensitivity to slow (linear) speeds develops slowly in monkeys.

Gaps

  • Brain and behavioral responses in childhood?
  • What influences developmental shifts?
    • Why fast speeds
    • Why linear patterns?

How Do Children’s Brains Respond to Flow?

  • If infant-like: stronger responses to fast, linear flows.
  • If adult-like: stronger responses to slow, radial flows.
  • If in-between:
    • fast + radial OR
    • slow + linear

Brain responses to flow

Methods

  • Time-varying optic flow patterns.
  • Steady-state visual evoked potentials (SSVEPs).
    • Event-related EEG technique.
    • Phase-locked responses at low-order harmonics.
  • n=29 4-8 year-olds.

2 deg/s translation

4 deg/s rotation

8 deg/s radial

1F1 Results Summary

  • Highly responsive channels over right lateral cortex
  • Radial & rotation >> translation
  • Amplitude and phase differences

3F1 Results Summary

  • Highly responsive channels over medial cortex
  • Speed, but not pattern tuned, 2 < 4 = 8 deg/s
  • Amplitude and phase differences

Results Summary

  • Anatomical separation of responses
    • speed (medial)
    • vs. pattern (lateral)
  • Radial & rotation <> translation, phase & amplitude
  • Speed tuning

Developmental Effects

  • Children adult-like in many respects
    • Lateral “pattern” responses @ 1F1
    • Medial “speed” responses @ 3F1 (and 1F2)
  • Children activate fewer channels

Behavioral responses to flow

Methods

  • Time-varying optic flow
    • Radial, linear
    • {2 deg/s, 8 deg/s}
    • {5, 10, 15, 20%} coherence (adults)
    • {15, 30, 45, 60%} and {20, 40, 60, 80%} (children)

Methods

  • Side by side displays
    • Signal/noise
    • Choose side with signal
    • 2AFC, 10 s response period

Methods

Children’s responses p(correct)

Adults’ responses p(correct)

(Adamiak et al. 2015)

Statistical modeling

Speed effects in children

Speed effects in adults

(Adamiak et al. 2015)

Pattern effects in children

Pattern effects in adults

(Adamiak et al. 2015)

Behavioral Summary

  • Children’s EEG: highest to fast speeds, radial (& rotational) patterns
  • Children’s behavior: more accurate to detect fast speeds, radial patterns
  • Adults more accurate to detect slow speeds, radial patterns

What influences developmental shifts?

Potential factors

  • External
    • Environment
  • Internal
    • Posture
    • Locomotion, head, eye movements

Head mounted eye tracker data from “coupled” infant/mom dyads

Adolph, K. (2015). Active vision in passive locomotion: real-world free viewing in infants and adults. Databrary. Retrieved February 18, 2017 from http://doi.org/10.17910/B7.123

Findings

  • Infant (passengers) experience faster visual speeds than mother
  • Controlling for speed of locomotion, environment

Experienced flow across cultures

Illustrative Speed Histograms - 6 weeks

(R. O. Gilmore, Raudies, and Jayaraman 2015)

Illustrative Speed Histograms – 34 weeks

(R. O. Gilmore, Raudies, and Jayaraman 2015)

Illustrative Speed Histograms – 58 weeks

(R. O. Gilmore, Raudies, and Jayaraman 2015)

Pattern Correlation Results

(R. O. Gilmore, Raudies, and Jayaraman 2015)

Conclusions: Measuring experienced flow

  • Time stationary >> time in motion
  • Time in motion increases, faster in U.S.
  • Fast speeds, broad speed distributions
  • Linear flow >> radial or rotational flow

Simulating developmental change

\(\\begin{pmatrix}\\dot{x} \\\\ \\dot{y}\\end{pmatrix}=\\frac{1}{z} \\begin{pmatrix}-f & 0 & x\\\\ 0 & -f & y \\end{pmatrix} \\begin{pmatrix}{v\_x{}}\\\\ {v\_y{}} \\\\{v\_z{}}\\end{pmatrix}+ \\frac{1}{f} \\begin{pmatrix} xy & -(f^2+x^2) & fy\\\\ f^2+y^2 & -xy & -fy \\end{pmatrix} \\begin{pmatrix} \\omega\_{x}\\\\ \\omega\_{y}\\\\ \\omega\_{z} \\end{pmatrix}\)

Geometry of environment/observer: (x, y, z)
Translational speed: (vx, vy, vz)
Rotational speed: (ωx, ωy, *ω**z*)
Retinal flow: \((\\dot{x}, \\dot{y})\)

Parameters For Simulation

Parameter Crawling Infant Walking Infant
Eye height 0.30 m 0.60 m
Locomotor speed 0.33 m/s 0.61 m/s
Head tilt 20 deg 9 deg



(Kretch, Franchak, and Adolph 2014)

Parameters for Simulation

Geometric Feature Distance
Side wall +/- 2 m
Side wall height 2.5 m
Distance of ground plane 32 m
Field of view width 60 deg
Field of view height 45 deg

Flow Direction Distributions by Geometry & Posture

(R. O. Gilmore, Raudies, and Jayaraman 2015)

Simulated Flow Speeds (m/s)

Type of Locomotion Ground Plane Room Side Wall Two Walls
Crawling 14.41 14.42 14.43 14.62
Walking 9.38 8.56 7.39 9.18

Simulation Conclusions

  • Posture influences optic flow speeds & patterns
    • Crawling: faster speeds, more translational flow
    • Proximity to ground and pitch of head
  • Geometry matters relatively little

Summing up

  • Infants commonly experience fast, linear optic flows
    • Body size, posture, head position/stability
    • Similar patterns across cultures
  • Brain and behavioral responses to optic flow develop throughout childhood
    • Still immature in 5-8 year-olds
  • Changes in the statistics of experienced optic flow shape brain and behavioral development in infancy, and likely beyond

A good crisis?

Open, transparent, and reproducible research practices – including open data sharing – have changed my work.

Let’s not waste a good crisis

  • Databrary.org: share video, procedures/tasks, data
    • Video the best way to capture, share procedures
    • Gilmore, R. O., & Adolph, K. E. (2017, February 9). Video can make science more open, transparent, robust, and reproducible. Retrieved from http://osf.io/3kvp7
  • Open Science Framework (OSF): share non-identifiable materials, data

Let’s not waste a good crisis

Acknowledgements

Collaborators: Karen Adolph (NYU); Jeremy Fesi (U.S. Marine Corps Research); John Franchak (UC-Riverside); Swapnaa Jayaraman (Indiana University); Kari Kretch; Ennio Mingolla, (Northeastern); Florian Raudies (LinkedIn); Amanda Thomas (Swarthmore).

Support: NSF BCS-1147440, NSF BCS-1238599, NICHD U01-HD-076595

Stack

This talk was produced in RStudio version 1.0.136 on 2017-02-28. The code used to generate the slides can be found at http://github.com/gilmore-lab/temple-2017-02-27/. Information about the R Session that produced the code is as follows:

sessionInfo()
## R version 3.3.2 (2016-10-31)
## Platform: x86_64-apple-darwin13.4.0 (64-bit)
## Running under: OS X El Capitan 10.11.6
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## loaded via a namespace (and not attached):
##  [1] backports_1.0.4 magrittr_1.5    rprojroot_1.1   htmltools_0.3.5
##  [5] tools_3.3.2     yaml_2.1.14     Rcpp_0.12.8     stringi_1.1.2  
##  [9] rmarkdown_1.3   knitr_1.15.1    stringr_1.1.0   digest_0.6.11  
## [13] evaluate_0.10

References

Adamiak, William, Amanda Thomas, Shivani Patel, and Rick Gilmore. 2015. “Adult Observer’s Sensitivity to Optic Flow Varies by Pattern and Speed.” Journal of Vision 15 (12): 1008. doi:[10.1167/15.12.1008](https://doi.org/10.1167/15.12.1008).

Gilmore, R. O., F. Raudies, and S. Jayaraman. 2015. “What Accounts for Developmental Shifts in Optic Flow Sensitivity?” In 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), 19–25. doi:[10.1109/DEVLRN.2015.7345450](https://doi.org/10.1109/DEVLRN.2015.7345450).

Gilmore, R.O., C. Hou, M.W. Pettet, and A.M. Norcia. 2007. “Development of Cortical Responses to Optic Flow.” Visual Neuroscience 24 (06): 845–56. doi:[10.1017/S0952523807070769](https://doi.org/10.1017/S0952523807070769).

Gilmore, R.O., A.L. Thomas, and J. Fesi. 2016. “Children’s Brain Responses to Optic Flow Vary by Pattern Type and Motion Speed.” PLOS ONE 11 (6): e0157911. doi:[10.1371/journal.pone.0157911](https://doi.org/10.1371/journal.pone.0157911).

Hou, C., R.O. Gilmore, M.W. Pettet, and A.M. Norcia. 2009. “Spatio-Temporal Tuning of Coherent Motion Evoked Responses in 4–6 Month Old Infants and Adults.” Vision Research 49 (20): 2509–17. doi:[10.1016/j.visres.2009.08.007](https://doi.org/10.1016/j.visres.2009.08.007).

Jouen, François, Jean-Claude Lepecq, Olivier Gapenne, and Bennett I Bertenthal. 2000. “Optic Flow Sensitivity in Neonates.” Infant Behavior and Development 23 (3–4): 271–84. doi:[10.1016/S0163-6383(01)00044-3](https://doi.org/10.1016/S0163-6383(01)00044-3).

Kiorpes, Lynne, and J. Anthony Movshon. 2004. “Development of Sensitivity to Visual Motion in Macaque Monkeys.” Visual Neuroscience 21 (6): 851–59. doi:[10.1017/S0952523804216054](https://doi.org/10.1017/S0952523804216054).

Kretch, Kari S., John M. Franchak, and Karen E. Adolph. 2014. “Crawling and Walking Infants See the World Differently.” Child Development 85 (4): 1503–18. doi:[10.1111/cdev.12206](https://doi.org/10.1111/cdev.12206).

Raudies, F., and R.O. Gilmore. 2014. “Visual Motion Priors Differ for Infants and Mothers.” Neural Computation 26 (11): 2652–68. doi:[10.1162/NECO\_a\_00645](https://doi.org/10.1162/NECO_a_00645).

Yu, Chen Ping, William K. Page, Roger Gaborski, and Charles J. Duffy. 2010. “Receptive Field Dynamics Underlying MST Neuronal Optic Flow Selectivity.” Journal of Neurophysiology 103 (5): 2794–2807. doi:[10.1152/jn.01085.2009](https://doi.org/10.1152/jn.01085.2009).