Stanford Seminar - Driving Exploratory Visualization through Perception & Cognition

00:58:12
https://www.youtube.com/watch?v=0Nr5m683wGI

摘要

TLDRThe talk delves into how data visualization is fundamentally transforming engagement with the world, emphasizing the necessity of effective visual tools for decision making. It examines the cognitive processes involved in interpreting visual information and presents a structured research approach to develop novel visualization systems tailored to diverse audiences. The research includes automating visualization design, enhancing the accessibility of data representation for individuals with intellectual disabilities, and integrating augmented reality for field data analysis. Each of these threads underscores the importance of perceptual understanding in creating insightful visual representations.

心得

  • 🌐 Data is transforming decision-making across disciplines.
  • 🔍 Effective visualization tools enhance exploratory data analysis.
  • 🎨 Color choice is critical for accurate data interpretation.
  • 💬 Accessibility is vital for including individuals with IDD in data discussions.
  • 📈 AR technology can provide contextual insights during field analysis.
  • 🔬 Research involves empirical studies for optimal visualization methods.
  • 👩‍💻 Non-experts can benefit from automation in visualization design.
  • 🧠 Understanding cognitive processes is essential for effective visual presentations.

时间轴

  • 00:00:00 - 00:05:00

    Data is significantly influencing our engagement with the world, playing a vital role in public communication and decision-making. My research investigates the use of visualization tools for exploratory data analysis and their impact on various fields, enabling better understanding and decision-making across disciplines.

  • 00:05:00 - 00:10:00

    The rise of big data comes with challenges, including understanding a greater variety of users, data complexity, and the increasing need for intuitive data analysis tools. Visualization is becoming commonplace even among non-experts, making it crucial to develop effective communication methods for varying audiences.

  • 00:10:00 - 00:15:00

    Despite advancements in machine learning and computational tools, exploratory data analysis remains essential. Visualization helps in understanding data patterns that raw statistics might miss, highlighting the need for a human-centric approach to data interpretation.

  • 00:15:00 - 00:20:00

    Through an example, I illustrate how first-order statistics can be misleading: datasets with identical statistics can show vastly different distributions when visualized, emphasizing the power of visualization in discerning qualitative differences.

  • 00:20:00 - 00:25:00

    The process of deriving insights from visualization involves complex cognitive and perceptual processes that translate visual data into knowledge. My research models these processes to improve data visualization techniques and enhance analytical effectiveness.

  • 00:25:00 - 00:30:00

    Our research includes understanding domain-specific challenges and determining how optimal representations can enhance analysis. We derive models, conduct empirical studies, and create tools to optimize data visualization and exploratory systems for various applications.

  • 00:30:00 - 00:35:00

    We're focusing on three key areas: automating effective visualization design, overcoming cognitive barriers in visualization for individuals with intellectual disabilities, and integrating augmented reality with analytics for real-time situational analysis during field operations.

  • 00:35:00 - 00:40:00

    Designing effective visualizations, particularly around color selection, involves understanding both aesthetic and perceptual aspects. Poor color choices can lead to misinterpretation, highlighting the critical nature of careful design in visual analytics.

  • 00:40:00 - 00:45:00

    Through collaboration, we've developed methodologies to model color usage in visualizations, leading to better algorithms that can aid non-experts in making effective color choices based on perceived data differences.

  • 00:45:00 - 00:50:00

    We've created tools that maximize data representation effectiveness while allowing customization based on user needs. These tools leverage perceptual insights to facilitate better visual communication and improve the accessibility of data interpretation.

  • 00:50:00 - 00:58:12

    Ongoing research includes evaluating how traditional visualization practices may hinder accessibility for individuals with intellectual disabilities, aiming to redefine guidelines to better serve diverse user groups in terms of data representation.

显示更多

思维导图

视频问答

  • Why is visualization important in data analysis?

    Visualization helps users explore information using their sense of sight, facilitating quicker and deeper insights from complex data.

  • What challenges are associated with color use in data visualization?

    Choosing effective color encodings is difficult as poor color choices can lead to misinterpretations, sometimes affecting critical decisions in fields like medicine.

  • How does the research address accessibility for individuals with intellectual disabilities (IDD)?

    The research identifies existing barriers in visualization practices for IDD and proposes design guidelines to enhance their understanding and engagement with data.

  • What is the role of augmented reality in data visualization?

    Augmented reality is used to provide contextually rich environments for data analysis, allowing users to interact with data in ways that traditional displays can't achieve.

  • How do different visualization modalities impact user perception?

    Different environments, such as AR and VR, affect how users interpret visual information, including depth cues, color contrasts, and overall engagement.

  • What are some principles for effective visualization design?

    Effective visualization design considers perceptual processing, cognitive load, and the contextual relevance of data being presented.

  • Can non-experts effectively create visualizations?

    The research aims to develop algorithms and guidelines that empower non-experts to produce high-quality visualizations with minimal expertise.

  • How does this research relate to the broader context of data science?

    The work explores innovative methods to enhance data visualization practices, ultimately contributing to data literacy and improving decision-making across various domains.

查看更多视频摘要

即时访问由人工智能支持的免费 YouTube 视频摘要！
字幕
en
自动滚动:
  • 00:00:11
    so it goes without saying that data is
  • 00:00:13
    fundamentally changing the way we see it
  • 00:00:15
    engage with the world over the past year
  • 00:00:17
    especially data has become a daily
  • 00:00:19
    fixture in public communication driving
  • 00:00:22
    everything from personal decision making
  • 00:00:23
    to public policy
  • 00:00:25
    and visualization provides everyone from
  • 00:00:28
    scientists to humanists to everyday
  • 00:00:29
    people with a means for readily
  • 00:00:31
    exploring information using our sense of
  • 00:00:33
    sight and our intuitions
  • 00:00:35
    so what my research explores is how we
  • 00:00:37
    can leverage the brain's ability to make
  • 00:00:39
    sense of visual information to design
  • 00:00:41
    novel visualization tools for
  • 00:00:43
    interactive exploratory data analysis
  • 00:00:46
    you can see a subset of these projects
  • 00:00:47
    here and with a fantastic team of
  • 00:00:49
    students and collaborators we work with
  • 00:00:51
    stakeholders to help analysts in domains
  • 00:00:53
    ranging from literary scholarship
  • 00:00:56
    to biology to emergency response
  • 00:00:58
    to make sense of their data and to guide
  • 00:01:00
    more effective and efficient decision
  • 00:01:02
    making and data exploration
  • 00:01:06
    so when we think about data science one
  • 00:01:08
    of the first things that comes to mind
  • 00:01:10
    is this idea of big data but we're also
  • 00:01:12
    seeing challenges in terms of new users
  • 00:01:15
    new data variety and complexity and new
  • 00:01:17
    questions that people want to tackle
  • 00:01:19
    with that data more and more people are
  • 00:01:21
    looking to leverage data to the point
  • 00:01:22
    where graphs and charts become a common
  • 00:01:24
    fixture on the evening news
  • 00:01:26
    and more of these folks are approaching
  • 00:01:27
    data without formal training and
  • 00:01:29
    analytics
  • 00:01:30
    further we're developing techniques to
  • 00:01:32
    bring data together from different
  • 00:01:34
    sources to enrich our perspectives and
  • 00:01:36
    applications ranging from social media
  • 00:01:37
    analysis to personalized medicine and in
  • 00:01:41
    bringing new people and new data forward
  • 00:01:44
    we're also raising new questions to be
  • 00:01:46
    asked of that data
  • 00:01:48
    now many of you might be sitting there
  • 00:01:50
    thinking so what why can't we just
  • 00:01:52
    compute the right answer and not worry
  • 00:01:54
    about any of these problems well we've
  • 00:01:56
    seen unprecedented advances in machine
  • 00:01:58
    learning that offer us really powerful
  • 00:02:00
    computational tools
  • 00:02:02
    i don't want to sell that short
  • 00:02:04
    but i also want to argue that we still
  • 00:02:05
    need to bring people into exploratory
  • 00:02:07
    analysis we don't always know what we're
  • 00:02:10
    looking for and statistics while
  • 00:02:12
    incredibly powerful don't always give us
  • 00:02:15
    the information that we need
  • 00:02:17
    so let me illustrate this with a quick
  • 00:02:19
    example here are four first order
  • 00:02:21
    statistics for four different data sets
  • 00:02:24
    would you say these four distributions
  • 00:02:26
    are pretty similar
  • 00:02:29
    many of you have probably guessed that
  • 00:02:31
    i'm messing with you a little bit here
  • 00:02:34
    so all of these data sets have the same
  • 00:02:35
    first order statistics but when i
  • 00:02:38
    visualize the data sets they have
  • 00:02:40
    qualitatively very different structures
  • 00:02:42
    and we can see that immediately here we
  • 00:02:44
    have linear trends constants with
  • 00:02:46
    outliers and parabolic structures we
  • 00:02:48
    have words to describe how this data
  • 00:02:50
    behaves and we qualitatively understand
  • 00:02:52
    the differences even if we can't readily
  • 00:02:55
    enumerate them
  • 00:02:56
    but what's happening here
  • 00:02:58
    how do we build knowledge and data using
  • 00:03:00
    visualizations that we can't with
  • 00:03:01
    statistics so i'm going to focus on just
  • 00:03:04
    these two scatter plots to try to
  • 00:03:05
    explain this
  • 00:03:07
    the process of interpreting data happens
  • 00:03:09
    over time it starts with our sense of
  • 00:03:12
    sight essentially light from the red and
  • 00:03:14
    the orange points goes into the eye
  • 00:03:16
    fires off a set of signals to start the
  • 00:03:18
    process of interpreting information
  • 00:03:20
    and at the end of this process we reach
  • 00:03:22
    the far side of the spectrum insights
  • 00:03:25
    the knowledge that we generate using
  • 00:03:27
    visualizations
  • 00:03:28
    for example one insight we might infer
  • 00:03:30
    from this data is while these data sets
  • 00:03:32
    have identical statistics they have
  • 00:03:34
    qualitatively different shapes that we
  • 00:03:36
    can describe as either linear or
  • 00:03:38
    parabolic
  • 00:03:39
    but how do we get from our sense of
  • 00:03:41
    sight to insight from structures and
  • 00:03:43
    patterns to knowledge what is going on
  • 00:03:45
    in between well i want to argue that by
  • 00:03:48
    understanding the perceptual and
  • 00:03:49
    cognitive processes between sites and
  • 00:03:52
    insights we can start to understand how
  • 00:03:55
    the ways we visualize data determine the
  • 00:03:56
    knowledge that people can infer from
  • 00:03:58
    that data
  • 00:03:59
    and my work models how people generate
  • 00:04:01
    knowledge from visualization and uses
  • 00:04:03
    these models to create new tools and
  • 00:04:05
    techniques for analyzing data
  • 00:04:08
    to give you the high level overview of
  • 00:04:10
    our research process we generally
  • 00:04:12
    achieve these ideas in three phases the
  • 00:04:15
    first involves discussions with domain
  • 00:04:16
    experts to identify what the problem
  • 00:04:19
    they're trying to solve is
  • 00:04:20
    and what the current limitations are in
  • 00:04:22
    their existing approaches
  • 00:04:24
    we then conduct empirical studies to
  • 00:04:26
    model how we can optimize our
  • 00:04:27
    representations
  • 00:04:29
    based on needs and context of the
  • 00:04:31
    analysis problem and we use this data to
  • 00:04:33
    drive new algorithms for supporting
  • 00:04:35
    interactive data exploration and embody
  • 00:04:38
    these models and approaches in
  • 00:04:39
    interactive exploratory analysis systems
  • 00:04:42
    where we develop analysis tools and work
  • 00:04:44
    with stakeholders to iterate on those
  • 00:04:46
    tools and optimize for their particular
  • 00:04:49
    sets of needs
  • 00:04:51
    so we've used this process to create new
  • 00:04:53
    models guidelines algorithms and systems
  • 00:04:55
    across a broad range of applications you
  • 00:04:57
    can see the list of broader areas that
  • 00:05:00
    we're currently working on in the lab
  • 00:05:01
    here i'm happy to jump in in individual
  • 00:05:04
    conversations and talk about any of
  • 00:05:06
    these areas or other future areas that
  • 00:05:09
    might be fun opportunities for us all to
  • 00:05:11
    collaborate
  • 00:05:12
    but for today i really want to focus on
  • 00:05:15
    three specific problem areas that
  • 00:05:18
    illustrate this process
  • 00:05:20
    so these areas the first of these areas
  • 00:05:23
    is automating effective visualization
  • 00:05:24
    design and this is a problem that
  • 00:05:26
    pervades across domains and it's where
  • 00:05:28
    i'll spend most of my times a day
  • 00:05:30
    the second of these investigates the
  • 00:05:32
    cognitive barriers that current
  • 00:05:33
    visualization practices create trying to
  • 00:05:36
    figure out how we can make data more
  • 00:05:38
    accessible for people with intellectual
  • 00:05:39
    and developmental disabilities in the
  • 00:05:42
    context of fiscal self-advocacy
  • 00:05:44
    and the last thread that i'll talk about
  • 00:05:46
    explores how augmented reality mobile
  • 00:05:48
    data cloud computing and immersive
  • 00:05:50
    analytics all come together to support
  • 00:05:51
    situated data analysis in field
  • 00:05:54
    operations in earth science and
  • 00:05:55
    emergency response
  • 00:05:58
    so to get started designing effective is
  • 00:06:01
    is just thought out a challenging
  • 00:06:03
    problem something as simple as choosing
  • 00:06:04
    the right colors requires understanding
  • 00:06:06
    how people perceive color how the
  • 00:06:08
    semantics of color align with data how
  • 00:06:10
    the relationships between colors
  • 00:06:12
    highlight key patterns
  • 00:06:13
    and if that wasn't already enough
  • 00:06:15
    choosing colors that actually look good
  • 00:06:17
    together
  • 00:06:18
    and this is an important challenge is
  • 00:06:20
    color is among the most common methods
  • 00:06:21
    for encoding data it's first off pretty
  • 00:06:24
    and our brains are actually really
  • 00:06:26
    really good at synthesizing it so in my
  • 00:06:28
    dissertation work i constructed several
  • 00:06:30
    systems for applications in biology that
  • 00:06:32
    showed how color usages can increase
  • 00:06:34
    scalability by 10-fold or more
  • 00:06:37
    however we see color used in all kinds
  • 00:06:40
    of visualizations from scatter plots to
  • 00:06:42
    surfaces to maps and beyond
  • 00:06:44
    but as i mentioned
  • 00:06:46
    color is really hard to do writes poorly
  • 00:06:48
    designed color encodings may not just
  • 00:06:50
    look bad
  • 00:06:51
    but can lead to misinterpretation and
  • 00:06:54
    have even caused papers to be withdrawn
  • 00:06:55
    from top journals like nature and
  • 00:06:57
    science
  • 00:06:58
    work with doctors done by michelle
  • 00:07:00
    barkin and her colleagues show that
  • 00:07:02
    effective color choices can actually
  • 00:07:04
    improve diagnosis rates by 30 or more so
  • 00:07:07
    choosing good colors can actually become
  • 00:07:09
    a matter of life and death
  • 00:07:11
    two of the biggest things that make
  • 00:07:13
    coloring coatings hard are that we don't
  • 00:07:15
    have good models for mapping
  • 00:07:17
    color differences to data differences
  • 00:07:20
    and even with the models that we do have
  • 00:07:23
    using color well currently requires
  • 00:07:25
    substantial design expertise
  • 00:07:27
    so in collaboration with my students as
  • 00:07:29
    well as folks at tableau iupui and lanl
  • 00:07:32
    what i've looked to do is figure out how
  • 00:07:35
    can we better model and understand what
  • 00:07:37
    it means to use color well in order to
  • 00:07:39
    drive algorithmic solutions that help
  • 00:07:42
    people use color more effectively
  • 00:07:45
    so a lot of my work in the space fits
  • 00:07:47
    into two primary threads
  • 00:07:49
    how do people see colors and biz and how
  • 00:07:51
    do we design algorithms that help people
  • 00:07:53
    use color more effectively
  • 00:07:55
    a lot of this has been very
  • 00:07:56
    methodological focusing on how we can
  • 00:07:58
    model the ways people perceive colors
  • 00:08:00
    directly as a function of the design
  • 00:08:02
    components of the visualization so what
  • 00:08:05
    these techniques aim to do is let us use
  • 00:08:07
    experimental data to probabilistically
  • 00:08:09
    model how precisely people can discern
  • 00:08:11
    color differences across a wide range of
  • 00:08:14
    visualization designs and once we
  • 00:08:16
    figured out how people see differences
  • 00:08:18
    in color we can look more globally
  • 00:08:20
    developing metrics and algorithms for
  • 00:08:22
    choosing sets of color that effectively
  • 00:08:24
    communicate properties of our data
  • 00:08:26
    for today i'll focus more on the
  • 00:08:27
    algorithmic side of this work
  • 00:08:30
    specifically exploring how we can use
  • 00:08:31
    data mining practices to automatically
  • 00:08:33
    bootstrap color encoding design
  • 00:08:38
    so to set the stage for a lot of this
  • 00:08:40
    work the viz community has historically
  • 00:08:42
    used the cie lab color space rather than
  • 00:08:44
    rgb as a way to map color to data
  • 00:08:48
    lab consists of three axes that
  • 00:08:49
    essentially mimic the cells in the eye
  • 00:08:51
    and we can use lab for viz is unlike rgb
  • 00:08:54
    it's what we call approximately
  • 00:08:56
    perceptually linear that is one unit of
  • 00:08:58
    euclidean distance
  • 00:09:00
    approximately corresponds to one fifty
  • 00:09:02
    percent just noticeable for j and d
  • 00:09:05
    which is the smallest difference that
  • 00:09:06
    people can reliably distinguish and rate
  • 00:09:09
    greater than chance
  • 00:09:11
    so ca the cia lab gives us this really
  • 00:09:13
    nice one-to-one mapping between data and
  • 00:09:15
    color differences that's critical for
  • 00:09:17
    data analysis and that in practice fails
  • 00:09:20
    absolutely miserably so let me show you
  • 00:09:23
    how
  • 00:09:24
    cie lab states that we should be able to
  • 00:09:26
    see seven different colors in this bar
  • 00:09:29
    chart
  • 00:09:31
    no need to adjust your monitor as you
  • 00:09:33
    can probably see this expectation just
  • 00:09:35
    simply doesn't pan out
  • 00:09:37
    so this designers have hand tuned color
  • 00:09:39
    sets for data representation using
  • 00:09:41
    decades of experience to employ these
  • 00:09:43
    models as heuristics in crafting better
  • 00:09:46
    encodings
  • 00:09:47
    by essentially just using well-defined
  • 00:09:49
    and well-developed intuitions however
  • 00:09:52
    even these encodings can start to break
  • 00:09:54
    down as we change the size of our marks
  • 00:09:57
    or even as we change the kinds of
  • 00:09:59
    visualizations that we're using
  • 00:10:02
    and encodings we want to figure out how
  • 00:10:04
    do we balance discriminability the
  • 00:10:06
    ability to preserve important data
  • 00:10:08
    differences with the range of desired
  • 00:10:10
    data differences we need to encode based
  • 00:10:12
    on the data sets that we have
  • 00:10:14
    so to achieve this goal we created a
  • 00:10:17
    different method to proactively
  • 00:10:18
    anticipate how color difference
  • 00:10:20
    perceptions change as a function of the
  • 00:10:22
    visualizations that we use
  • 00:10:24
    that is we wanted to create models that
  • 00:10:26
    allow us to match the differences we see
  • 00:10:28
    in colors to the differences that exist
  • 00:10:30
    in our data we developed three different
  • 00:10:32
    models using this approach the first
  • 00:10:34
    focusing on what we call diagonally
  • 00:10:36
    symmetric marks like points in a scanner
  • 00:10:38
    plot the second on elongated marks like
  • 00:10:40
    bars and a bar chart and the third on
  • 00:10:42
    asymmetric elongated marks like lines in
  • 00:10:45
    a line graph in total we collected over
  • 00:10:47
    34 000 data points to see these models
  • 00:10:50
    across a range of crop reviewers
  • 00:10:52
    for the sake of time i'm going to skip
  • 00:10:54
    the nitty gritty of these methods i'm
  • 00:10:56
    more than happy to chat about the
  • 00:10:57
    details of these experiments online or
  • 00:10:59
    you can check them out in the paper
  • 00:11:00
    but i do want to touch on what it is
  • 00:11:03
    that we found
  • 00:11:05
    so our studies show that our ability to
  • 00:11:07
    perceive differences in data encoded by
  • 00:11:09
    color vary according to the design of
  • 00:11:11
    our visualization and in particular
  • 00:11:27
    to elongation are asymptotic but could
  • 00:11:29
    be quite dramatic so this dependent
  • 00:11:31
    models predict that we lose about 70 of
  • 00:11:34
    the data differences predicted by cie
  • 00:11:36
    lab in practice
  • 00:11:38
    but when we consider the geometries of a
  • 00:11:40
    data point we can actually buy back
  • 00:11:42
    about 30 percent of that difference
  • 00:11:43
    meaning we can resolve differences a lot
  • 00:11:45
    more reliably if we choose colors that
  • 00:11:48
    are tailored to our visualization design
  • 00:11:51
    and the other cool bit with these
  • 00:11:52
    results is just how cleanly we can make
  • 00:11:54
    these predictions we actually were able
  • 00:11:57
    to compute normalizing constants using a
  • 00:11:59
    range of allowable mark sizes the feeden
  • 00:12:01
    or euclidean distance formulation in
  • 00:12:03
    order to predict with a surprisingly
  • 00:12:05
    high degree of precision how well people
  • 00:12:07
    could actually discern color-coded data
  • 00:12:09
    in bits
  • 00:12:12
    so one of the ways we might use these
  • 00:12:14
    models is to evaluate our own practices
  • 00:12:16
    and understand where current approaches
  • 00:12:18
    may fall
  • 00:12:19
    so you'll recall that designers often
  • 00:12:21
    will hand adjust for the short ci lab
  • 00:12:25
    and most color invis comes from a tool
  • 00:12:27
    called color brewer
  • 00:12:29
    ramps and color are hand designed
  • 00:12:31
    typically considered the gold standard
  • 00:12:34
    set of encodings available for people to
  • 00:12:36
    use so we can use our computational
  • 00:12:39
    models to predict the effectiveness of
  • 00:12:42
    existing encodings based on known
  • 00:12:44
    minimum parameters of the viz for
  • 00:12:46
    example imagine that we have scatter
  • 00:12:47
    plots with 10 pixel wide points and line
  • 00:12:49
    graphs with four pixel thick lines these
  • 00:12:51
    roughly correspond to the standards that
  • 00:12:53
    we see
  • 00:12:54
    on a 15 inch laptop
  • 00:12:56
    so we can use our models to anticipate
  • 00:12:58
    which of our nine step brewer sequential
  • 00:13:01
    encodings that designer that we're
  • 00:13:03
    designed for using cartography
  • 00:13:05
    will be robust to these scatter plots
  • 00:13:07
    and line graphs that is which will allow
  • 00:13:09
    us to distinguish between each of our
  • 00:13:11
    nine steps and which are likely to
  • 00:13:13
    create ambiguities in our data
  • 00:13:15
    using our models we actually found that
  • 00:13:17
    13 of the 18 color brewery encodings
  • 00:13:20
    actually failed to preserve sufficient
  • 00:13:22
    color differences for these
  • 00:13:23
    visualizations
  • 00:13:25
    and so this failure of these ramps
  • 00:13:27
    suggests the need to consider markerware
  • 00:13:29
    models in encoding design we're actually
  • 00:13:31
    losing much of the fidelity that we
  • 00:13:33
    assume even in expert crafted designs
  • 00:13:36
    due to assumptions about fixed
  • 00:13:37
    discriminability embedded in
  • 00:13:39
    conventional models
  • 00:13:41
    so this kind of creates a big problem it
  • 00:13:43
    suggests that we need to rethink the
  • 00:13:45
    ways that we craft ramps
  • 00:13:49
    so as we saw earlier designers have a
  • 00:13:51
    fine-tuned sense of how to build good
  • 00:13:54
    coloring coatings but
  • 00:13:56
    even they don't have it perfectly
  • 00:13:57
    correct yes
  • 00:13:59
    however it's really hard for the vast
  • 00:14:01
    majority of us to generate colors with
  • 00:14:02
    high perceptual and aesthetic quality
  • 00:14:05
    it's just hard to do well and it's also
  • 00:14:07
    incredibly time consuming so what's the
  • 00:14:10
    average visualization developer to do or
  • 00:14:12
    to put this more simply how do we choose
  • 00:14:14
    the right colors for our data especially
  • 00:14:16
    as data grows in both size and
  • 00:14:18
    complexity
  • 00:14:19
    well one approach we can use is to apply
  • 00:14:22
    our models to proactively fix encoding
  • 00:14:25
    challenges to encourage effective
  • 00:14:26
    visualizations support experimentation
  • 00:14:29
    and even conduct postdoc correction
  • 00:14:31
    so for example if we design a scatter
  • 00:14:33
    plot like we see here we like our color
  • 00:14:35
    choices
  • 00:14:36
    we may actually lose important
  • 00:14:38
    differences in the data as our points
  • 00:14:40
    shrink due to factors like added data or
  • 00:14:42
    smaller displays
  • 00:14:44
    we can use our models to anticipate when
  • 00:14:46
    this will happen push apart the ends of
  • 00:14:48
    our encoding
  • 00:14:50
    and then re-interpolate the encoding to
  • 00:14:52
    preserve uniformity
  • 00:14:54
    similarly if our visualizations change
  • 00:14:56
    we can use our metrics to pull colors
  • 00:14:58
    closer together to make better use of
  • 00:15:00
    the available encoding space and allow
  • 00:15:02
    more room for aesthetic choice
  • 00:15:05
    so this approach is great if you have a
  • 00:15:07
    solid baseline selecting from a set of
  • 00:15:09
    predefined tools
  • 00:15:11
    might also lead to a little bit of deja
  • 00:15:13
    vu
  • 00:15:14
    as you can see in this set of visits
  • 00:15:15
    from the 2018 proceedings of the visual
  • 00:15:17
    analytics track at the top biz
  • 00:15:19
    conference we use color a lot and maybe
  • 00:15:22
    rely a bit more heavily than we realize
  • 00:15:24
    on can ramps like those created by color
  • 00:15:26
    brewer
  • 00:15:28
    in fact in just a brief survey of that
  • 00:15:30
    track we found 25 percent of papers from
  • 00:15:32
    that proceeding contained at least one
  • 00:15:34
    viz with a red blue ramp
  • 00:15:36
    so essentially this is causing your
  • 00:15:38
    visualization to blend in with the crowd
  • 00:15:41
    even if you're okay with your viz not
  • 00:15:43
    standing out there are situations where
  • 00:15:45
    we may need unique colors for example if
  • 00:15:47
    we're working on branded presentations
  • 00:15:49
    we may need to use the colors associated
  • 00:15:50
    with that brand
  • 00:15:52
    but there's not exactly a ton of demand
  • 00:15:54
    out there for a lift pink ramp well so
  • 00:15:56
    what are we to do well this leads us to
  • 00:15:59
    the challenge we've tackled in this work
  • 00:16:00
    which is how do we enable people to
  • 00:16:03
    create effective custom qual ramps
  • 00:16:07
    to solve this we developed a simple
  • 00:16:08
    algorithm that lets you create high
  • 00:16:10
    quality um ramps using just a single
  • 00:16:13
    guiding color so in this case let's use
  • 00:16:15
    skype blue
  • 00:16:16
    well we can accomplish this by computing
  • 00:16:18
    characteristic structures in designer
  • 00:16:21
    ramps
  • 00:16:22
    interpolated in a perceptual color space
  • 00:16:25
    we can then apply those ramps to a given
  • 00:16:27
    color to generate a set of color ups
  • 00:16:29
    with different visual characteristics
  • 00:16:31
    that developers can select from and
  • 00:16:33
    refine
  • 00:16:36
    so most color ramp design methods lie
  • 00:16:38
    along a spectrum from tools that offer
  • 00:16:40
    pre-designed palettes like color brewer
  • 00:16:42
    to those that allow full control of the
  • 00:16:43
    user like photoshop so selecting from a
  • 00:16:46
    set of chan ramps is great
  • 00:16:48
    if one of the handful of ramps that you
  • 00:16:51
    can choose from meets your needs exactly
  • 00:16:53
    but you have no means for adjusting them
  • 00:16:56
    what you see really is what you get
  • 00:16:58
    on the other hand tools like color
  • 00:17:00
    picker for data seen here on the right
  • 00:17:02
    provide a lot more agency to the user
  • 00:17:04
    you can choose two endpoints in a
  • 00:17:06
    perceptual space and interpolate those
  • 00:17:08
    endpoints at equal intervals
  • 00:17:10
    but you have no guidance for choosing
  • 00:17:12
    those colors
  • 00:17:14
    and this all gets to the interesting and
  • 00:17:16
    nuanced recommendations we have for
  • 00:17:18
    building effective color ramps so what i
  • 00:17:20
    have listed out here is a set of 17 best
  • 00:17:23
    practices from a recent survey exploring
  • 00:17:25
    decades of color use and fizz however
  • 00:17:27
    coloring design is really hard it
  • 00:17:29
    combines aspects of perception and
  • 00:17:31
    aesthetics in ways that have to play
  • 00:17:32
    nice with each other and many of these
  • 00:17:34
    concepts are heuristics that require
  • 00:17:36
    substantial design expertise to apply
  • 00:17:39
    effectively
  • 00:17:40
    so i'm going to remove all the
  • 00:17:42
    guidelines that fall into this heuristic
  • 00:17:43
    category and leave only the concrete
  • 00:17:46
    mathematical guidelines
  • 00:17:49
    now that i have these i'm going to
  • 00:17:51
    remove those that are really hard to
  • 00:17:53
    implement to design time without
  • 00:17:55
    substantial color science expertise
  • 00:17:59
    and here's our results
  • 00:18:02
    so hopefully you're seeing now why
  • 00:18:04
    creating ramps is pretty hard
  • 00:18:07
    and even if you have the expertise to
  • 00:18:09
    follow many of these practices we've
  • 00:18:11
    actually found that in practice
  • 00:18:13
    designers don't typically follow many of
  • 00:18:15
    these rules
  • 00:18:17
    so what are we going to do well
  • 00:18:20
    we turned instead to a concept called
  • 00:18:22
    design mining which is an approach that
  • 00:18:24
    uses data mining to model and reproduce
  • 00:18:26
    design practices and we use this design
  • 00:18:28
    mining approach to capture the
  • 00:18:30
    properties of effective color use in
  • 00:18:32
    visualization
  • 00:18:33
    our algorithm relies on the fundamental
  • 00:18:35
    observation that ramps are essentially
  • 00:18:37
    just curves constructed and interpolated
  • 00:18:39
    in a 3d space
  • 00:18:40
    so we assert that the structure of the
  • 00:18:42
    curve controls the aesthetic appearance
  • 00:18:44
    of a ramp and the way is that we sample
  • 00:18:46
    the curve control the perceptual
  • 00:18:48
    properties of the ramp
  • 00:18:50
    these curves could traverse the 3dci lab
  • 00:18:52
    volume but that gets a little
  • 00:18:54
    mind-bending when you're trying to look
  • 00:18:56
    at these volumes on a 2d display so i'm
  • 00:18:58
    going to just show these as a series of
  • 00:19:00
    projected 2d curves
  • 00:19:02
    one in hue space on the left and one in
  • 00:19:04
    lightness and chroma on the right
  • 00:19:07
    so our algorithm consists of six steps
  • 00:19:09
    roughly categorized into three phases
  • 00:19:12
    the first two steps create a uniform
  • 00:19:14
    training corpus we use to derive our
  • 00:19:15
    models
  • 00:19:16
    the second to cluster the curves based
  • 00:19:19
    on their structural properties and
  • 00:19:20
    compute characteristic structures
  • 00:19:22
    describing each cluster
  • 00:19:24
    and the final pair takes these clusters
  • 00:19:26
    from an abstract space and anchors them
  • 00:19:29
    in color space to generate our encodings
  • 00:19:31
    i'll walk through each of these at a
  • 00:19:32
    high level here but as always happy to
  • 00:19:35
    dive into more detail after the talk
  • 00:19:37
    so we started with a corpus of 222
  • 00:19:41
    handcrafted color ramps from known
  • 00:19:42
    sources in the community including color
  • 00:19:44
    lovers tableau r and color brewer and we
  • 00:19:47
    use this corpus as the training set for
  • 00:19:49
    our model
  • 00:19:51
    we normalized each of these ramps by
  • 00:19:53
    treating the colors in the ramp as
  • 00:19:54
    control points and fitting interpolating
  • 00:19:57
    b splines to these points once we had
  • 00:19:59
    our splines constructed we used arc
  • 00:20:01
    length interpolation to resample the
  • 00:20:03
    curves to uniform number of points along
  • 00:20:05
    roughly equidistant intervals in cie lab
  • 00:20:10
    we clustered the normalized curves
  • 00:20:12
    according to their physical structures
  • 00:20:14
    again more details about this
  • 00:20:15
    mathematically are in the paper but i
  • 00:20:17
    want to show you a little bit of the
  • 00:20:18
    results so here the 2d projected plots
  • 00:20:21
    of these curves are shown in gray and
  • 00:20:23
    the basic idea is that we can align
  • 00:20:26
    these curves by moving rotating and
  • 00:20:28
    reflecting them to minimize the overall
  • 00:20:29
    distance between control points
  • 00:20:32
    we then use either acadian's clustering
  • 00:20:34
    based on structural features of each
  • 00:20:36
    curve or a bayesian clustering algorithm
  • 00:20:38
    applied to an elastic shape descriptor
  • 00:20:40
    to group curves sharing similar
  • 00:20:42
    structures
  • 00:20:43
    once we've done this grouping we can
  • 00:20:45
    compute a characteristic curve for each
  • 00:20:47
    cluster
  • 00:20:48
    by identifying the mean structure from
  • 00:20:51
    our set of curves
  • 00:20:52
    and as you can see here in these
  • 00:20:54
    projected plots most of these curves are
  • 00:20:56
    far from the linear and uniform
  • 00:20:58
    structures that are recommended by
  • 00:20:59
    design heuristics or design tools like
  • 00:21:01
    color picker for data where we're doing
  • 00:21:03
    just this linear interpolation in a
  • 00:21:05
    perceptual color space
  • 00:21:07
    instead they kind of wiggle all over the
  • 00:21:08
    place in interesting ways that create
  • 00:21:11
    subtle but would turn out to be critical
  • 00:21:13
    affective shifts
  • 00:21:16
    so once we have our characteristic curve
  • 00:21:18
    we can apply that curve to generate a
  • 00:21:20
    ramp by adjusting the curve according to
  • 00:21:22
    the relative lightness distribution
  • 00:21:25
    basically we start with the curve
  • 00:21:26
    positioned according to the mean
  • 00:21:28
    lightness model and we then find a point
  • 00:21:30
    in the curve that is closest to our
  • 00:21:32
    desired color
  • 00:21:33
    once we found this point we shift the
  • 00:21:36
    ramp such that we minimize the lightness
  • 00:21:38
    variation from our original model
  • 00:21:40
    and then anchor the resulting ramp in
  • 00:21:42
    color space to get our new color ramp
  • 00:21:45
    we tested this approach in three ways
  • 00:21:48
    including a direct replication of
  • 00:21:51
    existing designer practices
  • 00:21:53
    and stress testing with poor color
  • 00:21:55
    choices
  • 00:21:56
    and we also conducted a formal study
  • 00:21:57
    with professional designers to evaluate
  • 00:22:00
    the perceptual and aesthetic
  • 00:22:01
    characteristics of our ramp
  • 00:22:03
    so details about the first two
  • 00:22:04
    evaluation methods are in the paper and
  • 00:22:06
    for the sake of time i'm just going to
  • 00:22:08
    touch briefly on our experimental
  • 00:22:10
    evaluation
  • 00:22:12
    so in this evaluation we pseudo randomly
  • 00:22:14
    seeded 25 ramps using models generated
  • 00:22:17
    from the two different clustering
  • 00:22:18
    algorithms and compared these with two
  • 00:22:21
    baseline approaches
  • 00:22:22
    so our generated curves are shown here
  • 00:22:25
    on the right and we compared these with
  • 00:22:27
    first a linear baseline
  • 00:22:29
    where we linearly interpolated between
  • 00:22:32
    two colors at least 40 units of
  • 00:22:34
    lightness apart selected from our
  • 00:22:35
    designer corpus
  • 00:22:37
    and then we used a designer baseline
  • 00:22:39
    where we chose 25 ramps directly from
  • 00:22:41
    our designer corpus at random so this
  • 00:22:43
    baseline gives us a theoretical
  • 00:22:45
    threshold for what a high quality
  • 00:22:47
    coloring coding should look like
  • 00:22:49
    and we use these ramps in a series of
  • 00:22:51
    visualizations where participants
  • 00:22:53
    completed a target identification task
  • 00:22:56
    finding the point in the ramp closest to
  • 00:22:57
    a given value to test the perceptual
  • 00:22:59
    discriminability of each ramp
  • 00:23:01
    we use stimuli with an aesthetic
  • 00:23:03
    question simply asking people to select
  • 00:23:06
    how pleasant they found the colors in
  • 00:23:07
    the viz to be
  • 00:23:09
    so we conducted the study with three
  • 00:23:11
    different visualization types scatter
  • 00:23:13
    plots heat maps and chloroplast maps we
  • 00:23:16
    recruited 35 designers from around the
  • 00:23:18
    world to complete the study and our
  • 00:23:19
    designers had an average of 6.2 years of
  • 00:23:22
    self-reported professional design
  • 00:23:23
    experience
  • 00:23:25
    cutting to the chase here we can
  • 00:23:27
    actually categorize these results
  • 00:23:28
    according to both our perceptual so kind
  • 00:23:31
    of the how good is this ramp for a
  • 00:23:33
    typical visualization task and aesthetic
  • 00:23:36
    measures so in these graphs you'll see
  • 00:23:38
    our approach in green and the baselines
  • 00:23:41
    in blue so for our perceptual measures
  • 00:23:43
    we found that participants were actually
  • 00:23:45
    more accurate with our approach than
  • 00:23:47
    with linear ramps and tended towards
  • 00:23:49
    being more accurate with our approach
  • 00:23:50
    than with designers designer ramps so i
  • 00:23:52
    should note that that was a trending
  • 00:23:54
    towards it was not statistically
  • 00:23:56
    significant
  • 00:23:57
    we found the same pattern with
  • 00:23:58
    aesthetics participants tended to prefer
  • 00:24:01
    our approach to linearly interpolated
  • 00:24:03
    ranch
  • 00:24:04
    and found our models at least as
  • 00:24:07
    aesthetically pleasing as designer ramps
  • 00:24:09
    so the main takeaway here is that for
  • 00:24:11
    these visualizations we find that expert
  • 00:24:14
    participant pool found ramps generated
  • 00:24:16
    using our approach comparable to those
  • 00:24:18
    handcrafted by designers
  • 00:24:21
    well to make these results a little more
  • 00:24:23
    actionable we embedded our models in a
  • 00:24:25
    coloring coding design tool called color
  • 00:24:27
    crafter that essentially provides a
  • 00:24:29
    front end for this algorithm the tool
  • 00:24:31
    allows people to specify target colors
  • 00:24:33
    to generate a set of ramps they can
  • 00:24:34
    manipulate those ramps using a sequence
  • 00:24:36
    of affine transformations including
  • 00:24:38
    translation rotation scaling and
  • 00:24:40
    reflection along all the various axes of
  • 00:24:43
    color space
  • 00:24:44
    the transformation edits allow users to
  • 00:24:46
    quickly customize their encodings
  • 00:24:48
    without sacrificing the quality of the
  • 00:24:51
    resulting rail
  • 00:24:52
    and through this approach even novice
  • 00:24:54
    visualization designers and developers
  • 00:24:56
    can rapidly create custom color ramps
  • 00:24:58
    that embody best practices in
  • 00:24:59
    visualization design and it only really
  • 00:25:02
    requires them to specify one single
  • 00:25:04
    guiding color
  • 00:25:06
    so you can see an example of some of the
  • 00:25:07
    results of this approach here for fun i
  • 00:25:10
    seated a set of ramps using carolina
  • 00:25:12
    blue and you can see the results by
  • 00:25:14
    inputting the target color into the tool
  • 00:25:16
    we're actually able to generate a series
  • 00:25:18
    of color encodings to choose from then
  • 00:25:20
    i'll adhere to our designer properties
  • 00:25:23
    all include our target color but have
  • 00:25:25
    slightly different affects and slightly
  • 00:25:27
    different aesthetic quality
  • 00:25:32
    so
  • 00:25:32
    as promised i wouldn't briefly revisit
  • 00:25:35
    how this work embodies my research
  • 00:25:36
    approach
  • 00:25:37
    we started by characterizing the problem
  • 00:25:39
    of why it's hard to choose good colors
  • 00:25:41
    for this showing that it's a complex
  • 00:25:43
    multi-dimensional space and
  • 00:25:45
    considerations
  • 00:25:46
    from there we develop probabilistic
  • 00:25:48
    models of how people perceive colors in
  • 00:25:50
    viz
  • 00:25:51
    um and we use these models to see an
  • 00:25:53
    algorithm and corresponding tool to
  • 00:25:55
    bootstrap encoding design so the end
  • 00:25:57
    result of this process is fundamental
  • 00:25:58
    knowledge of how visitors work
  • 00:26:00
    models that allow us to make them better
  • 00:26:02
    and tools that make it easier for people
  • 00:26:04
    to create good visualizations
  • 00:26:07
    and as i mentioned i want to toss out a
  • 00:26:09
    little discussion of some of our ongoing
  • 00:26:11
    work in case folks are interested in
  • 00:26:12
    following up about potential
  • 00:26:13
    collaboration and one challenge that's
  • 00:26:16
    come up in discussions with geologists
  • 00:26:18
    and other in the earth sciences is how
  • 00:26:20
    drastically different color maps can
  • 00:26:22
    change the kinds of structures they see
  • 00:26:24
    in their data so we're currently
  • 00:26:26
    exploring how we can interactively infer
  • 00:26:28
    the parameters of ideal encoding choice
  • 00:26:30
    by mining user preferences between color
  • 00:26:32
    map designs
  • 00:26:34
    applied to a scientist's own data using
  • 00:26:36
    bayesian optimization we're also
  • 00:26:38
    exploring alternative methods for using
  • 00:26:40
    statistical models of viz viewed in
  • 00:26:42
    automated settings
  • 00:26:44
    so under um some of the work that's been
  • 00:26:46
    recently funded as part of my career
  • 00:26:47
    we're starting to investigate how we can
  • 00:26:49
    use statistical models of viz
  • 00:26:52
    statistical models of vis-interpretation
  • 00:26:54
    across a range of designs to try to
  • 00:26:56
    automate the process of estimating how
  • 00:26:58
    well a given viz will communicate a
  • 00:27:00
    specific set of patterns and the aim
  • 00:27:02
    here is providing intelligent design
  • 00:27:04
    support tools through rapid situated
  • 00:27:06
    evaluation
  • 00:27:09
    so next i want to pivot away from color
  • 00:27:11
    and touch on a different aspect of my
  • 00:27:13
    research which is how do we leverage
  • 00:27:14
    cognitive processing in visualization
  • 00:27:17
    design so we have a wide range of
  • 00:27:18
    projects in this area ranging from
  • 00:27:20
    looking at behavior and decision making
  • 00:27:22
    to the point i want to talk about today
  • 00:27:23
    which is actually understanding
  • 00:27:25
    cognitive accessibility so data is being
  • 00:27:28
    used to justify a wide range of policies
  • 00:27:30
    and decisions but who are we leaving out
  • 00:27:32
    in our current data communication
  • 00:27:34
    practices
  • 00:27:35
    this project aims to understand barriers
  • 00:27:37
    in existing practice for people with
  • 00:27:39
    intellectual and developmental
  • 00:27:40
    disabilities and to develop principles
  • 00:27:42
    for overcoming these barriers so a lot
  • 00:27:45
    of the work i'm going to talk about is
  • 00:27:46
    still ongoing but i wanted to spend more
  • 00:27:48
    time talking about the work in this
  • 00:27:50
    space that has been published
  • 00:27:53
    so this work is motivated in large part
  • 00:27:56
    by a collaboration with the coleman
  • 00:27:57
    institute for intellectual and
  • 00:27:58
    developmental disabilities they're very
  • 00:28:01
    interested in self-advocacy that is how
  • 00:28:03
    can people with disabilities have more
  • 00:28:05
    agency in the policies that affect them
  • 00:28:08
    policy making is increasing by data
  • 00:28:10
    we've seen that especially in light of
  • 00:28:12
    all of the recent data that's come out
  • 00:28:14
    in the epidemiological space
  • 00:28:16
    but in conversations with psychiatrists
  • 00:28:18
    clinicians and care partners current
  • 00:28:20
    tools for analytics simply are not
  • 00:28:22
    working for people with intellectual
  • 00:28:24
    developmental disability and we wanted
  • 00:28:26
    to understand why and how we could do
  • 00:28:29
    better
  • 00:28:30
    part of understanding why is challenging
  • 00:28:32
    the fundamental assumption and
  • 00:28:33
    visualization that everyone processes
  • 00:28:35
    visual information in the same way
  • 00:28:38
    people with idd have differences that
  • 00:28:40
    emerge in the visual information
  • 00:28:42
    processing components of the human
  • 00:28:43
    cognitive system so we turn to education
  • 00:28:46
    and psychology to try to understand
  • 00:28:48
    where our current assumptions might be
  • 00:28:50
    falling short
  • 00:28:52
    and in ongoing work we're taking more of
  • 00:28:54
    a generative approach
  • 00:28:55
    trying to understand um
  • 00:28:58
    how what kinds of representations are
  • 00:29:00
    intuitive for people with idd using more
  • 00:29:02
    of a participatory design approach since
  • 00:29:04
    this work is ongoing i want to focus on
  • 00:29:06
    the first thread that is what best
  • 00:29:08
    practices are or aren't working for
  • 00:29:11
    people with idd
  • 00:29:13
    and i want to take a step back to
  • 00:29:15
    explain what i mean by idd so ipd or
  • 00:29:18
    intellectual and developmental
  • 00:29:19
    disability is characterized by
  • 00:29:21
    limitations in social conceptual or
  • 00:29:24
    practical skills and it affects over 200
  • 00:29:26
    million people worldwide these
  • 00:29:28
    disabilities often result in differences
  • 00:29:30
    in memory attention visual comprehension
  • 00:29:32
    or math comprehension and as you might
  • 00:29:34
    imagine
  • 00:29:35
    all of these are critical skills for
  • 00:29:36
    people trying to work with data
  • 00:29:38
    visualizations
  • 00:29:40
    in practice these limitations mean that
  • 00:29:42
    the sheer complexity of existing tools
  • 00:29:45
    including complex multi-view designs
  • 00:29:47
    challenges and interaction and building
  • 00:29:49
    insights over time that traditional
  • 00:29:51
    tools like tableau and power bi are
  • 00:29:53
    designed to support
  • 00:29:55
    end up being prohibitively challenging
  • 00:29:57
    in interviews with psychiatrists and
  • 00:29:59
    through our own observations from the
  • 00:30:00
    educational literature we actually
  • 00:30:02
    devised a few specific interventions
  • 00:30:04
    that might challenge existing biz
  • 00:30:06
    guidelines
  • 00:30:07
    but have the potential to enhance
  • 00:30:09
    accessibility
  • 00:30:11
    the first of these is probing
  • 00:30:13
    just what charts will work
  • 00:30:15
    uh so in this we tend to have this
  • 00:30:17
    concept of chart choosers i recommend
  • 00:30:19
    they are recommenders systems that
  • 00:30:21
    choose the single best biz for a given
  • 00:30:22
    task or given type of statistic we want
  • 00:30:24
    to interrogate
  • 00:30:26
    however we know from interviews that pie
  • 00:30:28
    charts for example are totally
  • 00:30:30
    inaccessible and we wanted to explore as
  • 00:30:32
    how well different charts might support
  • 00:30:35
    given tasks and if this differs from our
  • 00:30:38
    conventional practices in our
  • 00:30:39
    conventional chart users and
  • 00:30:41
    visualization
  • 00:30:43
    we also wanted to explore the idea of
  • 00:30:45
    discretization so this suggests that
  • 00:30:48
    continuous representations like lines
  • 00:30:49
    and bars are simple clean and optimal
  • 00:30:52
    our research in mathematical processing
  • 00:30:54
    for children with intellectual
  • 00:30:55
    disabilities runs contrary to this while
  • 00:30:58
    kid's abilities to generalize abstract
  • 00:31:00
    quantities is limited studies show that
  • 00:31:02
    the approximate number system which is
  • 00:31:04
    essentially the ability to roughly
  • 00:31:05
    interpret how many objects are present
  • 00:31:07
    in a scene
  • 00:31:08
    or people with idd may be comparably
  • 00:31:12
    efficient to traditional populations
  • 00:31:14
    meaning that discretization may actually
  • 00:31:16
    be quite beneficial
  • 00:31:19
    and in visualization using semantic
  • 00:31:22
    information like we see here is actually
  • 00:31:24
    kind of a big no-no it adds visual
  • 00:31:26
    complexity with questionable benefit
  • 00:31:29
    however research and education shows
  • 00:31:31
    semantic pictograms actually
  • 00:31:33
    significantly enhance visual visual
  • 00:31:35
    reasoning for children with down
  • 00:31:36
    syndrome and we wanted to explore if
  • 00:31:38
    different methods for visual semantics
  • 00:31:41
    might have a similar effect in data
  • 00:31:44
    representation so to try to probe a lot
  • 00:31:47
    of these questions we designed a mixed
  • 00:31:48
    method study exploring the effects of
  • 00:31:51
    chart type discretization and visual
  • 00:31:53
    semantics with 34 participants with and
  • 00:31:56
    without idd looking at proportion and
  • 00:31:58
    time series data in a series of fiscal
  • 00:32:01
    self-advocacy questions
  • 00:32:03
    we collected data about objective
  • 00:32:04
    performance across different tasks as
  • 00:32:06
    well as subjective data about preference
  • 00:32:08
    and usability i'll leave details about
  • 00:32:10
    the methods the q a or the paper but the
  • 00:32:13
    tldr is that we found traditional biz
  • 00:32:15
    guidelines
  • 00:32:16
    make data inaccessible
  • 00:32:18
    so we were able to generate four key
  • 00:32:20
    design guidelines for enhancing data
  • 00:32:22
    accessibility
  • 00:32:24
    the first of these is resonates pretty
  • 00:32:26
    well with what we had heard from
  • 00:32:28
    practitioners that we just want to avoid
  • 00:32:31
    pie charts people with idd perform
  • 00:32:33
    significantly worse than chance using
  • 00:32:36
    pies whereas performance with stack bar
  • 00:32:38
    charts or tree maps was comparable to
  • 00:32:40
    our non-idd participants
  • 00:32:43
    and while we found mixed effects on the
  • 00:32:45
    use of semantics using familiar
  • 00:32:47
    metaphors sparingly such as just this
  • 00:32:50
    very clean iconography actually greatly
  • 00:32:53
    increased data comprehension we found in
  • 00:32:55
    our subjective feedback that these
  • 00:32:57
    effects were directly related to this
  • 00:32:59
    idea of comprehension people saw the
  • 00:33:01
    semantics the meaning of the data when
  • 00:33:02
    they looked at the charts and it made it
  • 00:33:05
    easier to try to make sense of the
  • 00:33:06
    information they were processing
  • 00:33:10
    people with idd also relied more heavily
  • 00:33:12
    on visual metaphors like bars when
  • 00:33:14
    making staircases or trees tree maps
  • 00:33:17
    from making pieces of paper as part of
  • 00:33:19
    their visual reasoning processes
  • 00:33:23
    we also found that semantics could be
  • 00:33:25
    prohibitive overly complex
  • 00:33:27
    representations performed poorly and
  • 00:33:29
    both objectively and subjectively
  • 00:33:32
    the true takeaways people just felt
  • 00:33:33
    overwhelmed and it took significantly
  • 00:33:35
    more time to try to complete the task
  • 00:33:38
    when representations were more complex
  • 00:33:42
    finally discretization also helped some
  • 00:33:46
    we found that when discrete marks were
  • 00:33:48
    axis aligned like we see here in this
  • 00:33:50
    bar chart performance and preference
  • 00:33:52
    were both high however when they were
  • 00:33:54
    not access aligned as in the pie chart
  • 00:33:56
    here on the right people found
  • 00:33:57
    themselves resorting to counting
  • 00:33:59
    strategies
  • 00:34:00
    leading to longer time on task and lower
  • 00:34:02
    overall preference
  • 00:34:05
    so we're actively working on this
  • 00:34:06
    problem as there's still so much to
  • 00:34:08
    understand and do to make this work
  • 00:34:10
    actionable and one of the projects that
  • 00:34:12
    we're gearing up to run is a
  • 00:34:13
    participatory design workshop that will
  • 00:34:15
    explore how people with idd naturally
  • 00:34:18
    tackle simple data communication
  • 00:34:19
    problems related to everyday life
  • 00:34:22
    we're also working with the coleman
  • 00:34:23
    institute to explore how we might
  • 00:34:24
    automatically adapt visualizations to
  • 00:34:26
    become more accessible and my student
  • 00:34:29
    kikibu has been working with them to
  • 00:34:30
    change their products like their chart
  • 00:34:32
    builder to use more accessible practices
  • 00:34:36
    so for the final thread that i want to
  • 00:34:38
    walk you through today i'm actually
  • 00:34:39
    going to talk a little bit about how we
  • 00:34:41
    can move data off the monitor and into
  • 00:34:43
    the real world using augmented reality
  • 00:34:45
    to allow people to explore data within
  • 00:34:47
    the context that data describes
  • 00:34:49
    leading hopefully it's a more timely and
  • 00:34:51
    actionable analysis
  • 00:34:53
    and the driving problem behind this work
  • 00:34:55
    is data use in field research and
  • 00:34:57
    emergency response so in these scenarios
  • 00:35:00
    data is typically collected in the field
  • 00:35:02
    with scientists and responders carefully
  • 00:35:04
    assessing field sites in small teams
  • 00:35:07
    at the end of the day or even at the end
  • 00:35:08
    of the operation this data is brought
  • 00:35:10
    back to the command center for analysis
  • 00:35:12
    using traditional analytics tools
  • 00:35:14
    so in interviews with a range of experts
  • 00:35:17
    in these fields we found that it was
  • 00:35:18
    almost universal practice
  • 00:35:21
    but that this process introduced some
  • 00:35:22
    problematic divides for example it
  • 00:35:25
    removed analysis from the times and
  • 00:35:27
    places where people could actually do
  • 00:35:29
    something with the data for example one
  • 00:35:31
    scientist described flying a large
  • 00:35:33
    fixed-wing drone in a remote sign in
  • 00:35:34
    greenland
  • 00:35:35
    um running a month-long survey and
  • 00:35:38
    returning home only to realize that a
  • 00:35:40
    good chunk of their data simply didn't
  • 00:35:42
    record so this ends up being a million
  • 00:35:44
    dollar mistake
  • 00:35:46
    we talked to analysts they noticed that
  • 00:35:48
    analyzing data out of context meant that
  • 00:35:51
    people in the field were actually
  • 00:35:52
    operating on stale data and in the
  • 00:35:55
    emergency response scenarios this could
  • 00:35:56
    create potentially life-threatening
  • 00:35:59
    applica situations
  • 00:36:01
    especially when we're talking about
  • 00:36:02
    things like search and rescue or
  • 00:36:03
    wildland fire where field sites might be
  • 00:36:05
    dynamically changing
  • 00:36:08
    we also found that this paradigm meant
  • 00:36:10
    that most analysts didn't have any
  • 00:36:11
    access to the context that the data
  • 00:36:13
    described and if they found errors or
  • 00:36:15
    key information in their data they
  • 00:36:17
    simply weren't a position to act on
  • 00:36:19
    those observations
  • 00:36:20
    because they weren't out in the field
  • 00:36:22
    they were back in the operations center
  • 00:36:24
    so in this line of work
  • 00:36:26
    we're looking to understand how we might
  • 00:36:28
    design immersive analytics tools that
  • 00:36:30
    leverage non-traditional displays to
  • 00:36:31
    embed the analysts within the data to
  • 00:36:34
    overcome these temporal and spatial gaps
  • 00:36:35
    in analysis
  • 00:36:37
    in other words how can we shift from
  • 00:36:38
    seeing data as a thing that lives in a
  • 00:36:40
    spreadsheet
  • 00:36:42
    to one part of a larger often dynamic
  • 00:36:44
    space where data is informed by situated
  • 00:36:46
    context and can immediately inform
  • 00:36:49
    action and decision
  • 00:36:51
    so my work in the space predominantly
  • 00:36:53
    approaches this challenge from two sides
  • 00:36:55
    bottom up through design and perception
  • 00:36:56
    experiments and top down through
  • 00:36:58
    application
  • 00:36:59
    on the design side we're thinking about
  • 00:37:02
    how do we create visualizations
  • 00:37:03
    seamlessly blend the real and virtual
  • 00:37:05
    worlds and we're specifically using an
  • 00:37:07
    empirical approach to understand how the
  • 00:37:09
    ways we visualize data in augmented
  • 00:37:11
    reality
  • 00:37:12
    shift our perceptions of that data both
  • 00:37:14
    in terms of its relation to the real
  • 00:37:16
    world and its ability to effectively
  • 00:37:18
    convey critical statistics
  • 00:37:20
    on the application side we're taking
  • 00:37:22
    more of a problem-driven approach to
  • 00:37:24
    understand how visualizations can
  • 00:37:25
    leverage the capabilities of immersive
  • 00:37:27
    devices for people to use data more
  • 00:37:29
    effectively
  • 00:37:30
    for example in collaboration with nrel
  • 00:37:32
    we explored how experiential data
  • 00:37:34
    storytelling in augmented reality
  • 00:37:36
    could help communicate the use and
  • 00:37:37
    benefits of hydrogen fueling more
  • 00:37:39
    effectively than traditional 2d media
  • 00:37:41
    like a slideshow
  • 00:37:44
    so part of the challenge of designing
  • 00:37:46
    such systems and why we lack guidelines
  • 00:37:48
    for effectively visualizing data in ar
  • 00:37:50
    is that visualization has a long
  • 00:37:52
    love-hate relationship with 3d we know
  • 00:37:55
    that on 2d monitors 3d visualizations
  • 00:37:57
    can cause a lot of problems for example
  • 00:37:59
    3d pie charts like this distort
  • 00:38:02
    perceived percentages it can be really
  • 00:38:04
    tricky to try to resolve the individual
  • 00:38:06
    dots in 3d scatter plots or heights and
  • 00:38:08
    3d bar charts and we even see these
  • 00:38:10
    tools as adding labels to try and
  • 00:38:12
    simplify this task
  • 00:38:14
    well not using 3d is conventional design
  • 00:38:17
    wisdom for traditional bids
  • 00:38:19
    but it's also possible that stereoscopic
  • 00:38:22
    viewing in ar and vr may actually reopen
  • 00:38:25
    this channel for visualization and i see
  • 00:38:28
    we've got a hand up if you want to go
  • 00:38:29
    ahead and jump in
  • 00:38:34
    um
  • 00:38:35
    yeah i wasn't sure when to interrupt or
  • 00:38:37
    not but i guess i figured
  • 00:38:40
    so i see uh some of them uh had a lot of
  • 00:38:43
    different hypothesis tests and kind of
  • 00:38:46
    these very interesting ways of trying to
  • 00:38:48
    think about the the
  • 00:38:50
    yeah the process so it seems almost
  • 00:38:53
    database seems to be like a bundle of
  • 00:38:55
    decisions a lot of the time that seems
  • 00:38:58
    to kind of almost like meld together and
  • 00:39:01
    i was wondering if you could uh
  • 00:39:03
    comment a little bit about how
  • 00:39:05
    do you see of what exactly is being
  • 00:39:07
    falsified when you do some kind of
  • 00:39:09
    hypothesis test uh particularly on the
  • 00:39:12
    last section and and here i think even
  • 00:39:14
    more so so when you're trying to compare
  • 00:39:17
    these these different slides in and how
  • 00:39:19
    you exactly attribute these comparisons
  • 00:39:21
    back to a given hypothesis so for
  • 00:39:24
    example you think you had some donuts
  • 00:39:25
    and then
  • 00:39:26
    then that was the line but then it seems
  • 00:39:28
    like you know oh but this one has an
  • 00:39:29
    axis the other one doesn't have an axis
  • 00:39:31
    and the different ways and like what
  • 00:39:33
    exactly are you yeah thank you
  • 00:39:35
    yeah so i mean i don't like we try to
  • 00:39:37
    sample a long a continuum and right we
  • 00:39:39
    can think of this
  • 00:39:40
    continuum as being from reductionist
  • 00:39:43
    evaluation to holistic evaluation so the
  • 00:39:45
    more holistic side we're trying to get
  • 00:39:47
    to that point of ecological validity
  • 00:39:49
    where the kinds of things we're
  • 00:39:50
    assessing reflect what people are doing
  • 00:39:52
    in practice and the environments they're
  • 00:39:53
    viewing things in in practice so for
  • 00:39:55
    example in the last study with
  • 00:39:57
    intellectual and developmental
  • 00:39:58
    disability we're really thinking about
  • 00:40:00
    this idea of how do people interpret
  • 00:40:03
    this kind of fiscal data in common
  • 00:40:04
    representations so we use statistical
  • 00:40:07
    tasks to collect the objective data but
  • 00:40:09
    we also use subjective measures to have
  • 00:40:11
    people walk us through what are they
  • 00:40:12
    seeing why do they prefer it what makes
  • 00:40:14
    sense what doesn't what's easy what's
  • 00:40:15
    hard um so that's how we're trying to
  • 00:40:17
    get it the more holistic evaluation and
  • 00:40:19
    there are even examples which i don't
  • 00:40:21
    have time to talk about today where the
  • 00:40:23
    way we evaluate something is we build
  • 00:40:25
    something put it out in the wild and see
  • 00:40:27
    what knowledge people get and what
  • 00:40:28
    findings they generate so that's that
  • 00:40:30
    kind of far side of holistic evaluation
  • 00:40:32
    there's also the very reductionist side
  • 00:40:34
    which is where we want to try to
  • 00:40:35
    understand specific aspects of this to
  • 00:40:38
    try to make very low level design
  • 00:40:40
    guidance and so a lot of what i'm going
  • 00:40:42
    to get to here in a minute with this
  • 00:40:44
    study is focusing on that for example we
  • 00:40:46
    want to understand what are the
  • 00:40:48
    affordances of immersive analytics what
  • 00:40:50
    happens when i take data off the screen
  • 00:40:52
    and put it in a headset put it in ar put
  • 00:40:54
    it in vr and how my designs need to
  • 00:40:57
    change and a lot of these kinds of
  • 00:40:58
    studies often occur very early in the
  • 00:41:00
    design process because they're the kinds
  • 00:41:03
    of things that just are providing our
  • 00:41:05
    fundamental mappings and are ensuring
  • 00:41:08
    that the way data is represented
  • 00:41:10
    actually reflects the differences that
  • 00:41:11
    truly exist in our data so we're looking
  • 00:41:14
    at in that case measuring very
  • 00:41:15
    controlled differences in our
  • 00:41:17
    representation and trying to model how
  • 00:41:19
    the small variations in those
  • 00:41:20
    differences
  • 00:41:22
    translate to differences in
  • 00:41:23
    interpretation so i think to do
  • 00:41:26
    effective visualization to generate
  • 00:41:28
    effective guidelines we really need
  • 00:41:29
    samples all along that continuum from
  • 00:41:31
    the low level concrete quantitative
  • 00:41:33
    decisions the high level more subjective
  • 00:41:36
    more qualitative notions of how do these
  • 00:41:38
    low-level pieces all come together and
  • 00:41:41
    interact in interesting ways to try to
  • 00:41:43
    help us understand what makes that
  • 00:41:44
    visualization tick
  • 00:41:51
    cool awesome and that's actually a great
  • 00:41:53
    segue into
  • 00:41:55
    this next experiment unless you have a
  • 00:41:56
    quick follow on there
  • 00:42:03
    yeah very
  • 00:42:05
    oh i think you might be muted
  • 00:42:09
    so should you very quickly so my
  • 00:42:10
    understanding then is you would merge
  • 00:42:13
    both data sets at some point and
  • 00:42:17
    try to think about a kind of a quality
  • 00:42:20
    based approach as well as
  • 00:42:22
    concrete guidance on those different
  • 00:42:25
    qualities in some sense so i change
  • 00:42:26
    colors from red to blue
  • 00:42:28
    and i people said that red the change
  • 00:42:32
    from red to blue is not very good yeah
  • 00:42:34
    and the basic idea is that by having all
  • 00:42:36
    this data we're actually empowering
  • 00:42:37
    designers to do that merging and to make
  • 00:42:40
    the trade-offs in their specific
  • 00:42:42
    scenarios and in the specific context of
  • 00:42:43
    working in um i'm not convinced that we
  • 00:42:46
    can do that at this point automatically
  • 00:42:48
    i don't think we can totally replace the
  • 00:42:49
    human intuition and design but that's a
  • 00:42:51
    whole philosophical argument for later
  • 00:42:55
    cool
  • 00:42:56
    cool so the the a great point following
  • 00:42:59
    up on this though is that
  • 00:43:02
    not using 3d is conventional design
  • 00:43:04
    wisdom for traditional viz and some of
  • 00:43:07
    the leading figures in the field of
  • 00:43:08
    immersive analytics tend to think that
  • 00:43:12
    we generally want to follow the same
  • 00:43:14
    kinds of rules that we see in
  • 00:43:15
    traditional viz and in ar and br that
  • 00:43:17
    these kinds of 3d stereoscopic immersive
  • 00:43:19
    displays
  • 00:43:20
    may not actually fundamentally change
  • 00:43:23
    for example the fact that 3d isn't
  • 00:43:24
    always very good
  • 00:43:26
    um so some of these guidelines that you
  • 00:43:28
    see here may feel a little bit
  • 00:43:29
    counter-intuitive given the importance
  • 00:43:31
    of immersive devices such as the idea
  • 00:43:33
    that 3d perspective doesn't help or that
  • 00:43:35
    3d nav is hard to do and where
  • 00:43:37
    when we do it every day in the real
  • 00:43:38
    world and we wanted to test how these
  • 00:43:41
    ideas hold up in practice again getting
  • 00:43:43
    that idea of the kind of low-level
  • 00:43:45
    design specification
  • 00:43:46
    so we investigated what happened so we
  • 00:43:48
    asked people to use the same
  • 00:43:50
    visualization on a desktop versus in vr
  • 00:43:54
    versus an ar
  • 00:43:56
    and so in the study people use 2d and 3d
  • 00:43:58
    visualizations encoding data using a
  • 00:44:00
    range of visual channels such as color
  • 00:44:02
    and size to estimate a series of
  • 00:44:04
    different statistics and we compared how
  • 00:44:06
    quickly and accurately people completed
  • 00:44:08
    these tasks as well as how much they
  • 00:44:10
    interacted with the data
  • 00:44:11
    and while i can chat about specific
  • 00:44:13
    results online the takeaway here is that
  • 00:44:15
    the same visualizations perform
  • 00:44:17
    differently on the desktop ar and vr
  • 00:44:19
    conditions in other words this design is
  • 00:44:21
    not one-size-fits-all displays but
  • 00:44:24
    rather we need different guidance
  • 00:44:25
    depending on the kinds of displays we're
  • 00:44:27
    using
  • 00:44:28
    so to highlight a few of our findings on
  • 00:44:30
    a 2d monitor it's really hard to resolve
  • 00:44:32
    whether a 3d scatter point or bar is
  • 00:44:34
    either smaller or further away from a
  • 00:44:36
    larger point however showing the same
  • 00:44:39
    scatter plot in ar or vr allowed people
  • 00:44:41
    to more readily disentangle size and
  • 00:44:44
    depth potentially reopening this
  • 00:44:46
    dimension
  • 00:44:47
    we saw an opposing effect for color
  • 00:44:49
    people struggle to interpret color-coded
  • 00:44:51
    data in ar and we can see an example of
  • 00:44:53
    this here putting the points over darker
  • 00:44:55
    objects makes points appear lighter and
  • 00:44:57
    vice versa so we kind of get this muting
  • 00:44:59
    effect of our color choices this is
  • 00:45:01
    related to a phenomenon known as
  • 00:45:02
    simultaneous contrast
  • 00:45:04
    and while ar degraded color
  • 00:45:06
    interpretation we actually found that it
  • 00:45:08
    increased engagement people walked
  • 00:45:10
    around and interacted with data
  • 00:45:11
    significantly more in ar than either the
  • 00:45:13
    desktop or vr and subjectively noted
  • 00:45:16
    that they were actually more comfortable
  • 00:45:17
    interacting with data in ar
  • 00:45:19
    still a ton more new impact here but
  • 00:45:21
    this offers interesting questions for
  • 00:45:23
    interaction design and immersive
  • 00:45:24
    analytics
  • 00:45:26
    and the larger takeaway from the study
  • 00:45:27
    is that we need new guidelines and
  • 00:45:29
    practices for designing effective data
  • 00:45:30
    visualizations using emerging
  • 00:45:32
    technologies
  • 00:45:34
    we explored some of these possible
  • 00:45:35
    designs in fieldvue which is a prototype
  • 00:45:37
    toolkit for field-based analytics so
  • 00:45:39
    building on a series of interviews with
  • 00:45:40
    scientists and emergency responders we
  • 00:45:42
    developed a multi-device application to
  • 00:45:44
    support field data collection analysis
  • 00:45:46
    practices
  • 00:45:47
    so our solution couples a mobile
  • 00:45:49
    application for data collection overview
  • 00:45:51
    analysis with detailed contextualized
  • 00:45:53
    visualizations in ar and the workflow
  • 00:45:55
    for this tool starts with our data
  • 00:45:57
    collection app where field analysts can
  • 00:45:59
    input a variety of information it's
  • 00:46:00
    automatically geotagged the information
  • 00:46:02
    is cached and when connected synced with
  • 00:46:04
    cloud data store or local network where
  • 00:46:07
    members of a field team can access a
  • 00:46:08
    common data store
  • 00:46:10
    and analysts then pull down this data
  • 00:46:12
    and view summary overview visualizations
  • 00:46:14
    on mobile devices or project geotagged
  • 00:46:16
    data into the real world to explore this
  • 00:46:18
    data in a situated context
  • 00:46:21
    so we released fieldvue as an open
  • 00:46:22
    source toolkit including a prototype
  • 00:46:24
    mobile overview and immersive detail
  • 00:46:26
    visualizations for three common
  • 00:46:28
    scenarios noted in our discussions with
  • 00:46:30
    analysts coordinating data collection
  • 00:46:32
    activities over a local team monitoring
  • 00:46:34
    data quality during field collection and
  • 00:46:36
    accessing data collected by autonomous
  • 00:46:38
    sensors such as overhead drones
  • 00:46:41
    to give you just a very quick overview
  • 00:46:43
    of how these visualizations work here
  • 00:46:45
    you see a scatter plot or a stratified
  • 00:46:48
    grid on a field site a team member can
  • 00:46:50
    identify regions of uncollected data
  • 00:46:52
    which are shown here in these big blue
  • 00:46:53
    squares and these areas are replaced by
  • 00:46:57
    situated color-coded scatter plot values
  • 00:46:59
    when data is provided analysts can then
  • 00:47:02
    go ahead and move into the missing areas
  • 00:47:04
    to collect new data and you'll see the
  • 00:47:06
    overlay disappears when the analysis is
  • 00:47:08
    within the missing region to avoid
  • 00:47:09
    including field of view
  • 00:47:12
    so this next example shows how we might
  • 00:47:14
    use the fieldview interface to connect
  • 00:47:16
    with aerial imagery that's collected by
  • 00:47:18
    a drone the scatter plot points indicate
  • 00:47:19
    available imagery with colors
  • 00:47:21
    corresponding to temperature measures
  • 00:47:23
    collected by drone sensor we can click
  • 00:47:25
    on the points to show the overhead data
  • 00:47:26
    with the images remaining just outside
  • 00:47:28
    the central field of view to preserve
  • 00:47:30
    the analyst's abilities to maintain
  • 00:47:31
    awareness of the environments
  • 00:47:33
    the points in the periphery indicate
  • 00:47:35
    data outside the user's primary view
  • 00:47:37
    with their size and position cueing to
  • 00:47:39
    distance and orientation relative to the
  • 00:47:41
    user so the basic idea here is that
  • 00:47:43
    we're trying to use this overhead
  • 00:47:44
    imagery to provide immediate context for
  • 00:47:46
    current operations and to monitor
  • 00:47:48
    autonomous data collection operations in
  • 00:47:50
    real time while still maintaining
  • 00:47:52
    overall awareness and the responders
  • 00:47:55
    ability to operate within their local
  • 00:47:56
    environment
  • 00:47:58
    everything here is still quite
  • 00:47:59
    preliminary but artwork in the space
  • 00:48:01
    eliminated number of interesting
  • 00:48:03
    questions at the intersection of
  • 00:48:04
    robotics and data analytics for
  • 00:48:05
    emergency response operations that
  • 00:48:07
    myself and my collaborators are
  • 00:48:09
    currently exploring
  • 00:48:10
    i'm also in the preliminary phase of
  • 00:48:12
    exploring how we can design
  • 00:48:13
    visualizations that intelligently adapt
  • 00:48:15
    to user surroundings to maximize
  • 00:48:17
    information gain while minimizing
  • 00:48:18
    interference with the environment
  • 00:48:20
    for example we're looking at how we can
  • 00:48:22
    use information about lighting and
  • 00:48:23
    seeing composition to make better color
  • 00:48:25
    choices or how we might adapt the
  • 00:48:27
    complexity of a viz based on movement in
  • 00:48:29
    the local environments so part of
  • 00:48:31
    addressing these questions is also
  • 00:48:32
    understanding how the context provided
  • 00:48:34
    by situated this can actually a data
  • 00:48:37
    comprehension and decision making and in
  • 00:48:39
    turn how designs might best take
  • 00:48:41
    advantage of such games
  • 00:48:43
    so in the interest of time i'm just
  • 00:48:44
    going to sum up really quickly here
  • 00:48:46
    we've looked at three threads of
  • 00:48:47
    research to demonstrate the processes i
  • 00:48:49
    used to create effective visualization
  • 00:48:50
    systems for data exploration
  • 00:48:53
    starting with this idea of how do we use
  • 00:48:54
    color more effectively and moving into
  • 00:48:56
    how this might mediate human machine
  • 00:48:58
    teaming with large data sets in a mixed
  • 00:49:01
    initiative anal or sorry in uh
  • 00:49:04
    in uh cognitive accessibility and
  • 00:49:06
    finally exploring how ar can reduce
  • 00:49:09
    potential spatial and temporal gaps
  • 00:49:11
    analysis for applications like
  • 00:49:13
    environmental science and emergency
  • 00:49:14
    response and this represents a lot of
  • 00:49:17
    the work that we've done to date we have
  • 00:49:19
    lots of other fun work that's going on
  • 00:49:21
    between looking at using um hardware and
  • 00:49:24
    wearables to promote data science
  • 00:49:26
    literacy looking at collaborative human
  • 00:49:28
    machine teaming looking at the interface
  • 00:49:30
    of robotics and fizz and also looking at
  • 00:49:33
    how we might use making and papercraft
  • 00:49:35
    to build
  • 00:49:36
    informal activities for early childhood
  • 00:49:38
    data literacy
  • 00:49:40
    if folks are interested in connecting
  • 00:49:41
    about any of these projects please do
  • 00:49:43
    reach out for the sake of time i think
  • 00:49:46
    i'm just going to cut here to the end
  • 00:49:48
    and thank my collaborators
  • 00:49:51
    thank you all for having me today thank
  • 00:49:53
    you for you know listening and also take
  • 00:49:56
    any questions that you all might have
  • 00:49:58
    i'm looking forward to the discussion
  • 00:50:02
    hi
  • 00:50:03
    thanks so much for for a great talk
  • 00:50:05
    um i had a follow-up question regarding
  • 00:50:08
    kind of ar and color so you talked about
  • 00:50:10
    your work uh studying color and
  • 00:50:14
    um the importance of selecting selecting
  • 00:50:17
    that color and also your work in and
  • 00:50:19
    understanding
  • 00:50:21
    the challenges that ar poses in
  • 00:50:23
    selecting color given the background and
  • 00:50:26
    you briefly mentioned kind of looking at
  • 00:50:27
    lighting and seeing composition to to
  • 00:50:30
    select colors appropriately and i was
  • 00:50:33
    wondering if you can talk a little bit
  • 00:50:34
    more about
  • 00:50:36
    what makes color important in ar and
  • 00:50:39
    is there
  • 00:50:40
    reasons for maybe exploring other visual
  • 00:50:43
    attributes like motion or things that
  • 00:50:46
    may uh kind of not have the same issues
  • 00:50:49
    in ar uh and if not what uh
  • 00:50:53
    how can we kind of look at um
  • 00:50:56
    uh using perception
  • 00:50:58
    to kind of correct some of those issues
  • 00:51:00
    with color and different backgrounds
  • 00:51:03
    yeah series of great questions there's a
  • 00:51:04
    ton to impact there um i would say you
  • 00:51:07
    know one of the things i didn't have
  • 00:51:08
    time to talk about in that study is we
  • 00:51:09
    actually generated a ranking of what
  • 00:51:12
    visualization techniques or what visual
  • 00:51:14
    channels work best for which modalities
  • 00:51:17
    and this is where you know color was not
  • 00:51:20
    as good in ar as other modalities so if
  • 00:51:22
    we think about trying to privilege other
  • 00:51:24
    like position or size that might be more
  • 00:51:27
    precise
  • 00:51:28
    this understanding where color sits with
  • 00:51:30
    respect to all this allows us to
  • 00:51:32
    understand when do we actually need to
  • 00:51:33
    take on that challenge of working with
  • 00:51:34
    color
  • 00:51:36
    how we do it is going to be a little
  • 00:51:37
    more complicated color tends to be
  • 00:51:39
    really really good for
  • 00:51:40
    high level kinds of statistics when i
  • 00:51:42
    need to do things like estimating
  • 00:51:44
    averages or also for drawing attention
  • 00:51:47
    very very quickly right if i see a red
  • 00:51:49
    point in the field of blue points i'm
  • 00:51:50
    going to look right at it and so i think
  • 00:51:53
    those are the situations where we need
  • 00:51:55
    to think about
  • 00:51:56
    are there ways of making color better or
  • 00:51:59
    in ar do there happen to be better ways
  • 00:52:02
    of achieving the same kind of tasks and
  • 00:52:04
    i think there's a lot of open-ended
  • 00:52:05
    questions that
  • 00:52:07
    i'm looking forward to seeing where
  • 00:52:09
    things go because i think there's lots
  • 00:52:11
    and lots of cool stuff that could come
  • 00:52:12
    out of that
  • 00:52:15
    thank you so much great talk
  • 00:52:21
    i have a question thanks so much for
  • 00:52:23
    this really exciting talk and i think
  • 00:52:25
    kind of following up on the question um
  • 00:52:27
    you know you talked a little bit about
  • 00:52:29
    uh in in terms of ar vr
  • 00:52:32
    and then kind of a desktop side of
  • 00:52:34
    things it seems like there's a lot to
  • 00:52:36
    unpack there in terms of both like
  • 00:52:38
    different techniques for interactive
  • 00:52:40
    perception and the different ways in
  • 00:52:42
    which a person can interact with it but
  • 00:52:44
    then also on the low level perceptual
  • 00:52:47
    side and you know there it seems like a
  • 00:52:49
    big difference is in terms of these
  • 00:52:51
    depth cues
  • 00:52:52
    and you know there are lots of different
  • 00:52:55
    you know cues that we use for depth and
  • 00:52:57
    i'm wondering you know how much of this
  • 00:52:59
    is connected to the specific display
  • 00:53:02
    technology that we have versus inherent
  • 00:53:04
    in you know ar vr etc
  • 00:53:08
    yeah i mean it's a great question in
  • 00:53:10
    that study we were trying to minimize
  • 00:53:11
    the variation so we actually were using
  • 00:53:14
    either pass through with the um htc vive
  • 00:53:17
    and the zen mini or we were using just
  • 00:53:21
    the htc
  • 00:53:22
    vive so
  • 00:53:23
    we were trying to really minimize the
  • 00:53:25
    amount of device variation but that's
  • 00:53:27
    really going to be a thing especially
  • 00:53:28
    when you start to think about things
  • 00:53:29
    like interaction plus perception plus
  • 00:53:31
    field of view if i'm only seeing a small
  • 00:53:34
    portion of the display and i'm losing
  • 00:53:35
    all the information that's coming in the
  • 00:53:37
    periphery how is that changing my
  • 00:53:38
    decision making how is it changing how i
  • 00:53:40
    choose to move and interact in the space
  • 00:53:42
    so i think there are a lot of parameters
  • 00:53:44
    of the display technologies and display
  • 00:53:47
    hardware and the ways that changes how
  • 00:53:49
    we might intuitively interact with the
  • 00:53:51
    the representations that are still to be
  • 00:53:54
    explored and are really critical when we
  • 00:53:56
    think about how do we put these these
  • 00:53:58
    ideas into actionable contexts and
  • 00:54:00
    there's even the broader point that we
  • 00:54:03
    ran into that i
  • 00:54:05
    glossed over because it's a really hard
  • 00:54:07
    hardware challenge which is what happens
  • 00:54:09
    when we take these technologies outside
  • 00:54:12
    you'll notice all of our videos we're on
  • 00:54:14
    very cloudy days
  • 00:54:15
    it's because on bright and sunny days
  • 00:54:17
    you can't see anything in the headsets
  • 00:54:20
    so this is another challenge that
  • 00:54:22
    pertains to displays and perception and
  • 00:54:24
    all of the rich environmental contexts
  • 00:54:26
    that come into play
  • 00:54:28
    once you actually start to put these
  • 00:54:30
    things out into practice so i think that
  • 00:54:32
    this is a long way of saying we still
  • 00:54:33
    need a lot more work because all of
  • 00:54:35
    these different factors are definitely
  • 00:54:36
    come together going to come together and
  • 00:54:38
    influence efficacy and influence design
  • 00:54:51
    yeah
  • 00:54:52
    can i ask a question go for it yep sorry
  • 00:54:55
    yeah yeah uh yeah thank you for your
  • 00:54:57
    talk um so you mentioned uh the study
  • 00:55:00
    like comparing the arvr and the
  • 00:55:03
    2d like 2d analytics but do you have any
  • 00:55:07
    suggestions on like when to do the like
  • 00:55:10
    immersive and want to do 2d because some
  • 00:55:13
    people might argue like what is
  • 00:55:14
    necessary to do ar vr especially given
  • 00:55:17
    like 2d might be more accessible like
  • 00:55:19
    the smartphones laptops are more
  • 00:55:22
    like more often used than the ar vr has
  • 00:55:24
    said
  • 00:55:25
    so um it's interesting you bring this up
  • 00:55:27
    so we discussed this there is a paper
  • 00:55:29
    that came out at kai last year in grand
  • 00:55:31
    challenges for immersive analytics where
  • 00:55:32
    this was a huge point for us is figuring
  • 00:55:34
    out
  • 00:55:35
    what situations does it make sense to
  • 00:55:37
    use immersive spaces versus when does it
  • 00:55:39
    make sense to use traditional you know
  • 00:55:41
    desktop-based analytics
  • 00:55:43
    and my argument would be that there are
  • 00:55:46
    two situations that really come to mind
  • 00:55:49
    um this is just just my own opinion from
  • 00:55:51
    my own experiences this is not something
  • 00:55:52
    that's yet backed by data but there are
  • 00:55:54
    two situations in particular where this
  • 00:55:56
    kind of immersive space can make a lot
  • 00:55:58
    of sense
  • 00:55:59
    um one of these is when you're situating
  • 00:56:02
    data when having the context of the
  • 00:56:04
    physical environment is actually going
  • 00:56:06
    to
  • 00:56:06
    affect the kinds of decisions that you
  • 00:56:09
    need to make so if i am looking at a
  • 00:56:12
    burning structure and then i see my
  • 00:56:14
    scanner plot about the current heat map
  • 00:56:15
    distributor or heat temperature
  • 00:56:17
    distributions in the local region
  • 00:56:19
    that's going to give me a whole lot more
  • 00:56:21
    information it's going to be a whole lot
  • 00:56:22
    more actionable
  • 00:56:23
    um the same it comes into play when
  • 00:56:26
    you're starting to think about these
  • 00:56:28
    kinds of um
  • 00:56:30
    basically putting
  • 00:56:32
    information in the space where we need
  • 00:56:34
    to be able to act right i can
  • 00:56:36
    bring around my phone with me but
  • 00:56:38
    anytime i'm doing analytics on a phone
  • 00:56:40
    right this is why we don't do texting
  • 00:56:41
    and driving you have divided attention i
  • 00:56:43
    can pay full attention to my phone i can
  • 00:56:45
    pay full attention to what i'm doing and
  • 00:56:47
    i'm moving my attention back and forth
  • 00:56:48
    between devices so i would say the
  • 00:56:50
    second scenario is in those cases where
  • 00:56:52
    divided attention is less than ideal um
  • 00:56:56
    so those are more cases where having
  • 00:56:58
    that situated nature might be really
  • 00:56:59
    helpful and a lot of this might come
  • 00:57:01
    down to things like ambient situational
  • 00:57:03
    awareness that's promoted by
  • 00:57:05
    always having that display available to
  • 00:57:07
    you
  • 00:57:08
    but maybe you're not always fully
  • 00:57:10
    attending to the display and if you need
  • 00:57:12
    full attention you can bring that data
  • 00:57:13
    back out into the environment so those
  • 00:57:15
    would be the two cases that i would
  • 00:57:17
    posit that this kind of situatedness is
  • 00:57:20
    actually beneficial beyond what we have
  • 00:57:22
    on a 2d display there are interesting
  • 00:57:24
    questions about interactive affordances
  • 00:57:26
    about the role of immersion about how
  • 00:57:28
    immersion and you know physical
  • 00:57:30
    reconstruction in space might actually
  • 00:57:32
    influence spatial comprehension but i
  • 00:57:34
    don't think we have enough in the
  • 00:57:36
    immersive analytics space
  • 00:57:38
    yet to say whether those benefits
  • 00:57:40
    outweigh the trade-offs of moving into
  • 00:57:42
    this new modality
  • 00:57:46
    yeah thank you
  • 00:57:49
    that's a good question this is all
  • 00:57:50
    getting into stuff that i think music
  • 00:57:52
    fields still need to figure out
  • 00:57:57
    this is great all right well if there
  • 00:57:59
    aren't any other questions then let's
  • 00:58:01
    thank danielle for that really exciting
  • 00:58:03
    talk and yeah thanks so much
  • 00:58:11
    you
标签
  • Data Visualization
  • Exploratory Analysis
  • Cognitive Science
  • Augmented Reality
  • Accessibility
  • Color Theory
  • Machine Learning
  • Data Literacy
  • Survey Analysis
  • Field Research