But now things are completely different. When I started developing the new course, I suddenly thought, “Wait a minute … I can ask Claude to do this for me.” And the more I tried, the more excited I became with the results. Let me describe why this is so powerful with a couple of examples.

Example 1: Reproducing and extending an existing chart

I wanted to reproduce the latest example of a truncated y-axis. But I wanted to show how it looks without truncation, and I also wanted to match the style I used for the other charts. Below, you can see the original chart and my reproduction/extension.

Example 2: Developing a chart showing a specific pattern

There’s a section of the course where I explore the idea of “visual patterns” and how some patterns jump out while others require deliberate inspection. After several attempts, I came up with this map that helps me show how patterns like big bubbles readily stand out, but others, such as blue bubbles surrounded by red bubbles or the other way around, require careful inspection.

Example 3: Creating different charts of the same data

At some point I focused on how different charts can encode exactly the same data but produce completely different effects. The one I created shows how perception changes when visualizing time series with a bar chart vs. a line chart.

Tips

Here are a few tips I have gathered while developing the course.

1. Create a project (or use Claude Code). Don’t rely on one single chat. Create a project. This way, you can use multiple agents in parallel (very useful!!) and progressively provide more context as the project develops. Note: I used the regular Claude web interface, but I could have done the same thing with Claude Code.

2. Provide context. I gave Claude as much context as I could by uploading files such as the draft of the slides I had done (I normally develop draft slides without images and put placeholders where images should go) and any other document that described the project. As the project unfolded, I kept adding files and notes to the context to give Claude more specific instructions and information to consider (many of these I would certainly reuse as instructions in future projects).

3. Create one separate chat/session for each chart or concept. Sifting through a long chat session is very tedious. If you cram everything in one place, you’ll find it hard to stay organized. But if you start a new chat for every chart or concept, you’ll be able to stay much more organized. Here, it is important to think about when in the future you might need to access these examples again; maybe for a new project or an update of your teaching material. Also, LLMs are known to degrade in performance the longer a chat goes on. So it’s good practice to start new chats for separate problems.

4. Use synthetic data. When you ask Claude to produce a chat that depicts a given concept, it often tries to find a dataset that fits your idea and produces a chat with the data it finds (e.g., by retrieving a dataset from Our World in Data or the World Bank). Sometimes this is desirable, and sometimes you already have the data you want. But I've found that, in most cases, I want something specific that no real dataset can provide because I want to illustrate an effect, and that effect will be depicted much more clearly if I allow and instruct Claude to create a “fake” dataset for that purpose. The beauty of these tools is that they can manage synthetic data creation for a given purpose, and I found the flexibility synthetic data affords priceless.

5. Reproduce existing examples (with proper attribution). Sometimes you have examples that you could use from previous courses, projects, or other people you know, or images you find through a web search. In those cases, what do you do? Do you use them directly or ask Claude to reproduce them? In my case, I decided to just cut and paste one or more images from the examples and ask Claude to reproduce them. There are two main reasons why I do it that way. First, I like the idea of charts that look stylistically similar. Second, I can edit the examples by adding or removing details as needed. Editing is crucial because sometimes you find an example that captures the logic you want to convey, but you are not entirely happy with a specific aspect of the example you found. In any case, when you use other people’s work, it is essential to add a note with proper attribution. In my case, I write something like, “Reproduced from … using Claude AI.” And I add a link to the original source.

6. Add more context as you go. Sometimes you find that, to express a given concept, you can make Claude more powerful by providing additional context on the fly. For example, in a section of my course, I developed charts that depict the problem with double-axis charts. In those cases, I fed Claude a few web pages from data visualization authors who address the problem in ways that I like, so that I was sure it had the full context and the right direction. Also, as I refine my slides, sometimes I copy and paste them (as an image) to show Claude the text that goes in the slides I am building and, sometimes, the slides right before and right after. It takes a few seconds, and it adds useful context.

7. Refine and be ready to discard. In most cases, Claude doesn't produce exactly what you want on the first try. This is normal and expected. Only after seeing a chart do you remember instructions you should have included in the first place. Or, more simply, Claude did not understand important aspects of your description. The most basic solution is to add more detail and instructions so Claude can produce what you want. But in some cases, it’s hard to describe the problem verbally when it pertains to a specific graphical aspect. For this reason, I often copy and paste the image produced by Claude and paste it back to Claude to tell it, “Hey, you see? This is the problem here!’ And most of the time, it works quite well. There is a third situation. Sometimes Claude just can’t produce what you have in mind. In those cases, I find that starting from scratch and using a different strategy is better than trying to “force” Claude to obtain what you want within the existing thread. The problem is that the context Claude has in the current chat, or sometimes even in previous conversations, is the source of the problem, so you need to start with a blank slate.

8. Stylistic consistency. If you don’t ask directly, Claude may use slightly different styles for different charts. In particular, it may use a given color palette in one chart and a different one in another. If you care about maintaining a consistent style, you have to ask Claude to use the same color palette or other stylistic elements across charts in the project. I am a big fan of ColorBrewer, so I often ask Claude directly to use CB palettes with my charts.

9. Annotations don’t work (yet). In some cases, Claude tries to add annotations to charts the same way a visualization designer would do to emphasize an element or explain a particular trend. Unfortunately, annotations still don't work well. Whenever I tried, Claude produced unusable images. I can’t tell whether Claude could do it better with more instructions. Honestly, annotations weren't super important in this project for me, so I didn't test it further. In any case, given the progress I have seen in the last year or two, I am pretty sure this will be solved soon.

10. Run several sessions in parallel. Claude takes time to produce a response to your prompt when the result is a chart. Unless you want to be waiting all the time when you issue a new instruction, you can work on two or three examples at a time. Just be sure you don’t go overboard, because keeping track of too many things and constantly switching context is very tiring for our minds. I find that working on two examples in parallel is the right balance for me. I am also learning to just use the pauses more productively, maybe by standing up and walking a bit, stretching my limbs, or making coffee or tea.

11. Give meaningful/memorable titles to the chats. This complements the idea of creating one separate chat for each chart. Think about when you’ll need to return to this project in the future. Naming your chart properly is crucial here! Unfortunately, I realized this too late, and I will be much more diligent in the future. Claude, like any other AI tool, will create a specific title based on the content. But a better approach for a project is to create a naming convention and use it consistently.

12. Keep a link and use good titles. At development time, it’s easy to fall into a frantic production of charts and chat. One thing I've noticed with these AI tools is that it’s easy for our minds to branch out, create a million different threads, and get caught up in the frenzy. But deliberate use is much better. In particular, keep in mind that you may need to return to these sessions later and find what you are looking for. For example, if you realize there is a mistake in one of the images produced. So, a good practice is to keep track of which chat you used to develop a specific chart in your slides. My preferred choice is to add a link in the notes, but any other tracking mechanism works.

I am looking forward to developing more courses in the near future. Right now, I'm developing a new course I will teach this fall on Statistics for Designers, and I'm excited to see how AI tools can help me create better images for my students. Other than creating images for slides, these tools can help develop incredibly powerful explainers for tricky concepts. I have been developing quite a few for visualization and statistics, and I'll definitely share some here as I develop more.

In the meantime, I hope you’ll test some of these ideas for yourself. If you have tried any of this before and have additional tips, please let me know!

One final note. I became more proficient with Claude Code only after developing this course. If I were to restart from scratch today, I would most probably use Claude Code rather than the standard chat interface. Code makes it much easier to create multiple agents working on parts of the problem and to track the images generated during the process. For example, I could have told Code to organize the produced images in a folder with specific names and structure to keep things organized. In any case, if you are more comfortable with the standard chat interface, feel free to use it. Just remember to create a project, add useful context, and develop a separate chat for each chart or concept.

This mindset, however, overlooks one important fact: that for any given data combination, there is a staggering number of possible solutions, and some of these solutions can be equally valid depending on the specific context and goal.

There is more. In data visualization theory, designers are instructed to prioritize representations that “encode” information with the highest precision possible. For example, a well-established fact is that encoding quantity with bar length (as in bar charts) leads readers to extract values more precisely than if the same value is represented by the area of a circle or the color of a symbol (as in choropleth maps).

What these models overlook, however, is that different representations of the same data can lead people to focus on different aspects and thus extract different information. So the problem is not how precisely one can extract information, but what information is extracted in the first place when a person is shown a given representation.

A 1997 study by Zacks and Tversky provides empirical evidence for this idea. When people were shown a bar chart or a line chart depicting the same values, they tended to interpret the data differently. With bar charts, they tend to interpret them as a comparison of values, whereas with line charts, as a trend. Interestingly, people sometimes interpreted trends in line charts even when the underlying data were discrete.

My student, Racquel Fygenson, and I ran a related experiment a few years ago. We showed participants different representations of the same data and asked which representation best matched a given statement. What we found is that with simple variations, it’s quite easy to make one statement more strongly activated than another.

For example, in the experiment depicted above, we compared this set of stacked and grouped bar charts and then asked participants to select which chart makes these statements more obvious:

1. “East companies’ sales are 70% clips.”

2. “In the West, companies sold more clips than staples.”

And what we found is that readers consistently select different charts, meaning that different charts are more apt for different messages, despite representing exactly the same data (with exactly the same visual primitives in this case).

The reality of data representation is that when we decide how to represent a given piece of information, we are implicitly telling our reader to look at a chart in a particular way, from a particular angle, and to extract a particular set of facts. The bad news is that we don’t have a practical full theory or model we can use to think about these effects systematically yet (my student Racquel, however, has made quite some strides in this direction with her work on visualization affordance - you should check her work!).

In the absence of a complete model here, I’ll offer three main classes of graphical variations of the same data that often affect interpretation: spatial arrangement, scaling, and data-transforming reconfigurations.

Spatial arrangement

By far the most impactful element in visualization is the use of space. When visualizations differ in terms of spatial configuration, you can rest assured that our eyes will parse the information differently. But how can visualizations arrange the same information in different ways? One way is by using different axis configurations. Most visualizations are based on a set of axes, and axes can often be configured in different ways. For example, axes can be orthogonal or parallel, positioned horizontally or vertically, rectilinear or polar, juxtaposed, overlayed, or nested. There are many possible variations. Take, for example, the chart mockups I created below.

They represent the same data but use different configurations, and as such, they can prompt the reader to extract different information.

Another common spatial variation is ordering. In many situations, the ordering of graphical elements in a visualization is arbitrary; therefore, the elements can be sorted using different criteria. The classic example is sorting a bar chart alphabetically or by ascending or descending values. You can rest assured that when sorting changes, our data processing changes as well. Sorting a bar chart is a simple change, but there are many other situations where sorting can have a very strong impact on interpretation.

Scaling

When we map a quantity to a visual feature, such as the length of a bar or the area of a bubble, we need to specify three elements: the data scale, the visual scale, and the mapping function. The data scale defines the minimum and maximum values considered in the data. The visual scale defines the minimum and maximum values used for graphical objects (e.g., the length of bars or the size of bubbles). The mapping function defines how to map data values to visual properties. Depending on how we define these values and the mapping function between data values and graphical properties, we can obtain radically different effects. For example, it makes a huge difference whether values are mapped to visual properties linearly or through another function. Similarly, it makes a big difference if we start the axis at zero or at the minimum value found in the data. The classic example of this effect is the “truncated axis” problem.

If we start the axis at a value greater than zero when creating a bar chart, we can significantly alter perceptions of differences between bars. But this problem is not restricted to bar charts, and it’s a general principle that applies to all mappings that exist between data values and graphical properties.

Data-transforming reconfigurations

In the previous sections, I stated that I am considering only how visual representations of the “same” information affect visual perception. The reality, however, is that often moving from one representation to another requires data transformations to accommodate the new representation. In other words, it’s not possible to use an alternative chart without transforming the underlying data first. Many of these cases exist. For example, a stacked bar or a pie chart requires transforming the values into proportions. A strip plot requires showing individual data points instead of aggregate statistics. A cumulative plot requires computing the running sum of all values.

A notable example is the cone-of-uncertainty plot and its alternative based on hypothetical trajectories. These plots have been studied extensively (e.g., in several studies by my colleague Lace Padilla, like this one), and the results of many studies indicate that people reach different conclusions depending on which one they use.

We may informally describe these charts as alternative representations of the same data, in the sense that the source data is the same, but they do require different intermediate transformations. In all of these cases, the charts can change dramatically, and because of that, they can communicate completely different messages.

Conclusions

This third post in the “VisThink” series covered how visual representations may lead readers to extract different messages from the data. This kind of knowledge is useful for both readers and designers. Designers can use these concepts to reason more effectively about available choices and their potential effects on interpretation. Readers can read charts more critically because they can better understand how the visual design employed by the designer induces them to privilege some information over others. All of this complements the ideas presented in the previous two posts, which focused on assessing data quality and biases, and understanding the effects of data transformation to visual representation. Now, you are equipped with almost all the elements needed to critically evaluate any visualization. The only missing element is the elements that contribute to framing a visualization, namely, titles, annotations, etc.

To test this idea, I decided to use one of my old papers titled “Taking Word Clouds Apart: An Empirical Investigation of the Design Space for Keyword Summaries.” In that paper, we created a design space for keyword summaries, namely, a list of words that summarize a set of documents and that are assigned quantitative values to each word (e.g., frequency or relevance). The design space was organized around two main axes: spatial layout and value encoding.

For spatial layout, we had three options:

• Column: arrange the words in columns

• Row: arrange the words in rows

• Spatial: arrange the words in a 2D “spiral” layout

For value encoding, we had the following options:

• No encoding: no value associated with the words

• Font size: font size encodes the value

• Color intensity: color intensity encodes the value

• Bar length: bar lengths encode the value

• Circle area: area size encodes the value

A summary table with examples appears below.

How difficult is it to reproduce this design space? Not hard at all. The first step I took was to upload the paper to Claude to provide the necessary context.

Then I specified that I was specifically interested in replicating the experimental simuli and asked it to produce a little interactive artifact. The first result was already quite good, but I wanted to refine a few elements and eventually developed this useful simulator of keyword summaries.

It’s much more satisfying to see this in action. Take a look at the video I recorded below, and you’ll see how the mini-app looks and how easy it is to get a sense of how effective different types of combinations could be.

One important pattern that I have observed while developing these mini-apps is that it is easy to generate ideas to expand on the original idea. For example, our initial space did not include color as a secondary variable to map onto the words, in addition to quantity. However, with LLMs, this requires only one additional prompt. This is an example of a keyword summary that employs an additional feature we did not anticipate in our paper.

What does this mean for the future of user studies? I do not have a definite answer; everything looks unpredictable lately. However, I know that reproducing experimental conditions has become increasingly easy, which could lead to more replications in the future. If you consider that subject recruitment can also be automated with platforms like Prolific, you can envision a time when going from idea to results could be surprisingly easy and quick. Add to this statistical analysis and reporting, and it’s not hard to imagine how we could soon have a proliferation of studies. Whether this is good or bad remains to be seen, but it seems clear that we are going in this direction.

And of course, this is not restricted to replication. The same logic can be applied to the design and deployment of new studies. Generate an idea; develop the experimental stimuli, tasks, and performance measures; implement the study; collect the results; and report the results. I am simplifying, but you see where I am going? It’s not that far-fetched!

I want to conclude with a related note. These small experiments led me to realize that design space exploration has become surprisingly straightforward. Since it’s now possible to quickly iterate on many ideas, it is also possible to quickly “experience” what a given visualization idea looks like. In turn, this makes it way easier to quickly pinpoint what is worth studying. Before the advent of LLMs, implementation was costly, and a lot of careful thought was required to determine what to test. But now you can just experience it with your eyes first and then decide. Again, the same question: is this good or bad? I tend to think this is good overall, but I am concerned that it can easily lead to sloppy or lazy work if one does not engage meaningfully with these tools.

And you? What do you think? Good/bad? Exciting/scary? Let me know what you think by leaving a comment below!

Leave a comment

• Misleading → Onus on the sender

• Misinterpreted → Onus on the receiver

It’s not an insignificant detail. When we label a visualization as misleading, we implicitly suggest that something is wrong with how the data is presented. Conversely, when we label a visualization as misinterpreted, we imply that the reader is at fault for misinterpreting the data.

While the two concepts seem separate, they are more closely linked than they appear. A visualization can lead readers to draw incorrect conclusions, but if a reader is attentive and sufficiently skilled, they can readily identify the problem. Conversely, a visualization may be fundamentally legitimate, yet the reader may draw incorrect conclusions due to a lack of necessary skills.

In visualization research, we tend to treat these two aspects separately. On the one hand, research on misleading visualization focuses on the mistakes designers make. On the other hand, research on visualization literacy focuses mostly on measuring people’s ability to interpret data from charts correctly.

This dichotomy between misleadingness and misinterpretation is important because, depending on where we focus, the interventions differ. Misleadingness leads us to focus on the information producer. How can we prevent or detect misleadingness? Misinterpretation leads us to focus on the consumer. How can we empower readers with the skills to reason effectively with charts?

If you think about it, an extreme view of misleadingness is that it’s nothing more than a lack of awareness and skills on the reader’s part. If something is misleading, the reader should be able to capture it. Of course, this is absurd, because we can’t expect everyone to have the highest possible data-reading skills. Also, designers and communicators have an ethical duty to convey information as objectively and transparently as possible.

Another extreme view is that if misinterpretation exists, it’s always the designer’s fault. This is also problematic for at least three main reasons. First, designers can’t possibly anticipate all the ways in which interpretation can go wrong. Second, they can’t design for a million different profiles. One solution could work for one person but not for another, so the “perfect” solution may not even exist. Third, every time a choice is made about what to represent and how, designers implicitly exclude other solutions, thereby potentially concealing information. There is no such thing as representing all the information there is. Designing is the act of choosing, and when you choose, you exclude.

There is more.

Let’s dig deeper into designers and readers.

Designers

Over many years of working in this space, I have become convinced that most misleading visualizations do not stem from malevolent intent. They stem from several factors that can co-occur and self-reinforce:

• Lack of skills/awareness. The designer is unaware of a problem with their visualization. The problem stems from a lack of skills and awareness.

• Lack of time and other constraints. Many designers work in a fast-paced environment with many constraints. Above all, this is true in data journalism, where graphic editors must find a solution to a given data communication problem within a limited timeframe.

• Narrative-first thinking (motivated reasoning). Many people approach data visualization with a preconceived notion of what they want to show. Sometimes even before they dug into the data!

I have also become convinced that many misleading visualizations stem more from specific choices of data set, statistics, and framing than from the visual representation one chooses to use. This indicates that an excessive focus on visual representation does not fully capture the problem, and both designers and readers should be aware of this.

There is a great paper that discusses exactly this idea, and I encourage everyone to read it. It’s titled “Misleading Beyond Visual Tricks: How People Actually Lie with Charts,” and it’s one of my favorite papers of the last few years.

Readers

It is evident that a lack of awareness and skills is a significant factor in misleadingness and misinterpretation. Many experiments have shown that people struggle to read even basic charts correctly. If one cannot understand what a scatter plot is, how can we expect them to capture subtle misleadingness stemming from how the data was collected or the specific angle the author proposes?

Another important aspect, however, is the extent to which the reader is aware of the problem and willing to act on it. Somehow, what is lacking is often his 1) the notion that data is not necessarily objective, and 2) the skeptical attitude necessary to become a more critical thinker of data and charts.

In other words, while skills are important, awareness and attitude are even more important. I have no idea how to address this, but I think that visualization educators should create more materials and learning opportunities to cultivate this attitude and to understand how data can easily mislead people.

In my own little corner, this is exactly what I am trying to do with my research and my online courses. My course on “Thinking Effectively with Data Visualization” is designed to develop that awareness and the skills needed to become a better data thinker. My hope is that I will be able to reach more and more people with this kind of material.

—

Of course, I can’t end this post without talking at least briefly about AI! I have been interested in how AI could play a positive role in this domain. As our information consumption becomes more and more mediated by dialogues with various types of AIs, will it be possible to have an AI to warn us about when our reasoning goes astray? Will LLM be capable of warning or guiding us? Maybe it’s far-fetched or maybe not. If you recall, I have been experimenting with LLM capabilities in this space. This is the post I wrote on this topic, and the results after only a few minutes of testing were not bad.

This occurred approximately 10 months ago. I can imagine that reasoning capabilities might already have improved enormously.

—

And you? What do you think about this whole idea? Please let me know if this rather philosophical yet practical exploration sparked any interesting thoughts. Please leave a comment below and let me know! I am interested in hearing your thoughts.

Leave a comment

Let me take a step back. One of the biggest bottlenecks in teaching visualization is creating material that makes key data visualization concepts apparent. It goes without saying that when one teaches visualization, it is necessary to illustrate ideas through visual examples. The problem is that this need has always been limited by the graphical capabilities of existing tools and the amount of time an instructor could spend on creating new examples. This is why most instructors borrow from one another, use examples they find online and in books, and supplement with visuals created with drawing and charting tools. Creating these visuals has always been slow and tedious. Furthermore, existing tools did not support the easy creation of interactive explainers, which are particularly useful in visualization.

Take teaching color perception, for example. One has to introduce a number of theoretical concepts (trichromacy theory, graphical encoding, etc.), but eventually it’s necessary to show color in action. How do different choices affect color perception? How does proper color selection compare to suboptimal choices? How does a given color map look when used on a map, a line chart, or a scatter plot? These are all questions one can answer by showing the use of color with specific examples.

I suspect this is why tools like Color Brewer become so popular and fundamental in visualization pedagogy.

You can select different color maps, change important parameters, and see the results immediately on a fictitious map. But now, building ten Color Brewers is only a few prompts away! Now I can feed Claude with a set of papers, my slides, a book chapter, or just a prompt, and ask it to build an interactive tool that exemplifies the main idea I want to illustrate. The first result is normally not great, but with a few prompts, I can quickly get to an amazing little interactive explainer.

Once you start thinking about what is possible, the opportunities are endless! Let me share an example I built a few days ago: a little app that demonstrates the concept of pre-attentive processing.

Pre-attentive processing is a concept that visualization researchers borrowed from vision science. The basic idea is that there are visual features we can perceive very quickly, anywhere they appear in our visual field, faster than the time it takes for our eyes to move. These features have been extensively studied in experiments in which a human subject is shown an image and has to determine whether a given target exists. Below is a classic example: deciding whether a red dot is present within a set of red dots, when the image is shown for only about 200 ms.

Creating this little app took me very little time (overall, probably between 30 and 50 minutes) and a handful of prompts. Watch this short video to see what the final version of the applications looks like.

Interestingly, I started with a somewhat vague idea, but as soon as I saw the interface, ideas came to mind for how to make it different and better. This is something I noticed with these tools. Since it’s so easy to create an initial idea, it’s way easier to come up with better ideas because iteration is so cheap. It reminds me of exploratory data analysis, where at first you don’t have a good idea of what insights the data can reveal. However, as soon as you start visualizing it, new questions emerge, leading to better ways to visualize the data.

Pre-attentive processing is one example among many others that could be used to build teaching materials for data visualization. Others that come to mind are: color perception, marks and channels, and integral and separable dimensions, but I am sure there are many, many more to explore. Our curiosity is the limit!

In fact, it may now be possible to build an online repository of these explainers for use in courses. The benefit could be immense if there is enough adoption.

After reading this, please give it a try and let me know how it went for you. Are there any specific data visualization concepts that could be taught with a little app built with an LLM? I’d be happy to explore more. Feel free to leave a comment if you want to suggest an explainer or if you develop one yourself!

Leave a comment

I can’t believe another whole year has passed. In these cozy days between Christmas and New Year, I am taking some time to reflect on what I have accomplished and what I would like to do in the new year. This year has been quite different from the previous year for this newsletter. I have struggled much more to be consistent with my writing, but at the same time, I am happy with the quality of what I have written and the educational material I have developed.

Main Themes

In 2025, I published 29 posts. Far fewer than last year’s 50 posts, but still a lot of food for thought.

Looking at the main topics, it is clear that my main focus has been on:

• AI and Vis. Surprising eh?! AI has become simply to big to ignore. While I am often fatigued by how much of the ongoing conversation is always about it, I have also been enjoying experimenting and learning what AI can do for data science and visualization.

• Thinking with visualization. This is the topic closest to my heart and the drum I have been beating for a while. I truly believe that learning how to think with data is a big mission and you will see more of it here.

• My online teaching services. A sizeable number of posts have been devoted to promoting my course on Maven. All the red dots are about me doing my best to let you know about the learning opportunity I had to offer.

Somehow, I have been struggling much more with my posting frequency. I don’t know if it was more about being overwhelmed by my focus on developing and promoting my online course, or just general fatigue. Looking at my posting timeline, it seems the summer months have been the hardest (maybe I was just slacking off?), and overall, I really struggled to maintain my four posts a month schedule.

Overall, I am happy that I have been focusing on a few topics. I think that going deeper into one or two areas is more effective than trying to cover a few random topics.

Top Posts

Looking at the raw statistics on Substack, two posts stand out as receiving far more attention than the others. The first one is the post in the “Illusion of Causality.” Somebody posted this article on Hacker News, and it picked up like crazy!

I am quite happy with this article because it highlights an underexplored problem with significant impact. We are surrounded by data communication that purports to “demonstrate” that A causes B, either explicitly or implicitly, but most of the time, such claims are unwarranted. Learning how to detect these patterns in charts is crucial.

The second post that went viral is this one on GenAI and visualization. I created this list for my course and thought it would be useful to share it with the world. Of course, these days it’s not hard to attract people’s attention with anything related to AI, but this post turned out to be very useful to many people.

Free Webinars

In the context of my online course “Thinking Effectively with Data Visualization,” I created a total of five free webinars. You can watch them by clicking the links below!

• Same Data, Different Charts: How Visuals Shape Data Messages

• Can You Trust It? 4 Ways Data Visualizations Mislead

• Don’t Get Fooled! Avoid The Illusion of Causality in Charts

• How NOT to Lie with Charts

• Don’t Take Numbers at Face Value: Bad Data Ruins Good Charts

I am super excited about how well attended these webinars have been (I had hundreds of people sign up) and the feedback I received. Next year I plan to keep organizing more of these seminars so stay tuned.

Plans for Year 2026

In the new year, I’d like to keep going back to my weekly post schedule. I want to keep writing about “thinking with data,” explaining why this is important and what we can do about it. I will keep exploring the intersection of data visualization and AI. For sure, you will see more experiments, but also more about how to use visualization to understand AI. I still have a series I started on this topic that I never completed due to the massive changes that happened recently. You will also continue to see educational offerings, but I am considering a new format to explore. Maybe I’ll try to provide more opportunities for hands-on work. Finally, I want to revive my interviews! In 2025, I recorded only one interview, and I wish I had done more!

And how about you? Please let me know your aspirations and whether thereis anything specific you’d like me to do with the newsletter. I’d be happy to receive requests!

Leave a comment

I attended a seminar by my colleague, David Bau, the other day, that got me thinking. David and his group work in an area of machine learning some call “mechanistic interpretability.” In this area, researchers examine models like those behind popular AI chatbots to understand how they work.

You may ask, “Why do we need to do that? If somebody built it, they must also know how they work, no?” The answer to this question is no. The reason why we don’t know how they work is twofold. First, models are trained on massive amounts of data, so the behavior they display is not “engineered” but learned from the data they are fed. Second, and probably more mind-blowing, the behavior we experience is an emergent property of a relatively simple prediction task, which is about predicting the next word in a sentence (the prompt you write when you write something in your favorite LLM). Let me repeat this last concept. All the amazing answers you receive from AI chatbots result from systems trained to do one thing: predict the next word. These systems have not been engineered by someone expecting these models to behave so coherently and to perform these amazing features. It was in all regards a “discovery.”

Why does this whole premise matter? It does because my colleague David, as well as many other brilliant scientists, study how these AI models work. In a way, and this is what I find most fascinating, humans have invented machines that we now need to study the same way we’ve been studying natural phenomena. The parallel with biology, or even more with neuroscience, is staggering. When we don’t know how something works, we start observing, prodding, hypothesizing, and looking closely until we can explain it and predict how it will behave. And this is what a small group of scientists like David is doing with AI models: they put them metaphorically “under the microscope” to derive and test hypotheses about how they work. This entire area of research is called “mechanistic interpretability,” and it’s a small niche full of dedicated people. If you want to get a taste of what this research looks like, take a look at the work done at Anthropic and Google PAIR, where researchers have been studying ML models for years.

An example: The Logit Lens

David presented a tool they developed called Logit Lens. You can see an example of the main data visualization the tool uses. This tool’s goal is to understand how information is processed within a transformer model, the backbone of modern language-based generative AI. Without going too much into the details, these models work as follows: given a piece of text, they predict the next word, then they use the new sentence containing the new word as an input to create another word until a stopping condition is met. This is what happens when you use your favorite AI tools.

The visualization below shows how this information is processed internally by the network.

The rows represent the tokens fed into the model. This is the equivalent of the prompt you write when you ask something to tools like ChatGPT (in many models, words are further split into smaller parts rather than whole words, as in the figure). The columns represent the layers of data processing steps that exist internally in a transformer model. In these layers, information is processed to capture increasingly abstract concepts. The words you see inside each cell are a representation of the main concept taking place within the network at the corresponding level. If you wish, you can interpret each word as what the network predicts as the most probable next token in that specific location of the network. The last row is very important because it represents what the model would predict if the computation were stopped there, at that layer. The bottom-right cell is even more important because it contains the actual prediction of the last word.

Let’s analyze the figure above a bit more closely. I typed “I like to drink tea with a bit of,” and the next word prediction (check the bottom-right cell) is “milk,” which makes a lot of sense. If you look at what the network predicts in the previous layers (see the last row), you’ll see it’s “sugar” until almost the end, then it switches to “milk.” If you look at the last column, you can also see what the model predicts as the next token at the previous step, as if I had not provided the full prompt. Check two rows to the last, which corresponds to writing the prompt “I like to drink tea with a,” and you’ll see that the model prediction is “friend,” which is quite plausible.

I hope this gives you a glimpse of how tools like the Logit Lens can help us understand how these models predict and reason about the input we provide.

If you want to learn more about how the Logit Lens idea works, I strongly suggest reading this essay on Less Wrong, which provides a more detailed explanation of the technique and several examples of insights one can derive from it.

What’s the role of visualization here?

After showing the examples above, I think it’s not too hard to imagine that data visualization could play a role here. Neural networks are complex objects that process a staggering amount of data. When you have plenty of data, vague goals, and the need to generate new hypotheses and intuitions, visualization is the most powerful tool in the data toolbox.

Visualization researchers have developed many techniques over the years to understand other types of neural networks, and new methods are being developed every day. However, my cursory look at the visualization academic community suggests that there is still too little focus on the problems of mechanistic interpretability. The most exciting work seems to come from industry labs rather than academia. Maybe a reflection of the fact that we don’t have easy access to production models and compute.

However, I am convinced that visualization can play a big role in this space. I have been interacting with my colleagues over the last few weeks and have been able to provide useful advice on visualization design. Visualization experts can help at least with three bottlenecks I see.

1. Scalability. The amount of data AI models must handle is enormous. Making visual representations scalable is an old problem with many potential angles of attack. Visualization experts can suggest ways to navigate and summarize large quantities of data in meaningful ways.

2. Abstraction. Visualization is not only about mapping data to graphical representations but also about designing useful abstractions and metaphors. Visualizing data processing in neural networks requires developing such abstractions, and visualization researchers are accustomed to inventing them.

3. Interactivity. When data is complex, large, multidimensional, and multimodal, there’s no single representation that can help answer all the questions. Interactivity is crucial in these cases because it helps define clear goals and adapt visual representations accordingly. Again, this is what visualization researchers have studied for almost 50 years and are well-equipped to suggest and invent clever interactive solutions to investigate complex neural networks.

While visualization researchers have developed many solutions for visualizing neural networks, there is still relatively little awareness of “mechanistic interpretability” and the fantastic opportunities that exist when focusing on supporting researchers who study how generative AI models work.

I hope to see more people involved in this area. I am myself moving first steps in this direction, and I hope I’ll be able to write about my progress here. Let me know what you think!

Leave a comment

Anyway, since my last update, I have published four new posts. The first one was a new entry in my series on “Thinking with Data Visualization.”

In this post, I focus on the role data transformation plays in visualization and how the way data is transformed influences interpretation. The main message is that data is extremely malleable, and data visualization designers must handle this aspect carefully, because inappropriate calculations can lead to many troubles.

The second post was another one on the use of AI in visualization. This time, focusing on the transparency problem.

While I have touched on this problem before, in this post, I go deeper into recent techniques for addressing transparency issues in LLM-generated data pipelines.

This post pairs well with the latest post I published, which is also about AI.

In this one, I focus on something I have noticed while performing visual data analysis with ChatGPT and other common AI tools. LLMs have made it possible to do things with data that were previously impossible or at least very hard. In the post, I talk about what these things are and my experience with them.

Finally, in mid-November, I also posted a personal selection of presentations I have attended at the IEEE VIS conference.

I have been so happy to attend IEEE VIS again this year. I have been attending the conference so many times over the years, and every time it feels like a special occasion to meet people I have known for ages from all around the world.

November has also been a special month because I completed the second cohort of my online course “Think Effectively with Data Visualization” (formerly titled Rhetorical Data Visualization). Teaching a new group of learners has been really exciting. Their engagement and acumen made every live meeting interesting and fun, and gave me an opportunity to assess and solidify the material. I plan to write more about the course, but I must say I am very satisfied with the results. Even though all the participants were highly advanced professionals, they found significant value in the course and the framework it presents. This gave me even more confidence in its value and the need to make this material accessible to as many people as possible. One of my goals in Spring will be to find new ways to deliver the main messages in this framework and to complete the series of posts I started here in this same topic.

If you are interested in taking the course with me, send me a message. I am considering delivering a new cohort at the end of January or early February, depending on how many people enroll.

In the next few weeks before the Christmas break, I intend to write a few more posts. I definitely want to publish at least one new article for the VisThink series. I am not sure yet what else I want to publish, but I have been inspired by many papers I saw at the VIS conference, so I may end up writing about one or two of them. One idea I am considering is to experiment more with videos, specifically by giving an overview of individual papers I liked. If you like this idea, please let me know by leaving a comment.

Leave a comment

That’s all for now. Thanks for reading!!

When you work with AI chatbots to carry out data analysis and visualization tasks, the main limitation is often your imagination. The more I experiment with LLMs, the more I discover new ways they can help me work with data in unique and unexpected ways. I thought about sharing some of the things I tried in the hope that this will help you expand the range and learn new ways to use AI tools with data. Here is a preliminary list of things I tried in data-driven projects using LLMs.

1. Search for the data you need. Before the advent of LLMs, you had to scour the Internet to find the data you needed. Now, you can ask your AI to look for it, and you’ll get a lot of useful information and options. I have tried a few times, and I was impressed by the results. You get access to data sources and a thorough description of the information they contain and how to access them.

2. Describe the data. When you obtain the data from a source, you first need to familiarize yourself with the source, the data structure and meaning, and the limitations. Now you can just ask your AI to do that for you. All LLMs I tried are really good at providing data summaries. They all tend to guess correctly not only the structure but also the meaning of each field, which is quite remarkable.

3. Describe the domain. You are not always familiar with the problem you are investigating; therefore, learning about the domain becomes crucial. Without domain knowledge, there is always a big risk of misinterpreting the trends and patterns one uncovers from the data. Now, AI tools can provide you with full context if you want to learn more. This is a remarkable capability. In the past, data and domain understanding tended to be very distinct and separate tasks; now, with AI tools, they are interrelated. But there is more than that: researching a specific domain has become way easier than in the past. Now you can ask specific questions about a specific domain problem, and an LLM will provide answers tailored to your specific needs and in a format more congenial to your learning style.

4. Suggest questions to ask. This is a completely new capability. If you ask an LLM to suggest questions to pursue, it will suggest ways in which you could slice and dice your data. Maybe you don’t need it, but every once in a while, it can suggest ideas that you would otherwise have pursued. When I feel like I am running out of creative steam, an AI can suggest new angles to look at a given data set. So after I have created a few charts and answered a few questions, I may ask, “What else should I ask here? Are there any other questions I should be asking that I have missed so far?”

5. Answer the questions with tables and charts. All major AI tools can now produce tables and charts when asked to perform data analysis and visualization. All tools have their own idiosyncrasies, but overall, they can do well if you know how to “tame” them. So far, I have had the best experience with ChatGPT: it’s much faster, more reliable, and better at understanding instructions.

6. Interpret the results (and debug your thinking). When you generate a chart, you can ask an LLM to describe what it sees and what the relevant trends are. I don’t think this functionality is completely developed yet, but if you ask, they answer something that makes sense most of the time. It’s particularly interesting that they can help you spot errors in your reasoning. In a past post, I tried to verify if ChatGPT can spot erroneous interpretations of data, and I was surprised by its evaluation skills. This means AI can also be used to debug your own thinking, which is one of my favorite approaches to AI use overall.

7. Suggest alternative ways to visualize the data. If prompted correctly, AI tools are capable of suggesting alternative ways to visualize the same data. I have tried a few times and, anecdotally, I was impressed with the results. So, if you are not sure how to visualize something or you feel there might be a better way than the way you suggested to use in your prompt, ask the AI to provide alternative recommendations and you’ll have a broader set of solutions to explore (I wish that would automatically lead to the production of multiple charts, but this does not happen automatically yet and if you ask the results are not great - current tools are not very good at producing multiple charts at once).

8. Explain how it created the results (transparency). Transparency is the biggest hurdle with AI tools right now. It’s easy to go from prompt to charts or from prompt to code to charts. But these solutions are not particularly transparent or interpretable, especially if you are not used to reviewing code yourself. One possible solution is to ask the LLM to explain how it arrived at the solution. This works well for code reviews because it explains, step by step, what the code does. However, it’s important to be aware of these explanations because the LLM may try to convince you that everything is right when, in fact, some errors exist in the data processing pipeline. Here is where data skills are absolutely necessary. There is no way you can evaluate the steps followed by an LLM unless you have a strong mental model of what operations make sense. It’s not about being skilled with code. Even when understanding code fully is not required, it is still necessary to understand precisely what a given data operation does and its implications. This is non-negotiable for anyone working with data.

9. Explain how to do something with code (or other tools). One exciting aspect of how “supple” LLMs are with code is that you can “code” and understand code even though you are not particularly proficient. The secret for me is to never ask too much and to break problems down into smaller, self-contained ones. If you ask the LLM to perform one step at a time and to describe to you what the code does, you can, most of the time, understand what is going on. This is also true when you know how to code part of the steps but need help with specific subtasks. LLMs are really remarkable with code, and asking them to solve smaller problems you can completely grasp is often the right way to go. Interestingly, this is not restricted to code. If you don’t know how to do something with, say, Tableau or Excel, you can always ask what the best way is to do it, and most of the time, you receive good guidance. I tried a few times and I have been quite happy with the results.

All these tasks come from my own experience testing LLMs for data analysis and visualization. Mostly out of curiosity to see what they are capable of and how the existing LLMs compare to one another. It goes without saying that in all of these tasks, LLM can get trapped into something and return terrible and even misleading results. Just in case you need to hear that: there is no substitute for critical thinking and evaluation. All the answers to these questions can be wrong. Terribly wrong. But in most cases, they are not and are incredibly helpful instead.

I am sure that in the future I will discover even more interesting ways to use LLMs for data analysis and visualization. I am also sure that we will soon use more specialized tools alongside general-purpose ones.

In future posts, I’d like to show you in more detail how some of these specific tasks can be carried out with LLM. If there is one in particular you are interested in, please let me know, and I will write a post about it. Also, if there are other interesting ways you use LLM for similar purposes, please let me know. I’d love to hear about your experience! Leave a comment below.

Finally, I hope to develop a new course in Spring/Summer based on these ideas. If you are interested, please leave a comment below and let me know what you’d like to learn. I think it would be really exciting to teach these skills to other people.

Leave a comment

Last week I attended the IEEE VIS conference in Vienna, the premier conference for data visualization research. It’s been a whole whirlwind of events, presentations, exhibitions, conversations, etc. The program has grown so large that it is impossible to attend all the sessions and stay up to date on all the contributions. I spent most of my time talking with people, but every once in a while, I dropped into a session that looked interesting. Here are some paper presentations that caught my attention. For each one, I’ll explain why I think it’s relevant work and you should take a look. Of course, there is much more than that, and I am sure there are many other hidden gems to discover in the program. If you want to explore the entire collection of papers, I strongly suggest you use John Guerra’s paper explorer. It’s an interactive visualization that lets you navigate the entire collection by your interests.

As you go through my list, keep in mind I have only attended the presentation and skimmed the papers in some cases. There may be many more details I am not aware of. Enjoy the curated list and let me know what you think!

Papers

👉 ReVISit 2: A Full Experiment Life Cycle User Study Framework

Creating useful infrastructures for visualization research is a hard task. Doing it while receiving a best paper award is even harder, because most researchers are skeptical about the research contribution of this type of work. ReVISit is a well-deserved exception to this unwritten rule. The authors have done a great job at developing a tool that is useful and potentially groundbreaking. Doing user studies for visualization research is very hard. reVisit simplifies many aspects by providing a scaffold for experimental design and automating or facilitating the most boring and time-consuming tasks. I was skeptical, but I completely changed my mind. If more people start conducting studies with reVisit, we will soon have a library of comparable experimental designs, which could be really valuable for understanding visualization research and for building on each other’s work. In our lab, we will soon try it out to see whether it helps or provides too much friction compared to our usual workflow. At this stage, I think it’s well worth the effort to at least try. If you want to try it yourself, you can find the project here: https://revisit.dev/.

👉 Beyond Problem Solving: Framing and Problem–Solution Co-Evolution in Data Visualization Design

This is also a best paper award. The authors analyzed the work of 11 professional visualization designers and derived a set of considerations regarding how we conceptualize the data visualization design process. The outcome is an interesting critique of existing models and a more grounded description of how design evolves over time in a project. I will not try to summarize everything here because it’s quite nuanced, but one strong point is that the project's requirements, as well as the conceptualization of the tasks to be performed with the visualization, evolve over time as the project unfolds. In other words, it’s only by doing the work that we understand what there is to be done, and we normally can’t have a full specification before we get started. The implications of this are many, including how to interact with clients and define project scope. I did not read the paper fully yet, but I want to dig deep into it because the dynamics it describes resonate very much with my own experience working on data visualization projects (by the way the same dynamic has always been try for me in developing research projects - I never know where I am going fully but I know that if I move a few steps I’ll have a clearer picture).

👉 Qualitative Study for LLM-assisted Design Study Process: Strategies, Challenges, and Roles

This is one of the innumerable papers at the intersection of AI and visualization presented at this year's conference. This one is particularly relevant for me because it covers an aspect of AI that I have been trying to develop in the newsletter: the role AI can play in the data visualization design and development process, and how it allows us to do things that were not possible or too hard before. In this paper, the authors use the well-established “design study methodology” developed by Sedlmair et al. about a decade ago and reflect on how AI tools can support the methodology's steps. The study is based on interviews with 30 visualization experts and provides a rich mapping between the development stages and the opportunities enabled by AI tools. They identify four main assistive roles for AI: Connector, Assistant, Simulator, and Programmer, and for each describe in detail how they map to specific stages of the development process. This is definitely a paper I want to read in more detail to compare their characterization to the one I have been developing over the last few months (be on the lookout for a new post on this topic soon!)

👉 “It looks sexy but it’s wrong.” Tensions in creativity and accuracy using genAI for biomedical visualization

This is very much in the same vein as the previous one. It’s another interview study to understand attitudes towards the use of GenAI in visualization. In this paper, the focus is on biomedical illustrations, particularly through a more critical lens that contrasts the many promises of AI with its actual implementation. The authors interviewed 30 expert illustrators and gathered information about their attitudes toward the use of GenAI in their work. The study produced a categorization of experts into four-quadrant plots, arranging individuals along two attitude axes: enthusiast vs. skeptical and adopter vs. avoider, which I think could be applied to other domains. The discussion session also includes an interesting set of observations comparing the speculated and the current landscape of GenAI. P.S. Isn’t the title great?!!

👉 Stitching Meaning: Practices of Data Textile Creators

This paper is the brainchild of our student Sydney, who, together with Eduardo, decided to investigate a very interesting phenomenon: many people around the world are creating textiles based on data. The most popular instantiation of this idea is the “temperature blankets,” a stitched textile people create that records the temperature at their location. It turns out there is a whole world behind textile visualization with a big community of people loosely connected through social media and private channels. The paper is based on the analysis of 159 artifacts and a survey with 61 creators. The result is a comprehensive taxonomy of visual patterns and practices, along with insights into the motivations and meanings of the people who create these beautiful artifacts. On a side note, Sydney gave a fantastic talk. Full of energy and beautiful pictures displayed on the big screen. She also carried on stage a data jacket and a pillow she had made by hand! I am very proud of her and the joy she brought to the room. You can find more information about her project here.

👉 Shifting Expectations for Encoding Rules Mitigates Misinterpretation of Connected Scatterplots

This paper also comes from our lab, but I was not involved in it. This is the work of our student Wen Xu with my colleague Lace Padilla (my go-to person for anything psychology on vis). The paper introduces the concept of “expectation” in visual representations. Expectations are ways in which you expect a chart to behave. For example, in a line chart, you expect time to go from left to right and values to increase upward and decrease downward. Expectations can be learned or innate, and they are everywhere once you start looking. The paper uses connected scatter plots (CSPs) to demonstrate this concept. In these plots, people often carry the idea that up is more in line charts and draw erroneous conclusions because, in a CSP, a line going up does not necessarily mean that a quantity is increasing. To overcome this problem, they test alternative designs that suppress or mitigate the expectation and find that people make fewer judgment mistakes. It’s a really neat study with a strong effect. I have a recommendation: if you read the paper, do not get too fixated with CSPs, and think about how this concept can be extended to many other graphical representations.

👉 An Analysis of Text Functions in Information Visualization

If the conference had a “usefulness award,” this paper would win it this year. This is definitely one of my top favorites. The authors analyzed a large collection of data visualizations, asking, “What roles does text play in this visualization?” In doing that, they create an entire taxonomy of text use in data visualization. This type of work stems from a stream of recent papers that recognize that elements beyond the graphical encoding play a major role in visualization. For example, last year, a similar paper covered annotations. The reason why I am so excited about this paper is that it is very helpful for design and teaching. In designing new charts, it can help authors select their solutions more mindfully. In teaching, it can help me, as an instructor, introduce the concept of text use in a much more nuanced and structured way. Instead of saying, “text is everywhere and very important,” I can explain exactly how it is used and what its function is. I am looking forward to using this work in my courses.

👉 Probing the Visualization Literacy of Vision Language Models: The Good, The Bad, and The Ugly

I was already aware of this paper before attending the conference because it was included in the list of papers we read in my VisGenAI course this semester. The authors embarked on a very difficult project: understanding how visual question-answering AI models interpret data visualizations. To do that, they selected a few open-source models and analyzed their internal attention mechanisms to understand what parts of the image and the question influence the response. The study gives us a glimpse into how these models “reason” and what we could do to improve their capabilities. It also offers a reproducible methodology for investigating the internals of these models, which, to the best of my knowledge, had not been done before for specific visualization literacy tasks. Kudos to the authors for making all the code available and the whole pipeline reproducible! They also have a nice video in the supplementary material that shows their application in action (you’ll need to download it to see it well). I strongly suggest you take a look because it’s really remarkable.

👉 Here’s what you need to know about my data: Exploring Expert Knowledge’s Role in Data Analysis

The ex-Utah group has been pursuing this idea for a few years. When we perform data analysis, the way we use visualization and interpret its content is heavily influenced by our background knowledge (other than our abilities). Surprisingly, this aspect is rarely at the center of the data visualization discourse. Our fixation with representation (pie chart or bar chart, anyone?) excludes super-relevant aspects like this one. In the past, this same group introduced the concept of “data hunches,” intuitions about specific patterns observed in the data that require domain expertise. In this new work, they expand on that by analyzing how data analysts perform analysis and the role their domain knowledge plays in it. They interviewed 14 domain experts and asked for “example[s] where the data just did not look right.” The result is a series of findings on the role of domain knowledge in data analysis. One of the main findings is that analysts are aware of the limitations that exist in data and often supplement these gaps with their domain knowledge. They also found that existing tools do not address the need to share this knowledge alongside data and visual representations. However, the participants also expressed skepticism about a solution that would require integrating yet another tool into their already complicated toolbox. This is a very interesting conundrum that I hope will be solved in future years.

👉 An Autoethnography on Visualization Literacy: A Wicked Measurement Problem

This paper is the result of a large group of people from different institutions who have been working in the visualization literacy space for several years (including my colleague Michelle Borkin and her PhD student Mackenzie Creamer). One of the key problems addressed by visualization researchers in this space is the development of tests to measure literacy levels. This can be useful for a number of purposes, including testing the effectiveness of different educational interventions. The problem with this goal is that visualization literacy is a complex concept, and no single test captures all the aspects one may be interested in. The paper presents an “autoethnography,” an investigation of common practices carried out by the same researchers who developed the study. The result is a series of useful reflections on the state of visualization literacy tests, what is missing, and what could be done in the future. This is a paper I want to read closely because I have been interested in this topic for a very long time. We published an early literacy test paper many years ago, and I have spoken about it in this newsletter for quite a while. Last year, I also organized a successful reading club on the topic where some of you participated. On a side note, Lily Ge gave a really impressive talk. Every few slides, she played a recorded audio snippet of one of the authors/subjects to exemplify a particular point. It was really remarkable and had a fantastic effect. I don’t know how she found the courage to do that, given how often people have technical difficulties at conferences with setups way less sophisticated. It was really impressive to watch. And fun! On another side note, I really like the “autoethnography” methodology. This kind of paper breaks the common skepticism of excessive subjectivity and provides examples of how more subjective methods can be extremely helpful in practice.

--

Before concluding, let me suggest you review my friend and colleague, Paolo Ciuccarelli’s, live posts from the conference. They have a very different take from mine and are very fun and insightful!

• What a start at IEEE VIS

• Another day in AI at IEEE VIS for me 🤖✨ (featuring a picture with your tryly!)

• The Day of Confirmations 😏

• Data, Design and Complexity

• Maybe we shouldn’t be using visualization at all

The problem with this solution is that it’s very opaque. As a user, you see only the response and have no way to understand how the LLM generated it unless you look directly at the code. While this may work for simple code and people familiar with coding, it does not offer a scalable solution for more complex output or a wider audience.

The good news is that researchers have started developing solutions to this specific problem.

After reading a few papers on this topic, I have identified two main approaches: task decomposition and code abstraction.

1. Task decomposition. (“forced transparency”) This solution requires the LLM to split the task into a series of observable sub-tasks. More precisely, to go from a prompt to a final visualization, the LLM must pass through several steps, including query interpretation, data processing, and visual mapping.

2. Code abstraction. (“induced transparency”) This solution starts from the generated code and attempts to generate an abstraction to make the steps implemented by the code easier to identify and connect. Typically, this ultimately involves some form of diagram or visual abstraction that enables the user to get a sense of the main data transformation operations the code performs.

Task Decomposition

The paper “ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Language” offers a great example of task decomposition. In ChartGPT, the task is split into two high-level tasks: data transformation and visual transformation. Data transformation includes the subtasks: select column, add filter, and add aggregations. Visual transformation includes: select chart type, choose encoding, and add sort. In the figure below, you can see the details with a specific example.

The advantage of performing such decomposition becomes evident when you observe the interface they built. The figure below shows the interface.

If you look carefully in the bottom-right corner, you can identify a panel where the model’s decision points are specified. You see the sequence of steps? Select columns, add filter, etc. This is the main advantage of this approach. The interface clearly and transparently depicts the steps, the values, and the rules used at each step.

This is quite neat if you think about it, because you can still interact using natural language with the model, but you can immediately observe how the model interpreted your request and what actions it took to produce a response.

Code Abstraction

An alternative path is to take the code the LLM generated a process it to create an abstraction of the operations and a depiction of the workflow. A solution of this kind is presented in “WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualization.”

WaitGPT parses the code according to a number of predefined rules to reconstruct the data pipeline steps generated by the LLM through code. The generated abstraction is then mapped to a diagrammatic representation that aims to guide the user in interpreting the processing steps.

As you can see, the tool generates an interactive diagram based on the code you see on the left. You can imagine something like that being integrated into existing AI tools as a widget or into other interactive tools like the notebook environment proposed in WaitGPT (see the left side of the image above).

Advantages and Disadvantages

Which of these solutions works best? I don’t think we can give a definite answer, but we can try to reason about the advantages and disadvantages of these two approaches.

The task decomposition approach is very neat and produces highly interpretable results. However, this approach is also way less flexible. What if the specific set of steps that constitute the data processing pipeline can’t express the specific transformations needed to answer my question? Task decomposition also forces systems to choose from a specific set of possible solutions, effectively negating more creative or unconventional solutions that the LLM may be able to generate.

Code abstraction does not have these limitations, but it does have others. First, for every new language, we need a new code parser that can convert code into diagrams. Second, diagrams can become so complex that they defeat the purpose. The flip side of more flexibility is more complexity. If the LLM is free to build solutions of unbounded complexity, then visual representations will also result in high complexity.

It’s difficult at this stage to decide which works best in any situation, and maybe future solutions will feature hybrid approaches that combine the two strategies. Task decomposition could be designed to allow greater flexibility, and code abstraction could be constrained more tightly.

I am curious to see how research in this space will evolve over the next few years or months (or should I say days?!). Even more relevant is to see whether these solutions will be included in any widely adopted tool. All major LLMs can already ingest data and produce charts, but none, as far as I know, address the transparency problem these methods aim to solve. Tools like ChatGPT, Claude, and Gemini could easily integrate these solutions into their current toolbox, and specialized data analytics tools could do the same.

Other Solutions?

One final question is whether these two approaches are the only ones available to increase transparency in data processing pipelines. What else could we possibly do?

One option is to have LLMs explain the pipeline to you. This is already possible with existing tools. You can ask your LLM to explain what the code does to you, and, of course, you can also instruct it to give you the explanation in your preferred format. In a way, this could work similarly to the “chain-of-thought” style we see in the “thinking” mode most LLMs today use. The LLM can lay out a visible pattern so that you can better understand what it has done.

Is that enough? I don’t know. I suspect that a more specialized solution would ultimately be more effective. But one could merge the self-explanatory capabilities of LLM with more structured approaches. That is, I could ask an LLM to process the code and generate an abstraction of the code that I could then visualize or communicate effectively.

What else?

Maybe transparency isn't that big a deal, and future improvements to these AI tools will render translation unnecessary. LLM could, for example, detect the need for disambiguation when a prompt is unclear and, at the same time, provide multiple correct solutions for the user to inspect and choose from.

What else?

I don’t know! If you have any ideas, please leave a comment below. I’d be curious to hear what other solutions could be ideated.

Leave a comment

The series mirrors the main elements that I teach in my online live course, “Thinking Effectively with Data Visualization.” The course helps you gain confidence in your data thinking and avoid costly mistakes. You can watch the first lecture for free by following this link: 👉 Introduction to VisThink. If you are interested in the course, see the details on the course webpage.

In the previous post of the series, I argued that visualization designers and analysts need to be very careful in understanding and evaluating the data they use, because the way the data has been generated and its true meaning have a huge impact on what is possible to infer and whether what is communicated with a chart is trustworthy.

Once this is cleared, we still have an important step to address before deciding which chart is more appropriate for our goal. We have to transform the data to give it the “shape” necessary to communicate the information we want.

If you think about it, it’s almost never the case that we receive or generate a data set and can directly visualize it without at least some minimal data processing. At the very least, we need to decide which available variables to use to create the chart we want.

My goal here is to help you develop an appreciation for how malleable data his and how the choices you make when transforming the data impact what is eventually communicated. I like to say that “data visualization” is a bit of a misnomer because most of the crucial choices we face as visualization designers involve deciding what “data shape” to use to communicate the information we want to share, rather than the visual forms needed to communicate it.

Data transformations

Creating a catalog of all possible transformations is almost impossible and potentially very tedious. Here, we will cover the most common operations one encounters when generating new visualizations.

• Variable selection. Data sets almost always include multiple variables, but each chart typically depicts only between two and three. Selecting which variable to use is not always straightforward, and alternative selections may be valid for the same question or communicative intent. For example, if I want to visualize how dangerous it is to drive in different areas of New York City, should I use the number of collisions, injuries, deaths, or something else?

• Aggregation. While not always necessary, in many cases, we need to aggregate data to depict the trends we are interested in. If I want to visualize how different areas of New York compare in terms of collisions, I need to aggregate the data by area (e.g., zip code) and count collisions. Here again, not all aggregations are equal, and different aggregations are possible for the same intent. For example, do I depict the data at the level of boroughs, zip codes, individual intersections, or something else?

• Statistics and calculations. Whenever we aggregate data, we need to specify an aggregate statistic to use. Do we use average, mean, percentage, or something else? It depends. Different choices in this space significantly impact what we can see and infer from the data.

• Filtering and ranges. Finally, it is not uncommon to need to focus on a specific subset of items or a specific value range in a visualization. However, the decision of what to include and what to exclude significantly impacts the meaning and context of the data. For example, if I decide to visually only see some types of collisions, I will not see how the selected types compare to the ones I decided to remove.

Side note: Before moving forward, I strongly suggest that you review my previous series on the role of data transformation in visualization. It’s very thorough and includes numerous examples of how specific transformations affect a visualization's appearance and what it communicates.

These are the most significant and common transformations we need to apply when creating new data visualizations.

How does transformation impact interpretation?

Transformation has a huge impact on interpretation for two main reasons. First, because the choices we make have an impact on what “constructs” we use to depict a given phenomenon. Remember, data is not reality; it’s a coarse depiction of reality. An expansion of our senses. Whenever we use a set of values to represent real-world objects, we have to keep in mind that a gap may exist between what we think these values represent and the reality they represent. This is what we called “data-reality gaps” in the previous post of the series.

Second, data transformation affects how data is visually represented. More specifically, different choices can result in (1) the need to use a different type of representation and (2) the visibility of different patterns in the data.

One of my favorite examples is this pair of maps. If we decide to aggregate the data at the individual zip-code level, we can create a map like the one on the left. If we want to get rid of artificial boundaries and create a density distribution from individual points, we can produce a map like the one on the right. Both stem from the same data set, but different transformations (zip code aggregation vs. density estimation) yield different representations and, more interestingly, different discernible patterns.

Going back to the set of transformations we covered in the previous section, let’s now summarize how they affect interpretation.

For variable selection, the impact is quite obvious. If you change the variables you depict, you focus on different aspects of the data. As mentioned above, sometimes different sets of variables are compatible with the same question, but they are not necessarily equivalent.

For aggregation, the main impact is the level of detail used to depict information. Let’s take a look at a few examples to understand what this means, starting with geographical data. If I have a set of events or objects with a spatial location, I can map these objects individually (highest granularity), aggregate them at a fine set of regions, or at progressively coarser regions. The same is true for temporal data: I can aggregate it at the day, month, or year level.

The impact of choosing any particular granularity level can’t be underestimated. Some patterns are only visible when a specific level is chosen. In particular, the choice of depicting individual items versus aggregations has, in some cases, a very large effect.

The other big choice is which statistics or calculations to use when aggregating data or calculating a derived value. All calculated numbers are, by definition, an abstraction, and each abstraction carries particular benefits and dangers. Averages, for example, reduce the noise of individual values and help make a general trend more visible. On the other hand, by hiding individual details, they also carry the risk of excessive simplification.

Finally, filtering and range determine what is and is not shown in a given visualization. The effect of this choice is crucial because visualization is fundamentally about comparison, and comparison occurs among the objects displayed, not among those not displayed. When we choose to filter out, we also choose to present data in a particular context. This is related to the WYSIATI (What You See Is All There Is) effect, described by Kahneman in his classic “Thinking Fast and Slow.” We tend to make judgments based on what we see, not on what is not presented.

Data transformation and “visual reasoning fallacies”

The ultimate reason data transformation is so important is that it shapes how we reason about data in a visualization. For each of the data transformations I have outlined above, there is one or more associated visual reasoning fallacies one can fall into, if not careful.

I will not cover the fallacies here. This post is already quite long. But, I plan to cover them in future posts of this series. These are the fallacies I plan to cover:

• Base rate bias

• Regression to the mean

• Hidden variability

• Sensitivity to outliers

• Faulty percentages

• Misleading ranks

• Cherry picking

• Ecological fallacy

• Simpson’s paradox

It’s important to note that many of these are often described as traditional statistical fallacies, which they are. However, my focus will be on how these fallacies manifest in data visualizations so that when you develop or read a chart, you’ll be able to quickly identify and diagnose the problem.

That’s all for now. Let me know if this post is useful to you! Please leave your comments below and share them with others who may be interested in these ideas! Thanks! 🙏

Leave a comment

Posts

In September and October, I posted a few new articles. There are two on AI and two on thinking with data visualization, which also happen to be the two main topics I am focusing on right now.

This first post made quite a splash (lists of things tend to be quite attractive, not sure why). In the post, I described the collection of papers I gathered for my new university course on Data Visualization & Generative AI. Since I published that post, I've started the course and have a lot more to report about the papers and the course itself. I plan to share more about these in upcoming posts.

In this second post, I wanted to start providing some guidance. I am spending more time experimenting with AI tools for data visualization, and the post reports on five strategies I found useful. This is an area where there is so much more to do! I want to continue experimenting with AI tools so that I can report back on what I learn. Eventually, I’d like to create a new course around it.

This fall, I decided to start a new series based on the online course I teach called “Thinking Effectively with Data Visualization.” The series is a way for me to translate into words many of the concepts that I teach. Eventually, I hope this will lead me to write a whole book about these topics!

In this post, I cover the problem of inadequate and misinterpreted data and how it affects everything else. I also describe the framework I developed to think systematically about limitations in the data.

Online Courses and Webinars

• Online course (new cohort!): A new cohort of my course “Thinking Effectively with Data Visualization” starts on October 27, 2025. If you want to know more, visit the course home page.

• Scholarships (80% discount): If you are a current student in an academic institution, I offer 80% discount scholarships. Just send me an email, and I will send you a code to enroll.

• Free webinars: I recently gave two webinars. You can watch the recordings here:

  • How NOT to Lie with Charts

  • Don’t Take Numbers at Face Value

Help Me Develop New Courses!

I am currently considering developing a new course for Spring. I want the course to be shorter than my current one and address a significant need people have. I am obviously considering doing something on AI for visualization, but I am also open to other ideas. Can you please send me your ideas here or in private? You can just add a comment below, if you wish.

Leave a comment

Research Corner

We have a new paper out! The title is “Cognitive Affordances in Visualization: Related Constructs, Design Factors, and Framework.” This paper is the brainchild of our student Racquel, who together with me and Lace, developed a whole the idea. I plan to write about it more extensively soon. The paper explores the theoretical underpinnings of “affordance” and its application to data visualization. Half of it discusses how affordance relates to other constructs used in visualization, while the second half focuses on a theoretical model for using affordance in visualization. Take a look and let me know what you think! There are a few more things I am currently working on. One focuses on exploring ethics in data visualization, and the other on understanding how charts communicate causality. I will keep you posted as these endeavors develop further.

Vis + GenAI Course

My university course is almost halfway through. We are at the 8th week. The course is a new experiment for me. The students read a selection of papers in the first half of the course, and then develop an application in the second half. We are almost at the end of the paper-reading phase, and I feel like both my students and I have learned a lot. The papers we read are based on the list I shared above. I am looking forward to reporting more about the format I have used for the paper reading phase (based on assigning specific roles to students), and what we have learned from these papers. There is so much going on in this space, and we will see so much more coming up in the next few months. I will be at the IEEE VIS conference in November, and I am sure there will be a ton of new research in this space.

—

That’s all for now. Thanks for reading! 🙏
Enrico.

The series mirrors the main elements that I teach in my online live course, “Thinking Effectively with Data Visualization.” The course helps you gain confidence in your data thinking and avoid costly mistakes. You can watch the first lecture for free by following this link: 👉 Introduction to VisThink. If you are interested in the course, see the details on the course webpage.

One fundamental notion that is often overlooked in data visualization practice is that no amount of design or data processing skills can overcome problems inherent in the data due to the way it was generated and collected. This is a very apt case for the classic statement, “garbage in, garbage out.” I am sure that while you read this, you may find my point obvious and ask, “Enrico, of course I know that bad data leads to bad charts!” Yeah … maybe. But how do you know when you have bad data? How do you judge if a number, for example, represents the thing you think it represents? Do you have a systematic procedure, or do you go by intuition?

In this post, I will share a framework I have developed to think systematically about evaluating your data, so that you can be more mindful of the impact it can have on the validity of your visualizations.

Validity?

Before we move forward, let’s clarify what we mean by validity. If you think about it, a visualization is not inherently valid or invalid until some information is extracted from it. In other words, the thing that is valid or invalid is the information extracted from the visual representation and the data it represents. With visualization, we often have two main situations. When we are producers, we interpret the content and generate our own understanding of the facts depicted by the representation. When we are consumers, we are presented with interpretations of facts that someone else has generated for us (e.g., news media, scientific reports, presentations). In both cases, the output of interacting with visualizations is a series of facts about the world that we, or somebody else, extracted from the data. The relevant issue here is whether these facts constitute a valid inference from the data and the representation or not. Let me provide you with an example to make this concept more concrete before I proceed with more details. The map below shows the total number of vehicle collisions in New York City by zip code. (This is an example I have used before in my newsletter - please bear with me.)

This representation, without a specific interpretation (or extracted message, if you will) is not inherently valid or invalid. It becomes valid or invalid only when paired up with a specific interpretation. Contrast these two possible titles for the same map.

• Title 1: “Collision Hot Spots: Busy Areas Experience Many Collisions”

• Title 2: “Collision Hot Spots: Dangerous Areas to Drive in NYC”

While Title 1 is a legitimate interpretation of the map Title 2 is not. Areas with more collisions (the darker spots) are not necessarily areas where driving is more dangerous, but just areas with more cars. Therefore, the first title is ok, the second is not.

Data Validity

When we derive claims from data visualizations, their validity depends on many aspects. First, the reality that the data represents, namely, the real-world phenomena. Second, the extent to which the data and calculations represent the concepts one uses in their argument. Third, the correctness of the claims with respect to what the visualization shows, that is, whether the visualization actually depicts what is claimed. The first element depends on your knowledge of the domain problem. The second depends on your knowledge of the data and its meaning (and limitations). The third depends on your ability to read data visualizations skillfully and correctly (e.g., your “data visualization literacy.”) Of these three elements, here we focus on your ability to understand the data and identify its limitations.

How do you assess the validity of your data? More precisely, how do you assess the ability of your data to support the claims you derive from it?

To guide you through this task, I have developed a framework that organizes the problems in stages, allowing you to think about data validity more systematically.

Data-Reality Gaps

The framework is depicted in the diagram below, which shows an idealized version of how data is generated. The first element on the left is the “population.” This term is commonly used in statistics to define the idealized set of possible elements that one could pick from to obtain a complete set of measurements. For example, the whole population of a country would be the population, literally, for a data set collecting survey data about a given topic. However, the term population does not need to refer to people only; it can refer to any type of entity. For example, if we want to measure the temperature around the globe, all possible locations would be represented in the population of that dataset.

Every data set draws items from the population through some selection mechanism, which may be more or less explicit. The way this selection is made has a huge impact on what inferences we can draw from the data. Aside from the obvious fact that measuring anything from a subset introduces some uncertainty regarding the accuracy of a given value, the procedures used to derive a dataset can also introduce significant distortions and gaps between what we think the data represents and what it actually represents. In other words, if we are not aware of the difference between what we aim to study and what the data contains, we can easily be led astray. If a mismatch exists between the idealized population and the data, we have what I call a “Representation Gap,” which is the situation where the data does not accurately represent what we assume it does.

Conceptually, however, this is not the only problem that can exist in a data set. A second logical step after selecting the elements drawn from our hypothetical population is to measure values from the selected items. However, measurement procedures can be faulty in numerous ways, resulting in a second gap: the “Accuracy Gap,” which occurs when the values do not accurately represent the measured reality. A good example is a sensor that is malfunctioning or influenced by environmental factors we are unaware of. Another example is the mismatch between what people respond to when they are administered a survey and what they actually believe.

Finally, data is rarely distributed in the format in which it was collected. Most often, data is pre-processed and transformed to produce the final dataset we use for our analysis. This final step, however, creates another level of indirection that often leads to misinterpretation. I call this third gap the “Interpretation Gap,” because its main manifestation is a misinterpretation of the real meaning of the values (often individual metrics) the data contains. The best example of this problem is what I like to call “black-box numbers,” individual numbers obtained through complex calculations and that are (often too casually) used as a faithful representation of the phenomenon they purport to depict. For example, what we commonly call “inflation” is based on the Consumer Price Index, which aggregates the prices of thousands of goods and services into a single number, based on the spending pattern of an “average consumer.” If prices increase rapidly only for a specific set of goods, such as housing and food, the entire CPI may not accurately represent the burden that specific segments of the population experience.

One additional element to consider, in addition to the three gaps, is the “Consistency Gap.” This gap stems from the fact that often data is collected over time and across multiple sources. When relevant differences in the way data is collected or processed exist between different sources or time spans, major misinterpretations can occur. A good example, borrowed from Ben Jones’ book “Avoiding Data Pitfalls,” from which I have also borrowed the term “data-reality gaps,” is this one about earthquakes.

Earthquakes are not really on the rise. What changed is the precision of the instruments used to detect the earthquakes, causing this apparent upward trend. If one is not careful about consistency, apparent trends can easily lead to faulty interpretations.

Performing Data Sanity Checks

The little framework I presented above can be used in practice to perform a sanity check before you start manipulating and visualizing your data. I suggest you ask yourself four main questions:

• Selection: What is included/excluded? Do the included elements match my mental model of the the reality I intend to depict?

• Recording: Are the values recorded in the data realiable and accurate. Are there ways in which the recording process may have introduced distortions and inaccuracies?

• Derivation: Are the derived values valid? Do they capture what they claim to capture? And to they represent the concepts I intend to depict?

• Consistency: Does any of the above change over time or space (or source)? Is it possible that different sources or time periods used different data collection standards?

While this does not ensure you catch every possible limitation in your data, it goes a long way in helping you avoid major flaws and interpretation errors.

One big topic I did not cover here, which should also be taken into consideration, is the effect of missing data. Missing values can also wreak havoc in your data analysis and visualization if you are not very careful in detecting and handling them. I will cover this problem in one of the future posts of the series.

That’s all for now. Let me know if this framework is useful to you! Plese leave your comments below and share it with others who may be interested in these ideas! Thanks! 🙏

I have multiple goals with this series. First, I want to provide you with a clearer understanding of what I cover in my course. Second, by writing and sharing my ideas with you, I hope to refine and further develop them. Third, I am considering writing a book based on these ideas, so writing is what I need to do anyway. I hope you’ll like the ideas in this series. Please share your ideas and comments with me. I’d love to know what you think!

Why focus on “thinking?”

It may seem strange that I even need to argue for the value of effective thinking, but I do think it’s important to explain why I think this is so important. In my experience, data visualization (and data science, more generally) is taught with one of the following specific focuses:

1. Focus on technology: How to use a particular tool.

2. Focus on design: How to design visual representations that communicate information effectively (and/or in an engaging or even surprising way).

3. Focus on perception: How notions of visual perception and cognition can help in designing more effective representations.

These are all great and important, but they miss a fundamental skill that, I would argue, is even more important than all the rest. This skill is thinking with data and data visualization. What do I mean by that? I will provide a more systematic explanation below, but on a first approximation, I mean being able to derive reliable information from data using visual representations of data. This type of skill is necessary for everyone. In particular, it’s necessary for both people whose job is to derive and communicate information from data, and those who consume this information for personal or professional use.

The main point I want to make here is that proficiency in using tools or designing visualizations is not enough if one is not skilled in assessing the validity of the facts and conclusions derived from data. My primary goal is to assist individuals like you who want to become better data thinkers.

Defining effective data thinking

There are many possible ways to define effective thinking with data visualizations. Here, I focus on a specific model I have developed over the years, teaching data visualization courses to college students and data professionals. To define effective thinking, I first need to show you one of my favorite diagrams.

Though very simple, this diagram shows a crucial fact about the role data and data visualization play in shaping our understanding of the world. What you see in the diagram is a simple (and highly simplified) set of steps that take place with any data visualization, either implicitly or explicitly. Everything begins with real-world phenomena, which are depicted by the Earth icon on the left. Data is nothing more than a recording of properties of real-world phenomena across many instances and often across time spans or spatial locations. Data is not reality. It’s a representation of reality and often a very coarse and limited one. But it’s also very useful because it enables us to reason in a way that is impossible to achieve with just our senses. We can’t observe a million points for many years around the globe. We use data to do that! Effectively, data is a tool to create a sort of human superpower. In the same way we can observe distant objects with a telescope, we can think about real-world events through data. What we can’t perceive directly with our senses, we understand through data and its visual representations. And this is where visualization plays a crucial role. Data itself is still very hard to comprehend with our senses (you can’t draw much from a data table with fifty columns and a thousand rows - let alone with even more), so what do we do? We create abstract representations to enable us to grasp the totality of what the data has to offer. If you think about it, it’s quite fascinating: since we can’t observe these phenomena with our eyes directly, we use a whole procedure to still see them with our eyes, but in an indirect, and admittedly convoluted, way.

So, what does this have to do with effective data thinking? It has everything to do with it, and this is why. When we analyze data and communicate our findings, we are not interested in the data itself; we are interested in the reality the data represents. In turn, thinking with data visualization is effective if and only if it helps us understand the reality represented by the data better. That’s my definition of effective thinking with data visualization.

Of course, this is much easier said than done. In most cases, we don’t have reliable tools to verify whether our understanding of reality has improved. However, identifying ways in which our thinking is flawed is much easier to do. We are aware of many ways we can deceive ourselves and important ideas to learn in order to overcome these issues. Because of this, the primary goal of this series (and my course as well as the book I eventually plan to write) is to share with you a series of mental models and frameworks that will help you identify gaps between your interpretation of the data and the reality it represents.

What the series covers

At the time of writing, I plan to follow the main structure of my existing course, which is organized around four main components:

1. Data: Often, the biggest problems with interpretation originate from a limited understanding of what the data represent and what its limitations are. Here, we will cover a systematic approach to identify specific types of problems that can lead to misinterpretation.

2. Transformations and calculations: Virtually all data sets require some data transformations before being visualized. At a minimum, they require the selection of specific variables and often some aggregations. However, all these operations can lead to a series of issues that impact interpretation. Here we will cover the most common transformations and the fallacies they can generate.

3. Visual representations: There is always a myriad of possible representations for a given piece of information. What are the possible alternatives, and how do they impact interpretation? Here, we go beyond the notion of “effectiveness” and focus on how design choices influence what messages readers extract from a visualization.

4. Contextual elements and framing: Charts always exist within a specific context, such as a presentation, a document, or a paper. What are the most common contextual elements, and how do they influence the way people read the charts? Here, we cover titles, annotations, and the general notion of “framing” as elements that influence what readers extract from a visualization.

For each of these elements, I have specific ways to help you consider how a data visualization communicates and how the information it conveys or the way you interpret it can lead to gaps with the reality described by the data. Stay tuned for the next posts. I plan to write one or more for each of the topics outlined above.

If you are interested in learning these skills in depth with my help, consider enrolling in my upcoming course “Thinking Effectively with Data Visualization.” The next cohort starts on October 27, 2025. The course has recorded video lectures and live meetings twice a week for three weeks. It’s a lot of hands-on activities and plenty of time to interact and discuss these ideas together. To get a feel for it, you can watch the first lecture here. You can also take a look at the syllabus by following this link.

If you are new to my newsletter, make sure to sign up for it! All posts are free, and you will receive updates on the series and many other topics directly in your inbox.

Subscribe now

If you like the ideas in this series, please share them in your preferred social media channels and let your colleagues and friends know! Thanks! 🙏

Share

This time, I tasked myself with analyzing the Boston Vision Zero Crash data set. This dataset collects information about vehicle crashes occurring in the Boston area. The dataset is quite simple. It contains information about when and where the crash occurred, whether it involved a motor vehicle, bicycle, or pedestrian, and the type of location (intersection, street, or other).

My goal is not to show you everything I did, but rather to share a few strategies I have developed while analyzing data with ChatGPT.

Strategies

Looking back at the many prompts I wrote for the analysis, I have identified four important strategies.

1. Be specific

There are a million ways one can ask ChatGPT to do something for them. I always obtain the best results when I am specific and thorough. For example, I don’t ask ChatGPT to “show me the trends over time” (a vague and underspecified request), but rather, I ask, “Generate a line chart showing weekly count of crashes.” Most of the time, I try to include information about (a) what type of chart I want, (b) what type of data transformation is needed, and (c) how I want the variables to be mapped. This is a good example: “Create a line chart showing the number of crashes by the hour of the day.” Here, I specify the plot I want to use (“line chart”), the metrics it needs to compute (“number of crashes”), and the granularity I want to use (“by hour of the day”).

2. Use coding tools (notebooks)

It would be fantastic if everything could be done within ChatGPT, but the reality is that some tasks require coding. The good news is that one does not need to be particularly proficient in coding, as ChatGPT and all other LLMs excel at producing code. One just needs to learn how to launch a notebook, such as Jupyter or Google Colab, and trust that they will be able to copy and paste code to try things out. When using code while not being proficient, three things work really well: (a) ask ChatGPT to produce code for one step at a time, (b) ask ChatGPT to explain what different parts of the code do, and (c) ask ChatGPT how to perform certain operations through code. I found that even if I don’t know how to do something, the notebook quickly becomes a learning-by-doing environment where ChatGPT tells me how to do something, and I try it out in the notebook (which is also a great strategy to deal with cognitive debt).

3. Ask for interpretations

LLMs bring a completely new element to the data analyst toolkit: a machine that can provide interpretations of the generated output. When a new chart is generated, you can ask ChatGPT to explain what the graph shows and why a certain relationship exists. Now, the results can be wildly inaccurate, but this is not the point. By providing you with some kind of reasoning, it can help you think about potential causes for effects that you observe. Here is a good example of this idea. I created a box plot showing the relationship between precipitation and the number of crashes. If you are not careful, you may expect that the number of crashes increases with the precipitation amount, but this is what the chart looks like:

How do you explain that? Well … you can ask ChatGPT and see if it has a sensible explanation. Here is what it returned:

It’s pretty impressive if you think about it. It provides a good account of the fact that even if more precipitation produces riskier conditions, the number of cars decreases, thus reducing the chances of collisions. (As a side note, this is a fantastic example of “ base rate bias,” a common interpretation problem I cover in my course on how to “Think Effectively with Data Visualization.”)

4. Ask for guidance

At some point during my analysis, I wanted to produce a heat map showing how collisions are distributed spatially. I knew that I had to ask it to produce a distribution using Kernel Density Estimation (KDE) (a statistical method to generate a continuous distribution from discrete data points), but I could not recall what parameters I had to set to produce the image I wanted. The first result did not have enough details (the one on the right), and I knew the problem was in the KDE settings.

As a next step, I asked it to explain to me how KDE works and what parameter I had to change to obtain the results I wanted, and this is exactly what it did. The image on the right is what I obtained after learning how to use KDE properly (FYI: the parameter to set was the “bandwidth”).

5. Ask, “What else?”

In one of my previous posts, I already demonstrated that ChatGPT can help you figure out what questions you can ask a dataset.

But a more interesting use of this idea is when you have already explored a few trends and you feel like you are hitting a wall. In this case, you can ask, “What else should I explore?” I tried this a few times and it works surprisingly well. Again, do not expect ChatGPT to be perfect and do the work for you. What ChatGPT does very well is to give you new paths, new ideas, so that you can continue with your own creativity.

Conclusion

There is a lot more I’d like to share, even from this simple interaction. Here, I focused only on a few useful strategies but I have more to share. In a future post, I’d like to highlight what does not work well and ideas on how LLM-based interfaces for data analysis could be improved. Stay tuned!

If you have any experience performing data analysis with ChatGPT, let me know how things work for you. Leave a commnt below. I’d love to hear your story.

Leave a comment

Thanks!

Prompt-to-Vis Systems

LLMs can be used to ask an AI system to generate charts that solve a particular problem, enabling data visualization specification using natural language instead of code, domain languages, or UI interactions.

• LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models

• ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Language

• Visualization Generation with Large Language Models: An Evaluation

• DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing

Image Synthesis for Vis

Most data visualization solutions based on LLM transform prompts into code that generates the desired charts. However, generative AI can also generate images directly, rather than relying on code. Here are a few papers that do exactly that.

• Let the Chart Spark: Embedding Semantic Context into Chart with Text-to-Image Generative Model

• viz2viz: Prompt-driven stylized visualization generation using a diffusion model

• Doom or deliciousness: challenges and opportunities for visualization in the age of generative models

Narrative Sequences

Building a sequence of charts and text is the core of data storytelling. LLMs can provide support in the ideation and implementation of these sequences, and the text introduces and describes each chart.

• Narrative Player: Reviving Data Narratives with Visuals

• DataTales: Investigating the use of Large Language Models for Authoring Data-Driven Articles

• DATAWEAVER: Authoring Data-Driven Narratives through the Integrated Composition of Visualization and Text

Captioning and Accessibility

Describing charts in terms of their structure and content is crucial for interpretation and accessibility. Can LLM provide the necessary support to support these important tasks? Can visually impaired users leverage GenAI to get easier access to visual content?

• Pluto: Authoring Semantically Aligned Text and Charts for Data-Driven Communication

• VizAbility: Enhancing Chart Accessibility with LLM-based Conversational Interaction

• MAIDR Meets AI: Exploring Multimodal LLM-Based Data Visualization Interpretation by and with Blind and Low-Vision Users

LLMs as Chart Readers

Can LLMs do some of the evaluative work that humans normally do with data visualizations? These papers examine the capabilities of LLMs and evaluate their performance in a series of interpretation and reasoning tasks.

• Probing the visualization literacy of vision Language Models: The good, the bad, and the ugly.

• How good (or bad) are LLMs at detecting misleading visualizations?

• How aligned are human chart takeaways and LLM predictions? A case study on bar charts with varying layouts.

User-Driven Output Verification

LLMs often make mistakes in data-related tasks, such as data transformation and visual mapping. Therefore, user interfaces are essential to help data analysts verify the generated output. These papers offer infrastructural and UI support for users to verify LLM output.

• Urania: Visualizing Data Analysis pipelines for natural language-based data exploration.

• WaitGPT: Monitoring and steering conversational LLM agent in data analysis with on-the-fly code visualization.

Evaluation and Benchmarks

Evaluating LLMs’ charting capabilities at scale is crucial for enabling the evaluation and validation of novel systems and techniques for AI-driven data visualization. These papers provide evaluation methods and benchmark data set creation to assess the performance of LLM-based data visualization systems.

• Automated Data Visualization from Natural Language via Large Language Models: An Exploratory Study

• VisEval: A Benchmark for Data Visualization in the Era of Large Language Models

• Natural Language Dataset Generation Framework for Visualizations Powered by Large Language Models

Understanding Real-World Use

To assess the use of LLMs for data visualization, it is essential to understand their practical application. These studies examine how people utilize LLMs for data-related tasks, including the challenges they face, the opportunities they present, and the strategies employed.

• Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling

• Data has Entered the Chat: How Data Workers Conduct Exploratory Visual Analytic Conversations with GenAI Agents

If you have recommendations for topics or specific papers to add, let me know!

Enjoy this list and look out for more posts on this topic. I will have more to report as my university course unfolds.

This is just a short message to let you know I am organizing a new free webinar for September 9th titled “How NOT to Lie with Charts.”

When you create your data presentations, how do you know that your charts and the conclusions you derive from them are true? What if you made a mistake? What if someone calls you out? What if you realize you've unintentionally misled your readers?

It happens to everyone. It’s hard. But it can be mitigated.

Creating truthful data stories is really hard, and we are all prone to mistakes. The more you try, the more you realize how hard it is. Reasoning with or from data is really hard!

In this webinar, I will walk you through the major elements needed to learn how to debug your data reasoning and reduce the chances of making mistakes.

Sign up soon!

👉 Webinar: How NOT to Lie with Charts

I am very excited to have a new webinar out, and I am looking forward to seeing you there!

Best,
Enrico.

One problem I see with all these approaches is that they seem to focus exclusively on teaching consumers of data visualizations how to spot problems. Still, they rarely focus on producers so that they can avoid some of these problems in the first place. It’s important to recognize that 1) developing truthful data visualizations is really hard, and 2) many issues with misleading data visualization stem from a limited ability to reason effectively with data.

As a first step toward teaching data professionals how to create more truthful data visualizations, I developed an initial set of 10 rules for creating truthful data visualizations. This is a work in progress, and I am not yet offering many details. My goal here is to gather your feedback and see if this resonates with you. Let me know what you think!

1. Develop an attitude

This is absolutely crucial. If you don’t start with having an explicit intent to be truthful and skeptical, there is no way you can avoid being fooled by data sooner or later. Skepticism is particularly important, especially when directed towards yourself. Being fooled by data is remarkably easy, and believing something simply because the data indicates it is also very easy. After many years of conducting data analysis and research, my experience with data has taught me to be extremely skeptical. Most large effects are either obvious or due to data errors, so my prior is always that something else explains the trend I am observing.

2. Understand the domain

It’s so tempting to jump on the data and start drawing conclusions. But the reality (and probably the most important secret of data analysis) is that drawing knowledge from data requires domain expertise. There is simply no way a person without domain knowledge can develop the same interpretive and evaluative skills as someone who understands the reality described by the data. Remember: we work with data because we are interested in the reality it (often partially and coarsely) represents. Yeah … dabbling with numbers and code is fun, but what matters is real-world facts and actions that stem from them, not numbers.

3. Understand the data

This is so often overlooked. Maybe it’s the most troubling problem I see. People tend to trust the data they find or receive from others. I have an exercise in class where students must select data from Our World in Data, a reputable source, and conduct research to understand where the data originates, how it was collected, and potential limitations associated with it. It’s always surprising to see how, even from such a reputable source, it’s hard to verify how the data was created, and it’s not rare to find several issues. This is important because there is simply nothing you can do at the data processing and visualization level to correct for problems that exist in the data. This is a classic case of garbage in, garbage out. If your data is not reliable or, more importantly, you have a wrong mental model of what it represents, you are at high risk of misrepresenting reality, no matter how careful you are with your visual representations.

4. Use appropriate numbers and calculations

You might ask, “What do you mean by appropriate?” When you work with data, the output of the process is to produce (implicitly or explicitly) a series of “data facts,” statements that represent your interpretation of patterns and trends you extracted from the data. This is often reflected in the titles and accompanying text you write, as well as the spoken words you use when presenting the results. Your interpretation, however, can be incorrect, and this is where insidious gaps can reside. So, appropriate numbers are those that accurately reflect your interpretations. If you think a number represents a given concept when in fact it does not, you are in trouble.

Imagine you want to analyze the NYC vehicle collision data set to determine the “most dangerous areas to drive in NYC.” You can use the number of collisions or the average number of people injured in each zip code area. However, if you use the number of collisions, you are in trouble, because those numbers depend on how many cars circulate in a given area, so it does not represent the level of “danger,” but rather how trafficked a given area is. Virtually all numbers have a gap between the concept you want to work with and the reality they represent, and being aware of these gaps is essential to producing truthful visualizations.

This problem becomes even more severe when the numbers we use are the product of complex calculations. So, while complex calculations and statistics are essential for deriving signals from noisy data, it is also crucial to be aware of the numerous limitations these numbers have.

5. Aggregate mindfully

In visualization, we often have the choice to represent the data as statistical aggregates or individual data points. We can display each vehicle collision on a map or simply count the number of collisions in each zip code area. Aggregation is useful because it reduces clutter and often makes it easier to observe trends and make comparisons. But it also necessarily hides information. There is a constant tension between these two needs: transparency vs. clarity. Higher granularity leads to greater transparency, but often results in less clarity. Lower granularity leads to less transparency but more clarity. There is no fixed rule on how to strike a balance between these two needs, but my experience suggests that people often err on the side of either too much or too casual aggregation. Exploring data at the lowest level of granularity possible is essential, at the very least, as a sanity check. Your highly granular charts may not end up in your slides, but having explored them gives you the peace of mind that you have checked what’s behind those aggregations.

6. Disclose uncertainty (within reason)

There are multiple sources of uncertainty when dealing with data. Uncertainty in how the values have been recorded, uncertainty in the summary statistics that you use, uncertainty in the interpretations and explanations you provide (to yourself and others). Being aware of these sources of uncertainty is the first step. You have to be conscious of the fact that data contains errors and inaccuracies, statistics are not exact numbers, and alternative explanations may exist for the same set of phenomena you observe in the data. Once you become aware of all these sources of uncertainty, you must decide how to disclose them to your readers. Jessica Hullman has a very interesting paper titled “Why Authors Don’t Visualize Uncertainty” on this very problem. She interviewed several professionals and asked them how they show uncertainty in their visualizations, and the results are very revealing. Many feel like exposing uncertainty dilutes the message. How do you want to deal with this problem? How do YOU want to deal with this problem? This is harder than it may seem because communicating all sources of uncertainty can become a daunting task and completely overwhelming for your audience. So, this is another area where a sensible trade-off is needed.

7. Segment your data

When we observe a trend on a visualization, we are often tempted to take note of it and move on to something else. However, when we do that, we miss the opportunity to use one of the most powerful moves in data analysis, which is data segmentation. Segmentation means that when you see a pattern, you want to see if the same pattern holds in a specific subset of your data. When I create a line chart showing the number of collisions by hour of the day, I also want to explore what this trend looks like in different geographical areas, seasons, vehicle classes, and other factors. These are all examples of “segmentations” that can reveal very relevant information. I have a personal habit of always segmenting the patterns I expose by potentially meaningful variables. Many trends change when you look at specific segments of the data, and exploring them mindfully is part of the procedures necessary to avoid fooling yourself and others.

8. Ask yourself: “Compared to what?”

Legendary economist Thomas Sowell wrote in Economic Facts and Fallacies, “Virtually nothing is going to be equally beneficial for all people, or equally detrimental. The real question is always: ‘Compared to what?’” In data visualization, we can use a similar principle. Whenever we see a trend and derive a conclusion from it, it’s useful to ask ourselves, “Compared to what?” This is because data is virtually always partial, and we are tempted to focus exclusively on what is under our eyes and not on what is missing. If we compare a set of objects, a large difference may appear very relevant within that set, but meaningless when viewed in the context of a much larger set of objects. Similarly, if we analyze a temporal trend for a specific time period, a significant shift may appear dramatic within that time horizon, but insignificant on a much longer timespan. Being mindful that all comparisons are potentially limited by the objects we have versus those that few could have is a good habit and often helps put data in a much broader context.

9. Scale visuals mindfully

Visualization is all about mapping abstract numbers into perceivable graphical properties of objects, that is, size, length, color, etc. The rules we use (or our tools use as defaults) to map values to visual properties have a strong impact on what we perceive in a chart. Because of that, it is crucial to be mindful of the different ways data can be scaled when mapped to visual properties. Many of the most well-known “misleading visualization” examples represent scaling problems at their core. Truncated axis? A scaling problem. Inappropriate area mapping? A scaling problem. Misleading double-axis charts? Another scaling problem. Often, choosing an appropriate scale is not as simple as avoiding blatant distortions. Some patterns may be easier to discern when data are scaled in a certain way than when scaled in another way. For example, cases exist where scaling data nonlinearly or truncating an axis can reveal important trends that would otherwise be difficult to discern. This is another example of the idea that there are no hard rules but just trade-offs that depend on the specific problem.

10. Use truthful titles

Titles are often the part of a visualization that is more prone to overclaiming. The need to be succinct, paired with the desire to capture people’s attention, often leads to titles that go beyond what can be inferred from the data. Titles, however, are incredibly important. Research demonstrates that people spend a significant amount of their attention on titles, and often, they are the only thing people remember from a chart. At the same time, titles are also where many others commit the sin of oversimplifying. One possible solution is to use only titles that describe what the chart is about, but this often results in very boring titles. Titles can also be more interpretive, but it’s important to ask yourself, “Does the data actually show that?” Another option is to use questions as titles. Questions draw people in and stimulate curiosity, and I think they should be used more often.

That’s all for now, folks. Let me know what you think. Do these guidelines resonate with you? Is there anything you’d like to add? Let me know in the comments below.

Leave a comment

💡 Hey … I have a course to teach you these skills!

If you're interested in learning these skills with me, consider signing up for the upcoming cohort of my Rhetorical Data Visualization course. You’ll meet with me and other students for a total of six live online workshops. The course includes:

1. Recorded video lectures to watch at home

2. Quizzes to test your knowledge

3. Six live meetings with hands-on activities

4. A final project

👉 Book a call with me. I’d be happy to provide you with a comprehensive preview of the course and address any questions you may have. I’d love to learn more about you.

P.S. The dates are tentative, so if they don't work for you, please let me know and I’ll try to make accommodations.

Summer is finally upon us (well, here in Massachusetts, it’s been a complete rollercoaster! 🎢) and everything seems to slow down. But May and June have been another combo of intense activity for the newsletter. I published a few solid posts (one went viral!), recorded a podcast, and promoted my webinar and upcoming course. Let’s review each in order.

Webinars and courses

A new cohort of my Rhetorical Data Visualization course is starting in mid-July. You can still sign up for it if you’re interested! You’ll learn how to detect and avoid misleading visualizations and design more truthful solutions. See details in my post.

I published two live webinars, and I'm quite happy with them. The first is about ways in which visualization can mislead, beyond the classic misleading visualizations with which we are all familiar. The second is about a narrower but important topic: how we perceive causality from certain types of charts and why we can get fooled by them.

Watch the recordings!

Watch the recordings by following these two links:

• Can You Trust It? 4 Ways Data Visualizations Mislead

• Don't Get Fooled! Avoid The Illusion of Causality in Charts

If you have ideas for future webinars that you would like me to produce, please share them by sending me a message.

Posts and podcasts

My post on the “Illusion of Causality in Charts” went viral. It’s one of the most successful posts I've ever published. If you have not seen it, I encourage you to take a look.

I also wrote about qualitative visualization. In May, I gave a private workshop, and I was asked to include information about qualitative visualization, which made me realize how foggy this specific area of data visualization is. The post includes initial ideas to define what qualitative data visualization is.

A few days ago, I posted something on “truthfulness” in visualization. Inspired by the webinars and my courses, I began exploring the concept of what it means for a chart to be considered “true.” You can expect more posts in the future exploring this important question.

Early in May, I interviewed Richard Brath to talk about the use of AI in visualization. Richard gives amazing demos of his experiments with AI. If you have not watched it yet, I strongly encourage you to take a look! On a side note, I plan to be more consistent with these video podcasts. There are a few more people I’d like to interview in the area of AI and visualization.

As a side note, you can also listen to the interviews on Spotify if you want to be mobile! You can find this interview here:

Future plans

• InfoVis course online. I devoted the first six months of 2025, in large part, to producing educational material for my readers. There is a lot more brewing! Early this year, I began re-recording the Information Visualization course, but I was unable to finalize it. Luckily, there is very little additional work to do, so I will use the summer to finalize it and make it available.

• Video podcasts. I’d like to make the podcasts more frequent. My idea is to create thematic podcasts to cover a whole topic in depth. I started with AI, and I’d like to finalize it before moving on to something else.

• Post series. I have lost traction on my series. I should definitely get back to producing series and then use them to build more permanent material. I was not able to finalize the one on Vis for ML because there is way too much new material to review. The good news is that in the fall, I will teach a seminar course at Northeastern University on Visualization and GenAI, and I am sure this will enable me to write more posts for the series.

And you?

I hope you are having a great summer! There are many news readers here. If you'd like to introduce yourself and share your interests, I’d be delighted to learn more about you. Feel free to send me a message with specific comments or requests.

Thanks for reading! 🙏
Enrico.