Chapter 15: Grammar of graphics with lets-plot

You want to colour the scatter points by course. In matplotlib that means adding c=results['course'] to ax.scatter(), but scatter expects a numeric array, not strings, so you need a colour map. The legend does not appear automatically, so you add ax.legend(). Now you manage the colour assignment yourself. Adding one data dimension turned a three-line chart into fifteen lines of housekeeping.

The grammar of graphics solves this differently. You describe what the data means and let the library handle the rendering. aes(color='course') says ‘colour represents course’: the library picks the palette, maps it, and adds the legend automatically. The trend line is just + geom_smooth(): one more layer, no manual computation.

The tool is lets-plot, a Python port of R’s ggplot2. By the end you will build any chart by composing three parts: data, aesthetics, and geometry. Chapter 16 (16-data-storytelling.ipynb) then applies a house style to both libraries.

Callout markers: book cover page.

Why the grammar of graphics?

There is another way to think about it. A chart isn’t a list of drawing commands: it’s a mapping from data columns to visual channels (position, colour, shape, size). State that mapping once, and the library figures out how to draw it. Add a layer (a trend line, a rug, a label) and you extend the mapping rather than calling a new drawing function. This is the Grammar of Graphics, introduced by Leland Wilkinson in 1999 and popularised by Hadley Wickham’s ggplot2 for R.

Lets-Plot (lets-plot.org) is JetBrains’ Python implementation of the same grammar. Its API mirrors ggplot2 so closely that R users can read Lets-Plot code without a translation guide. It renders to HTML in Jupyter, to PNG for reports, and (in its Pro edition) to interactive Datalore dashboards.

Alternatives that use the same grammar

Library	Language	Notes
ggplot2 (ggplot2.tidyverse.org)	R	The original; Lets-Plot mirrors it deliberately
plotnine (plotnine.org)	Python	ggplot2 port; similar API, Matplotlib backend
Lets-Plot (lets-plot.org)	Python / Kotlin	HTML-first output, fast, maintained by JetBrains
Vega-Altair (altair-viz.github.io)	Python	Different grammar (Vega-Lite), interactive

Already in your environment

uv add lets-plot          # for a standalone project

Learning objectives

By the end of Chapter 15 you will be able to:

#	Skill	Covered in
1	Explain the difference between declarative and imperative plotting	Sec. 1
2	Build a chart from `ggplot()`, `aes()`, and a `geom_*()`	Sec. 2
3	Distinguish mapping a variable from setting a fixed value	Sec. 3
4	Layer a statistical summary on top of raw data	Sec. 4
5	Facet one chart into many panels instead of looping over subplots	Sec. 5
6	Add titles, axis labels, and control colours with `labs()` and scale functions	Sec. 6
7	Name stat, position, and coord as grammar components, not one-off options	Sec. 7

1. Declarative vs. imperative

In Chapter 14, building a scatter plot meant calling ax.scatter() directly: you were the one deciding which function draws which shape. The grammar of graphics flips this around. You describe what the data means (this column is the x position, this column is the colour) and the library decides how to draw it. The same description works whether you add one point or one million, and stays valid even if you later change the chart type entirely.

from lets_plot import LetsPlot

LetsPlot.setup_html()

Rebuild the same dataset from Chapter 14: final scores across three courses and three Fall cohorts.

import pandas as pd

df = pd.read_csv("data/university_analytics.csv")

# Three courses share the same Fall track and run across three real
# cohorts: Fall 2022, Fall 2023, and Fall 2024.
courses = ["Machine Learning", "Data Structures", "Python Programming"]
semesters = ["Fall 2022", "Fall 2023", "Fall 2024"]

results = df[df["course"].isin(courses) & df["semester"].isin(semesters)].copy()
results.head()

	student_id	cohort	program	gender	region	guardian	has_internet	course_id	course	semester	enrollment_date	study_hours	attendance_pct	midterm_score	final_score	project_score	final_grade	passed
0	S0001	2023	Information Technology	F	South	Father	True	C01	Python Programming	Fall 2023	2023-09-04	10.8	88.6	49.9	58.2	61.6	C	True
2	S0001	2023	Information Technology	F	South	Father	True	C03	Data Structures	Fall 2023	2023-09-04	23.6	71.5	54.1	60.9	79.8	C	True
4	S0001	2023	Information Technology	F	South	Father	True	C05	Machine Learning	Fall 2023	2023-09-04	16.2	63.2	48.4	54.2	51.8	D	True
6	S0002	2023	Data Science	M	West	Mother	True	C01	Python Programming	Fall 2023	2023-09-04	17.4	67.0	93.6	60.5	67.0	B	True
8	S0002	2023	Data Science	M	West	Mother	True	C03	Data Structures	Fall 2023	2023-09-04	30.4	84.9	67.4	93.5	70.6	B	True

Here is the Chapter 14 scatter plot (study hours vs. final score) again, this time declaratively. ggsize(width, height) sets a fixed pixel size instead of leaving it to the browser: every chart in this notebook uses the same 500, 320 so they sit consistently in the page:

from lets_plot import aes, geom_point, ggplot, ggsize

ggplot(results, aes(x="study_hours", y="final_score")) + geom_point(alpha=0.4) + ggsize(500, 320)

Key Concept: The Grammar of Graphics

Every lets-plot chart is built from the same three pieces, combined with +: ggplot(data, aes(…)) declares the dataset and which columns map to which visual property, and one or more geom_() layers say what shape to draw with that mapping. Change the geom_() and the same aes() mapping produces a completely different chart type, often without touching anything else.

Activity 1 - Rewrite a Scatter Plot Declaratively

Take Chapter 14’s matplotlib scatter and express it in three lets-plot layers.

Setup: reuse the results DataFrame from this notebook.

1. Write ggplot(results, aes(x=‘study_hours’, y=‘final_score’)).
2. Add + geom_point(alpha=0.5).
3. Add + geom_smooth(se=False) for a trend line.
4. Compare: how many lines did the matplotlib version take for the same result?

Expected: scatter with trend line in 3 lines of grammar.

2. The grammar: `ggplot`, `aes`, `geom`

aes() (short for “aesthetics”) maps DataFrame columns to visual properties: x, y, color, fill, size. The geom_*() you add decides what shape represents each row. Swapping geom_point() for geom_line() or geom_bar() is often the only change needed to turn one chart type into another:

Key Concept: Three parts make every chart: ggplot, aes, geom

ggplot(data, aes(…)) owns the data and the default aesthetic mappings. aes() maps DataFrame columns to visual channels (x, y, color, size). geom_*() decides the shape (point, line, bar, histogram). Every chart you build in lets-plot is some combination of exactly these three.

from lets_plot import coord_flip, geom_bar, geom_lollipop, scale_y_continuous

course_means = results.groupby("course", as_index=False)["final_score"].mean()

ggplot(course_means, aes(x="course", y="final_score")) + geom_bar(stat="identity", fill="#4477AA") + ggsize(500, 320)

stat="identity" tells geom_bar() to plot the final_score column exactly as given, rather than its default behaviour of counting rows per category. Compare this to Chapter 14’s bar chart: the data preparation (groupby().mean()) is identical, only the drawing step changed shape.

position="dodge" places bars for each group side by side instead of stacking them, useful when you want to compare values across two grouping variables at once. The data needs a category column on x and a group column on fill:

course_semester_means = results.groupby(["course", "semester"], as_index=False)["final_score"].mean()

(
    ggplot(course_semester_means, aes(x="course", y="final_score", fill="semester"))
    + geom_bar(stat="identity", position="dodge")
    + ggsize(500, 320)
)

Activity 2 - Histogram, Declaratively

Rebuild the Chapter 14 histogram of final_score using geom_histogram(). Map x=“final_score” in aes(), and pass bins=20 to geom_histogram().

ggplot(results, aes(x="final_score")) + geom_histogram(bins=20, fill="#4477AA")

# TODO: build the histogram described above
...

Pro Tip: Use geom_lollipop() to reduce ink on ranked comparisons

For ranked comparisons with many categories, lollipop charts reduce ink without losing information. Replace geom_bar() with geom_lollipop(): it draws a dot at the value and a thin stem from the baseline, making individual values easier to read when bars would crowd together.

from lets_plot import labs

pass_rate_by_course = (
    results.groupby("course")["final_score"]
    .apply(lambda x: (x >= 60).mean())
    .reset_index()
    .rename(columns={"final_score": "pass_rate"})
    .sort_values("pass_rate")
)

# Preserve sort order as a categorical so lets-plot respects it on the axis
# pandas 3 uses StringDtype by default, which lets-plot cannot introspect;
# pd.Index(..., dtype=object) keeps categories as plain numpy object dtype
_courses = pass_rate_by_course["course"].tolist()
pass_rate_by_course = pass_rate_by_course.assign(
    course=pd.Categorical(
        _courses,
        categories=pd.Index(_courses, dtype=object),
        ordered=True,
    )
)

(
    ggplot(pass_rate_by_course, aes(x="course", y="pass_rate"))
    + geom_lollipop(size=4, color="#0369A1")
    + coord_flip()
    + scale_y_continuous(format=".0%")
    + labs(title="Pass Rate by Course", x="", y="Pass Rate")
    + ggsize(500, 320)
)

Activity 3 - Bar Chart from a Lollipop Chart

Confirm what geom_lollipop() bought you by comparing it against the bar chart it replaced.

1. Take the pass-rate chart above.
2. Replace geom_lollipop(size=4, color=“#0369A1”) with geom_bar(stat=“identity”, fill=“#0369A1”).
3. Compare: with three categories on the axis, does the bar or the lollipop version make the ranking easier to read?

Expected: the same ranking, with less visual weight in the lollipop version.

# TODO: replace geom_lollipop with geom_bar(stat="identity") and compare
...

3. Mapping vs. setting

aes(color="course") maps the course column to colour: each course gets its own colour, chosen automatically, with a legend. color="#4477AA" sets every point to the exact same fixed colour, with no legend, because there is nothing left to distinguish. Confusing the two is the single most common mistake when learning any grammar of graphics, in Python or R:

Key Concept: Mapping encodes data; Setting hardcodes a constant

aes(color=‘course’) maps the course column to colour: the library assigns one colour per unique value and generates the legend automatically. geom_point(color=‘#0369A1’) sets the colour to a fixed value for all points. Never put a constant inside aes(): that is the most common lets-plot mistake.

# Mapping: course determines colour, one colour per course, with a legend
ggplot(results, aes(x="study_hours", y="final_score", color="course")) + geom_point(alpha=0.6) + ggsize(500, 320)

# Setting: every point is the same fixed colour, no legend needed
ggplot(results, aes(x="study_hours", y="final_score")) + geom_point(color="#4477AA", alpha=0.6) + ggsize(500, 320)

Common Mistake: Putting a fixed value inside aes()

aes(color=“#4477AA”) tries to map the literal string “#4477AA” as if it were a column name. Lets-plot will either error or, worse, silently treat it as a constant category and draw a one-entry legend that says #4477AA. A fixed value is a setting and belongs as a plain keyword argument to the geom_*(), outside aes(). A column name that should vary per row is a mapping and belongs inside aes().

Activity 4 - Mapping vs Setting in Practice

Feel the difference by trying both on the same chart.

1. Build a scatter of study_hours vs final_score where color is mapped to ‘semester’.
2. Now remove the mapping and instead set geom_point(color=‘#0369A1’).
3. Observe: the mapped version gets a legend; the set version does not.
4. Try placing ‘#0369A1’ inside aes(color=‘#0369A1’) and note what goes wrong.

Expected: mapping creates a legend; setting applies one fixed colour.

4. Layering: raw data and a statistical summary together

Because every geom_*() is its own layer, you can stack a raw-data layer and a computed-summary layer on the same aes() mapping. geom_smooth() fits a trend line to the data it’s given, with a shaded confidence band, entirely declaratively:

from lets_plot import geom_smooth

(
    ggplot(results, aes(x="study_hours", y="final_score"))
    + geom_point(alpha=0.3, color="#4477AA")
    + geom_smooth(color="#EE6677", se=True)
    + ggsize(500, 320)
)

geom_smooth() uses a LOESS (locally weighted) smoother by default, which follows local curves in the data. Pass method='lm' when you want a straight regression line instead; se=True shows the confidence band either way, though the LOESS smoother needs statsmodels installed and the linear one does not.

Lets-Plot charts rendered in a Jupyter notebook are interactive by default: hovering over a data point shows a tooltip with the underlying values. The layer_tooltips() API lets you control exactly which fields appear and how they are labelled.

from lets_plot import layer_tooltips

(
    ggplot(df.sample(300, random_state=42), aes(x="midterm_score", y="final_score", color="program"))
    + geom_point(
        tooltips=layer_tooltips()
        .line("Student @student_id")
        .line("Program: @program")
        .line("Midterm: @midterm_score")
        .line("Final: @final_score"),
        size=3,
        alpha=0.7,
    )
    + labs(title="Score Correlation by Program", x="Midterm", y="Final")
    + ggsize(500, 320)
)

layer_tooltips().line('@col') renders each .line() call as one row in the hover tooltip, with @col substituting the column value at that point. Omit layer_tooltips() entirely to fall back to the default, which shows every mapped column.

geom_density() is the smooth-curve equivalent of a histogram, useful when comparing several distributions that would otherwise overlap into an unreadable stack of bars:

from lets_plot import geom_density

ggplot(results, aes(x="final_score", fill="course")) + geom_density(alpha=0.4) + ggsize(500, 320)

geom_boxplot() summarises a distribution as median, quartiles, and outliers per group, the declarative equivalent of Chapter 14’s sns.boxplot(). Because it’s just another geom_*(), it composes with aes() and faceting exactly like any other layer:

from lets_plot import geom_boxplot

ggplot(results, aes(x="course", y="final_score", fill="course")) + geom_boxplot(alpha=0.7) + ggsize(500, 320)

A boxplot gives you five numbers: minimum, first quartile, median, third quartile, and maximum. Swap geom_boxplot() for geom_violin() when you need the full density shape instead: a violin reveals skew or multiple peaks that a box hides, and the aes() mapping stays identical.

Key Concept: Layers compose in the order you add them

geom_point() + geom_smooth() draws points first, then the trend line on top. Reversing the order draws the line first and the points on top of it. This matters most when a layer has a solid fill that would otherwise hide whatever was drawn before it.

Pro Tip: Add marginal distribution plots with ggmarginal()

Lets-Plot’s ggmarginal() wraps any scatter plot with histograms or density curves along the x and y margins, letting you see both the bivariate relationship and the individual distributions without a separate figure:

from lets_plot import ggmarginal

p = (
    ggplot(results, aes(x="study_hours", y="final_score", color="course"))
    + geom_point(alpha=0.4)
    + geom_smooth(method="lm", se=False)
)
ggmarginal(p, type="density")

type accepts “density” (KDE curves), “histogram”, or “boxplot”. The marginals automatically inherit the color mapping so each course gets its own density curve.

Activity 5 - Layer a Raw Scatter with a Density Contour

Stack two geoms to show individual points and their density together.

1. Start with ggplot(results, aes(x=‘study_hours’, y=‘final_score’)).
2. Add + geom_point(alpha=0.3, size=2).
3. Add + geom_density2d(color=‘#DC2626’) as a second layer.
4. Add a trend line: + geom_smooth(se=True, color=‘#0369A1’).
5. Add tooltips: + geom_point(tooltips=layer_tooltips().line(‘score: @final_score’)).

Expected: three layers compose: points, contour lines, and trend band.

5. Faceting: one plot, many panels

Chapter 14’s three-panel histogram needed a manual loop over axes.flat, plus sharey=True to keep the comparison fair. facet_wrap() does both in one line, and shares scales across panels by default, the opposite of matplotlib’s default and exactly what you want for a fair comparison:

from lets_plot import facet_wrap, geom_histogram, ggsize

(
    ggplot(results, aes(x="final_score"))
    + geom_histogram(bins=15, fill="#4477AA")
    + facet_wrap(facets="course")
    + ggsize(700, 250)
)

Key Concept: Faceting replaces a loop with a declaration

facet_wrap(facets=“course”) splits the data by the course column and draws one panel per group, using the exact same aes() and geom_*() for every panel. There is no loop to write and no risk of accidentally giving one panel a different scale than the others, the bug from Chapter 14’s Common Mistake callout. Add a second facet variable with facet_grid() for a full grid of panels.

When you have two grouping variables, facet_grid() builds a full grid of panels: one row per level of one variable and one column per level of the other. For this dataset, rows are semesters and columns are courses:

from lets_plot import facet_grid

(
    ggplot(results, aes(x="final_score"))
    + geom_histogram(bins=12, fill="#4477AA")
    + facet_grid(y="semester", x="course")
    + ggsize(850, 450)
)

Activity 6 - Facet by Semester

Build a scatter plot of study_hours vs. final_score, coloured by course, faceted into one panel per semester.

ggplot(results, aes(x="study_hours", y="final_score", color="course")) \
    + geom_point(alpha=0.5) \
    + facet_wrap(facets="semester")

# TODO: scatter plot, coloured by course, faceted by semester
...

6. Titles, labels, and scales

Every chart so far has used whatever axis labels and legend titles lets-plot generates by default: column names from the DataFrame, which are fine for exploration but not for sharing. labs() replaces all of them in one layer. scale_color_manual() and scale_fill_manual() give you exact control over which colour maps to which group, overriding the automatic palette with hex codes you choose yourself:

from lets_plot import labs, scale_color_manual

(
    ggplot(results, aes(x="study_hours", y="final_score", color="course"))
    + geom_point(alpha=0.5)
    + geom_smooth(se=False)
    + labs(
        title="Study hours versus final score",
        subtitle="Each point is one student; lines show the per-course trend",
        x="Weekly study hours",
        y="Final score (0-100)",
        color="Course",
    )
    + scale_color_manual(values=["#4477AA", "#EE6677", "#228833"])
    + ggsize(500, 320)
)

Key Concept: labs() renames anything auto-generated from a column name

labs(title=…, x=…, y=…, color=…) is one more + layer. The named argument matches the aesthetic: use color= when you have aes(color=“course”) and fill= when you have aes(fill=“course”). scale_color_manual(values=[…]) takes over from the default palette entirely, one hex code per group, in the order the groups appear. Chapter 16 applies one shared palette and theme to every chart in the book; here, you are choosing colours by hand, one chart at a time.

Activity 7 - Label the Density Chart

Take the geom_density() chart from Section 4 and add a proper title, axis labels, and legend title with labs().

ggplot(results, aes(x="final_score", fill="course")) + geom_density(alpha=0.4) \
    + labs(title=..., x=..., y=..., fill=...)

# TODO: add labs() to the density chart from Section 4
...

7. Stat, position, and coord: the grammar’s other pillars

Two calls already slipped past without an explanation. position="dodge" placed the semester bars side by side in Section 2. coord_flip() turned the pass-rate lollipop chart on its side. Both are grammar components in their own right, not one-off options. Wilkinson’s original grammar names seven pieces: data, aesthetic mapping, geometry, statistical transformation, position adjustment, coordinate system, and facet. You already have four. Here are the other three.

Stat: the calculation behind the shape

Every geom_*() has a default statistical transformation. geom_bar() defaults to counting rows per category (stat="count"); passing stat="identity" back in Section 2 switched that off so the bar could plot a pre-computed mean instead. stat_summary() makes the computation part of the grammar itself: it groups by whatever is mapped to x, applies a function, and hands the result to whichever geom you name, no groupby() required beforehand:

from lets_plot import stat_summary

(
    ggplot(results, aes(x="course", y="final_score"))
    + stat_summary(fun="mean", geom="bar", fill="#4477AA")
    + ggsize(500, 320)
)

Position: what happens when marks overlap

position decides what a geom does when two marks would otherwise land on top of each other. position="dodge" (Section 2) shifts them sideways into side-by-side bars. position="jitter" adds a small amount of random noise, useful for a scatter where one categorical value stacks many students at the exact same x position:

ggplot(results, aes(x="course", y="final_score")) + geom_point(position="jitter", alpha=0.3) + ggsize(500, 320)

Coord: the space the geometry is drawn into

coord_flip() (Section 2’s lollipop chart) swapped which axis is which, without touching the data or the mapping. coord_cartesian() does the same kind of job as zooming: it changes the visible window over the data, not the data itself, so a stat layer computed from the full dataset, like the trend line below, is unaffected. That is different from filtering rows before plotting, which would recompute the trend from a smaller sample:

from lets_plot import coord_cartesian

(
    ggplot(results, aes(x="study_hours", y="final_score"))
    + geom_point(alpha=0.3)
    + geom_smooth(se=False)
    + coord_cartesian(xlim=(5, 15))
    + ggsize(500, 320)
)

Key Concept: Seven pillars, one grammar

Every lets-plot chart is data + aes + geom, optionally refined by stat (how raw rows become the numbers drawn), position (what happens when marks overlap), coord (the space they are drawn into), and facet (how many panels). Swap any one piece and the rest of the chart definition does not change: that is the whole idea of the grammar of graphics.

Activity 8 - Rebuild the Course Bar Chart with stat_summary

Recreate Section 2’s course-average bar chart, this time letting the grammar do the aggregation.

1. Start from ggplot(results, aes(x=‘course’, y=‘final_score’)).
2. Add + stat_summary(fun=‘median’, geom=‘bar’, fill=‘#4477AA’) to plot the median instead of the mean.
3. Add + coord_flip() so the course names are easier to read.
4. Compare: how many lines of setup did this take versus Section 2’s groupby().mean() version?

Expected: a horizontal bar chart of median final score per course, built with zero manual aggregation.

# TODO: stat_summary bar chart of median final_score per course, flipped
...

Capstone: the three-panel report, declaratively

Chapter 14’s capstone built a three-panel course report with a manual (1, 3) grid of Axes. Here, build the same idea (one histogram per course) as a single faceted chart instead of three separate panels assembled by hand.

Capstone Exercise - Faceted Course Report

Build one chart: a histogram of final_score, faceted by course, with geom_vline() marking the overall mean score so each course’s panel can be compared against it.

overall_mean = results["final_score"].mean()

ggplot(results, aes(x="final_score")) \
    + geom_histogram(bins=15, fill="#4477AA") \
    + geom_vline(xintercept=overall_mean, color="#EE6677", linetype="dashed") \
    + facet_wrap(facets="course") \
    + ggsize(700, 250)

Hint: geom_vline() takes a fixed xintercept, a setting, not a mapping, so it goes outside aes() just like the fixed colours in Sec. 3.

from lets_plot import geom_vline

overall_mean = results["final_score"].mean()

# TODO: faceted histogram with a reference line at overall_mean
...

Ellipsis

Resource	Why it matters
Wilkinson, L. (2005). The Grammar of Graphics, 2nd ed. Springer.	The theory behind the layered grammar; Lets-Plot, ggplot2, and Vega-Altair are all implementations of this framework
Wickham, H. (2010). A layered grammar of graphics. Journal of Computational and Graphical Statistics 19(1), 3–28.	The ggplot2 paper; most directly explains `geom_`, `aes()`, and `stat_` concepts that Lets-Plot mirrors
Lets-Plot documentation	The primary API reference; the gallery is the fastest way to find the right `geom_*`
ggplot2 book (Wickham, 2016)	Free online: Lets-Plot’s API maps closely to ggplot2, so this book is directly useful

Summary

Concept	Key rule
Declarative vs. imperative	Describe the mapping, let the library decide how to draw it
`ggplot(data, aes(...))`	Declares the dataset and which columns map to which visual property
`geom_*()`	Decides what shape represents each row; swap it to change chart type
Mapping	`aes(color="course")`, a column that varies per row, gets a legend
Setting	`color="#4477AA"`, a fixed value, no legend
`position="dodge"` / `"jitter"`	Places overlapping bars side by side, or scatters overlapping points apart
Layering	Stack `geom_*()` calls with `+`; later layers draw on top of earlier ones
`geom_smooth()` / `geom_density()`	Statistical summaries, computed declaratively from raw data
`geom_boxplot()` / `geom_violin()`	Distribution per group: box for five-number summary, violin for full shape
`stat_summary()`	Groups and aggregates inside the grammar itself, no `groupby()` needed
`coord_flip()` / `coord_cartesian()`	Reorients or zooms the drawing space without changing the data
`facet_wrap()`	One declaration replaces a manual subplot loop, with shared scales by default
`facet_grid(rows=..., cols=...)`	Full grid of panels for two grouping variables
`labs()`	Replaces any auto-generated axis label or legend title in one layer
`scale_color_manual()`	Maps groups to specific colours, overriding the default palette
`ggsize(width, height)`	Fixes pixel size so every chart renders consistently instead of at the browser’s default

Next: 16-data-storytelling.ipynb, covering what makes a chart good and applying this project’s house style to both matplotlib and lets-plot.