Learning used to be a separate project

Think about how you picked up a genuinely new skill in the old world, which is to say eighteen months ago. Say you wanted to learn Shiny, the framework for building reactive web apps in R. You read “Mastering Shiny”, or opened the docs, and worked through them. You started a toy project on the side. You hit a wall, searched, landed on Stack Overflow, and when nothing fit you distilled the problem into a reproducible example and asked your own question. Eventually you got unstuck and went back to the toy project.

That loop is iterative, and it works. But notice two things about it. It is linear: you pick one skill and march along it, and the march is slow. And it is separate from the work: the learning is its own project, with its own clock, sitting beside whatever you were actually trying to ship. First the course, then the doing.

Now the workbench is the classroom

Working with a coding agent collapses that separation. The platform you do the work in is the platform you learn in, and the learning happens on top of the real task rather than beside it.

Here is the shape of it. You are in the middle of something real – cleaning a messy dataset, editing a podcast, wiring up a report. You notice you keep handing the agent the same instructions. You have read, somewhere, that skills exist. So you say: “turn what we just did into a skill.” That is the whole move. You are not sitting down to study skill authoring; you are delegating it, in context, while the example is still warm. The agent writes the skill, you read it, you change a line, and now you have one. You learned what a skill is by making one, inside work you were doing anyway.

This is why the cycles are so short. The old loop measured progress in weeks – a chapter here, a worked example there. The new one measures it in minutes, because every iteration hands you something you can use immediately and feedback you can act on right away. You are not learning in order to work. You are learning by working.

Why “on my list this year” is anachronistic

Back to Eric’s aside. The phrase “I still haven’t authored my own skill, but it’s on my list this year” belongs to the old model. It treats a skill as a body of knowledge you must acquire before you are allowed to use it, the way you might resolve to finally learn measure theory. So naturally it lands on a list, next to the other things you will get to when a free week appears.

But there is nothing to acquire first. Consuming a skill and authoring one are one gesture apart. Eric is, by his own description, already a fluent consumer of skills; the distance from there to his first authored skill is a single sentence to the agent at the end of a task he was doing anyway. It is not a project for next quarter. It is the next thing he says out loud. The to-do-list framing is not a personal oversight, it is the residue of a learning model the tools have quietly retired.

The bottleneck is courage, not capability

If the old barriers are gone, it is worth being exact about what they were, because naming them shows what is left.

You used to be blocked by not knowing how – the technique lived in a book you had not read. You were blocked by not having the information – the answer sat in a thread you had not found. And you were blocked by the cost of bridging – even with the technique in hand, fitting it into your specific project took real work. An agent now supplies all three on demand: the how, the information, and the integration into the very file in front of you.

Strip those away and one thing remains. You have to be willing to say, out loud, to a machine: I don’t know how to do this – show me. You have to start before you feel ready and let the not-knowing be the price of admission rather than a reason to wait. That readiness deserves a name, so I will call it cognitive courage: the willingness to enter a problem you cannot yet solve and learn it in the doing.

Cognitive courage is now the binding constraint on how fast you learn. Not your ability, not your access to information, not your skill at integration, all of which are suddenly abundant. The scarce input is the nerve to begin.

Show the code that draws the diagram

suppressPackageStartupMessages(library(tidyverse))

blue   <- "#2C7FB8"
gold   <- "#EDB458"
grey   <- "#5B6770"
fill_n <- "#D6EAF2"   # NOW boxes
fill_t <- "grey95"    # THEN boxes
ink_n  <- "#1b3b52"

hw <- 0.95; hh <- 0.42                # box half-width / half-height

# ---- THEN: a straight, linear track (y = 3.6) ------------------------
ty <- 3.6
tx <- c(1.4, 4.0, 6.6, 9.2)
then_rect <- tibble(xmin = tx - hw, xmax = tx + hw, ymin = ty - hh, ymax = ty + hh)
then_txt  <- tibble(
  x = tx, y = ty,
  label = c("Pick a\nskill", "Study\nbooks & docs",
            "Get stuck,\nsearch, ask", "Finally\napply")
)
then_arr <- tibble(x = tx[-4] + hw, xend = tx[-1] - hw, y = ty, yend = ty)

# ---- NOW: a loop you enter through a gate (y = 1.0) ------------------
ny <- 1.0
gx <- 1.1
nx <- c(3.7, 6.3, 8.9)                # Try, Ask, Learn
now_rect  <- tibble(xmin = nx - hw, xmax = nx + hw, ymin = ny - hh, ymax = ny + hh)
now_txt   <- tibble(x = nx, y = ny,
                    label = c("Try it in\nreal work", "Ask the\nagent", "Learn"))
gate_rect <- tibble(xmin = gx - hw, xmax = gx + hw, ymin = ny - hh, ymax = ny + hh)
gate_txt  <- tibble(x = gx, y = ny, label = "Cognitive\ncourage")

# forward arrows: gate -> Try -> Ask -> Learn
now_arr <- tibble(x = c(gx, nx[1], nx[2]) + hw,
                  xend = c(nx[1], nx[2], nx[3]) - hw, y = ny, yend = ny)
# return loop: Learn -> Try, arcing up and over the row
ret <- tibble(x = nx[3], y = ny + hh, xend = nx[1], yend = ny + hh)

ggplot() +
  annotate("text", x = 0.15, y = 4.62, hjust = 0, fontface = "bold",
           size = 4.1, colour = grey,
           label = "THEN  ·  learning was a separate project") +
  annotate("text", x = 0.15, y = 2.62, hjust = 0, fontface = "bold",
           size = 4.1, colour = blue,
           label = "NOW  ·  learning happens inside the work") +

  # THEN track
  geom_rect(data = then_rect,
            aes(xmin = xmin, xmax = xmax, ymin = ymin, ymax = ymax),
            fill = fill_t, colour = grey, linewidth = 0.5) +
  geom_segment(data = then_arr, aes(x = x, y = y, xend = xend, yend = yend),
               arrow = arrow(length = unit(0.16, "cm"), type = "closed"),
               linewidth = 0.5, colour = grey) +
  geom_text(data = then_txt, aes(x, y, label = label),
            fontface = "bold", size = 3.05, lineheight = 0.92, colour = grey) +
  annotate("text", x = 10.75, y = ty, hjust = 0, fontface = "italic",
           size = 3.1, colour = grey, label = "weeks") +

  # NOW return loop (behind the boxes)
  geom_curve(data = ret, aes(x = x, y = y, xend = xend, yend = yend),
             curvature = 0.24, ncp = 16, angle = 90,
             arrow = arrow(length = unit(0.16, "cm"), type = "closed"),
             linewidth = 0.55, colour = blue) +
  annotate("text", x = (nx[1] + nx[3]) / 2, y = 2.34, fontface = "italic",
           size = 3.1, colour = blue, label = "repeat in minutes") +

  # NOW gate, forward arrows, boxes
  geom_rect(data = gate_rect,
            aes(xmin = xmin, xmax = xmax, ymin = ymin, ymax = ymax),
            fill = gold, colour = "grey35", linewidth = 0.5) +
  geom_text(data = gate_txt, aes(x, y, label = label),
            fontface = "bold", size = 3.05, lineheight = 0.92, colour = "grey15") +
  geom_segment(data = now_arr, aes(x = x, y = y, xend = xend, yend = yend),
               arrow = arrow(length = unit(0.16, "cm"), type = "closed"),
               linewidth = 0.55, colour = blue) +
  geom_rect(data = now_rect,
            aes(xmin = xmin, xmax = xmax, ymin = ymin, ymax = ymax),
            fill = fill_n, colour = blue, linewidth = 0.6) +
  geom_text(data = now_txt, aes(x, y, label = label),
            fontface = "bold", size = 3.05, lineheight = 0.92, colour = ink_n) +

  coord_fixed(clip = "off", xlim = c(0, 11.7), ylim = c(0.4, 4.95)) +
  theme_void() -> p

ggsave(
  here::here("posts", "2026-07-27-cognitive-courage", "courage-loop.png"),
  p, width = 8, height = 3.2, dpi = 160, bg = "white"
)

p

Encourage, literally

There is a reason the English word is built the way it is. To encourage someone is, etymologically, to put courage into them: en- (to make) plus courage. If the bottleneck is courage, then the most useful thing we can do for each other, and for ourselves, is exactly that. Lower the cost of admitting you do not know. Treat “show me how” as the strong move it is, not the embarrassing one.

So here is a small call to action, mostly to my fellow data scientists. Embrace the gap between what you can do and what you would like to. Shorten the loop until learning and working are the same motion. The next time you catch yourself filing a tool under “things to learn this year,” stop, open the work you are already doing, and ask the agent to do it with you, now. As of now, the skill isn’t the obstacle. The courage to reach for it, in public, mid-task, before you are ready, is what makes the learning of this exciting new era.

In an earlier post I argued that data scientists are unusually well positioned for the AI era because the role already demanded the right mix of skills. This is the complimentary insight. The floor rose, the tools are sitting right there, and the only upgrade still required is in your willingness to reach.

What I run

The setup is one open-source engine and one model. The engine is faster-whisper, a fast reimplementation of OpenAI’s Whisper. The model is ivrit-ai/whisper-large-v3-turbo-ct2, a Hebrew fine-tune of Whisper’s large-v3-turbo.

It all runs on a regular laptop: an AMD Ryzen 7 8845HS with 32 GB of RAM and no usable GPU. The machine does have an integrated AMD GPU, but faster-whisper only accelerates on NVIDIA cards, so in practice everything runs on the CPU. That turns out to be fine.

Three ways I use it

Transcribing online meetings. After a call, I run the recording through a small Claude Code skill that calls the local model and writes a timestamped transcript. Nothing leaves the machine, which matters when the meeting was not mine to upload to some third-party service in the first place.

Handing an LLM the full context, by talking. When I want a model, usually Claude Code, to really understand what I am after, typing is the bottleneck. People who work with these tools have made the point well: speaking carries far more context than typing. Typing pushes you toward short, careful, pruned input, while talking lets you spill everything that is actually in your head. I do this two ways, both local: a push-to-talk key that dictates straight into the prompt, and longer recordings I transcribe and hand over. More on the hotkey just below.

Recording while driving. This is how this very post came to be. I record a long voice memo in the car describing what I want to build, and when I get to my desk I drop the audio file into Claude Code together with a prompt. Claude transcribes it and already has the full context for the task. You are reading the output of exactly that loop.

Talking to the model with a hotkey

The “by talking” part started as recorded memos, but the version I reach for all day is live. I wired a push-to-talk dictation key into the laptop: I tap the ` key, just left of the 1, say whatever I want in Hebrew, tap it again, and a second or two later the transcribed text is pasted straight into whatever has focus, whether that is the Claude Code prompt, an editor, or a browser field. The same local model does the work, so it stays offline and free, and it handles Hebrew that the built-in dictation does not.

It runs as a small background daemon that loads the model once and stays resident, so each tap only waits for the transcription itself, never for the model to load again. It starts hidden at login, so the key is simply always there. A second shortcut, Ctrl+Alt+`, toggles dictation on and off, so when I actually want to type a backtick the key behaves normally again.

This is the piece that turns “hand the model full context by talking” into a habit rather than a chore. There is no app to open and no file to move: think out loud, tap, and the model has the paragraph you would never have bothered to type.

Is it good, and is it fast?

Quality. The Hebrew fine-tune is genuinely good on spontaneous, accented speech, well beyond the generic Whisper model of the same size. Numbers and dates sometimes come out spelled as words, which is a quick fix.

Speed. I measure it as the ratio of audio length to processing time. On my CPU the turbo model runs at roughly 1.0 to 1.4x real time: a 9.5-minute recording took about 9.5 minutes, and a 17-minute recording about 12, since the longer one had more silence for the voice-activity filter to skip. A typical meeting transcribes in about its own length, in the background, while I get on with something else.

Which model for which machine

You do not need my exact setup. The model you can run depends mostly on your RAM, and on whether you have an NVIDIA GPU:

The standout option is large-v3-turbo: at 809 million parameters it is roughly half the size of full large-v3 and several times faster, for almost the same accuracy. For a non-English language, skip the tiny and base models, which are weak outside English. Use small at the very least, and look for a fine-tune in your language before settling for a generic model. A dedicated fine-tune usually beats the generic model of the same size, which is exactly why I use the Hebrew one.

Setting it up yourself

I put the whole setup in a small, standalone repo you can clone, fork, or star: github.com/matanhakim/local-speech-to-text. It has three standalone parts: setting up the local Whisper model, the push-to-talk dictation daemon described above, and the transcription skill you can drop into Claude Code or run on its own. The model-setup guide doubles as a recipe you can hand to Claude Code or any coding agent: it inspects your hardware, picks a model that fits your RAM and your language, installs faster-whisper, and drops in a transcribe script you can reuse. The speech always runs locally: no cloud, no API key, no meter.

The next step on my side is to take an old laptop that does have an NVIDIA GPU and turn it into a small transcription server, which would let me run an even stronger model several times faster. For now, a regular CPU laptop already covers everything above, in a language the big tools quietly leave out.

The intersection: three circles

In 2010, Drew Conway drew what became the most-shared sketch in the field: the Data Science Venn Diagram.¹ It places data science at the overlap of three skill sets: hacking skills (the ability to work with data and code), math and statistics knowledge, and substantive expertise in some domain.

Show the code that draws the diagram

library(tidyverse)

unit_circle <- function(x0, y0, r = 1.35, n = 200, label) {
  t <- seq(0, 2 * pi, length.out = n)
  tibble(x = x0 + r * cos(t), y = y0 + r * sin(t), set = label)
}

circles <- bind_rows(
  unit_circle(0,     0.80,  label = "Math & Statistics"),
  unit_circle(-0.72, -0.45, label = "Hacking Skills"),
  unit_circle(0.72,  -0.45, label = "Substantive Expertise")
)

labels <- tribble(
  ~x,    ~y,    ~text,                     ~kind,
  0,     1.55,  "Math &\nStatistics",      "set",
  -1.25, -1.00, "Hacking\nSkills",         "set",
  1.25,  -1.00, "Substantive\nExpertise",  "set",
  -0.62, 0.42,  "Machine\nLearning",       "region",
  0.62,  0.42,  "Traditional\nResearch",   "region",
  0,     -0.80, "Danger\nZone",            "region",
  0,     -0.02, "Data\nScience",           "core"
)

p <- ggplot(circles, aes(x, y)) +
  geom_polygon(
    aes(fill = set, group = set),
    alpha = 0.35, colour = "grey30", linewidth = 0.4
  ) +
  geom_text(
    data = filter(labels, kind == "set"), aes(x, y, label = text),
    inherit.aes = FALSE, fontface = "bold", size = 3.6, lineheight = 0.9
  ) +
  geom_text(
    data = filter(labels, kind == "region"), aes(x, y, label = text),
    inherit.aes = FALSE, fontface = "italic", size = 3.0,
    lineheight = 0.9, colour = "grey15"
  ) +
  geom_text(
    data = filter(labels, kind == "core"), aes(x, y, label = text),
    inherit.aes = FALSE, fontface = "bold", size = 3.9, lineheight = 0.9
  ) +
  scale_fill_manual(
    values = c(
      "Math & Statistics"     = "#2C7FB8",
      "Hacking Skills"        = "#7FCDBB",
      "Substantive Expertise" = "#EDB458"
    ),
    guide = "none"
  ) +
  coord_fixed(clip = "off") +
  theme_void()

ggsave(
  here::here(
    "posts", "2026-06-24-data-scientists-ai-revolution", "conway-venn.png"
  ),
  p, width = 6.5, height = 5.5, dpi = 150, bg = "white"
)

p

The center of the diagram is the famous part, but the region that matters most for this argument sits at one of the edges. Where hacking skills and substantive expertise overlap without statistics, Conway drew the danger zone: people who, in his words, “know enough to be dangerous.” They can pull and shape data, they know the domain, they can even run a regression – but they cannot tell what the output actually means, so they produce something that looks like a legitimate analysis and is quietly wrong.

Now think about what a large language model does to that picture. It hands hacking skills to anyone who can type a sentence, so overnight far more people can pull data and run a model – and Conway’s danger zone gets a lot more crowded. What keeps a data scientist out of it is holding all three circles at once, which is exactly what the job has always demanded. So it is worth taking the circles one at a time, each one an independent reason the role compounds with AI instead of being threatened by it.

Hacking skills: you can direct an agent, not just vibe with it

Andrej Karpathy coined “vibe coding” in early 2025: building software by prompting, accepting suggestions, and never really reading the code. For someone with no engineering background, that is the only mode available – you take what the model gives you and hope.

Data scientists are not software engineers, but they have absorbed software engineering’s principles. We use version control with git. We know what unit tests are and why input validation matters. We know that code headed for production behaves differently from a script that only ever runs on our own laptop. Many R users have built a package, which is software engineering in miniature: tests, documentation, dependency management, the discipline of a public interface. We are not the people who write the world’s best edge-case tests, but we know the principles and how to apply them.

That base changes the relationship with a coding agent entirely. You are in the driver’s seat – reading the diff, smelling out a bad abstraction, asking for tests, insisting on a reproducible setup, catching the silent failure before it ships. You can build things that were out of reach before, larger in scope and with real interfaces and integrations, at a quality close to professional software, without ever having had the full engineering background. The habits that make a good R package carry straight over, too. Writing a function forces you to decide exactly what goes in, what comes out, and what counts as correct – and that same act of specifying is what gets useful work out of an agent. The clearer the contract you hand it, down to the test that has to pass, the better the result it gives back.

Substantive expertise: you are the human in the loop who actually knows the domain

Every data scientist specializes in something. It might be pharma and clinical trials, finance, marketing, or, in my case, socio-economic research, and usually it narrows further inside the field. That domain knowledge is the second circle, and it is what gives the phrase “human in the loop” any teeth.

The models hallucinate, and they overreach. A human in the loop earns their place only if they can tell when the output is wrong, and domain expertise is what makes that possible. It is what lets you catch the confident, plausible, wrong answer: the invented variable, the misread coefficient, the two findings stitched together in a way that does not actually hold in your field. Without that expertise you are not supervising the model at all; you are just passing its output along with extra steps.

This is what separates running a model from vouching for it. Anyone can prompt the tool and read back what it says. Knowing your field is what lets you stand behind the answer, or push back on it, and that is the only kind of human oversight that actually changes the outcome.

Math and statistics: you know what kind of thing a model is

This is the circle whose link to AI is the least obvious of the three, so it is worth being concrete. Under the hood, a large language model is a statistical model, and statistical training builds an intuition that carries straight over.

A statistician never takes a point estimate at face value. They ask what the spread around it looks like, how far to trust it, how much it would move under a different sample. That is exactly the right reflex for a system that can give you two different answers to the same question and is confidently wrong a measurable share of the time. You treat each response as one draw to be checked, never as the final word.

“How would I know if this is wrong?” is the question statistical training drills into you. Designing a check, holding out a test, quantifying the error – that is the core of the discipline, and it is also what separates evaluating an AI’s output from simply trusting it. Writing an eval for a prompt draws on the same instinct as building a validation set for a model.

You can smell a bad quantitative claim. Models produce plausible-but-wrong statistical reasoning all the time, from correlation dressed up as causation to a confidence interval that does not mean what it says. Statistical literacy is the filter that stops those claims before they reach a stakeholder.

The through-line is calibrated distrust. Conway’s danger zone exists precisely because hacking and domain knowledge without statistics produce confident nonsense, and AI makes confident nonsense cheap to produce at scale. Statistics is the circle that inoculates you against it.

Where it plays out: the data science workflow

Zoom in from who the data scientist is to what they do all day. The book that taught a generation of us, R for Data Science by Hadley Wickham, Mine Çetinkaya-Rundel, and Garrett Grolemund, opens with a workflow diagram that survives unchanged from its first edition to its second. Data flows through import, then tidy, into an iterative understand loop of transform, visualize, and model, and finally out through communicate. Tidying and transforming together are called wrangling. Wrapped around all of it is programming.

Show the code that draws the diagram

hw <- 0.86
hh <- 0.34
y0 <- 2

boxes <- tribble(
  ~stage,        ~x,
  "Import",      1.0,
  "Tidy",        3.0,
  "Transform",   5.0,
  "Visualize",   7.0,
  "Model",       9.0,
  "Communicate", 11.0
) |>
  mutate(y = y0, xmin = x - hw, xmax = x + hw, ymin = y - hh, ymax = y + hh)

main <- boxes |>
  mutate(xend_box = lead(x)) |>
  filter(!is.na(xend_box)) |>
  transmute(x = x + hw, y = y0, xend = xend_box - hw, yend = y0)

loop <- tibble(
  x = boxes$x[boxes$stage == "Model"], y = y0 - hh,
  xend = boxes$x[boxes$stage == "Transform"], yend = y0 - hh
)

frames <- tribble(
  ~xmin, ~xmax, ~ymin, ~ymax, ~col,
  0.0,  12.0,  0.95,  2.92, "#5B6770",
  3.9,  10.1,  1.18,  2.60, "#EDB458"
)

ggplot() +
  geom_rect(
    data = frames, aes(xmin = xmin, xmax = xmax, ymin = ymin, ymax = ymax),
    fill = frames$col, alpha = 0.10, colour = frames$col, linewidth = 0.5
  ) +
  annotate("text", x = 0.18, y = 2.76, label = "Program",
           hjust = 0, fontface = "bold", size = 3.4, colour = "#5B6770") +
  annotate("text", x = 7.0, y = 2.46, label = "Understand",
           fontface = "bold", size = 3.4, colour = "#B8860B") +
  geom_segment(
    data = main, aes(x = x, y = y, xend = xend, yend = yend),
    arrow = arrow(length = unit(0.16, "cm"), type = "closed"),
    linewidth = 0.5, colour = "grey25"
  ) +
  geom_curve(
    data = loop, aes(x = x, y = y, xend = xend, yend = yend),
    curvature = -0.45, ncp = 12,
    arrow = arrow(length = unit(0.18, "cm"), type = "closed"),
    linewidth = 0.6, colour = "#B8860B"
  ) +
  annotate("text", x = 7.0, y = 1.30, label = "iterate",
           fontface = "italic", size = 2.9, colour = "#B8860B") +
  geom_rect(
    data = boxes, aes(xmin = xmin, xmax = xmax, ymin = ymin, ymax = ymax),
    fill = "#D6EAF2", colour = "#2C7FB8", linewidth = 0.6
  ) +
  geom_text(
    data = boxes, aes(x = x, y = y, label = stage),
    fontface = "bold", size = 3.2, colour = "#1b3b52"
  ) +
  coord_fixed(clip = "off") +
  theme_void()

Map the tools onto the diagram and they fall into three groups.

The data work – import, tidy, transform, model, visualize – is where the models are genuinely strong. There is an old joke that a data scientist spends 80% of the time cleaning data and the other 20% complaining about cleaning data, and tools built for this work, like Claude Code and the data-aware Posit Assistant, are aimed straight at that 80%: the boilerplate, the join you have written a hundred times, the ggplot incantation you can never quite remember. But every dataset has its gotchas – the wrong encoding, the duplicated key, the column that quietly changed meaning after 2016 – and someone has to recognize them, join the right tables, and spot the edge case. That someone is the data scientist; the model does the typing.

The programming envelope is the first circle again, made concrete. The reason you can hand an agent the data work and trust what comes back is that you bring the engineering frame – reproducibility, tests, version control – that a journalist or a pure analyst opening Claude Code for the first time simply does not have. They are starting software development from zero. You are not.

Communication is the last stage and the most underrated. Conveying results used to mean fighting the tooling: nudging a bar two pixels to the left, re-googling how to add a second y-axis (which, yes, I know is a crime against data visualization – don’t @ me), wrestling a report into shape. That friction is largely gone. What is left is the part that was always both the hardest and the most valuable. Does the chart tell a story? Is there a narrative a decision-maker can actually follow? Are you quietly assuming your audience reads a chart as fluently as you do, when they do not? Data scientists spent years learning these principles in the cracks between the technical fiddling. Now the fiddling is cheap, so the hour you used to lose hunting for the right legend position goes instead into whether the headline finding survives contact with a busy reader. The skill that used to be a luxury is now where the whole job happens.

A few more things the role gives you

The two diagrams cover most of it, but a few habits of the job belong here too.

The reproducible mindset is the AI-supervision mindset. Data scientists already think in rerunnable pipelines – scripts, notebooks, Quarto, renv lockfiles – where any result can be rebuilt from scratch. That is exactly the discipline that lets you trust agent output: if you can regenerate the whole thing on demand, you can let a machine write the steps and still stand behind the result. Supervising an AI is not a new habit for us; it is the reproducibility habit pointed at a faster, less reliable collaborator.

An analysis is already a chain of well-defined steps, and agents do their best work on decomposed, well-specified tasks. The habit of breaking a problem into import, clean, model, report is the same habit that produces a good sequence of prompts.

Translation has always been the actual job – turning a fuzzy stakeholder question into a precise computational one, and the answer back into something a human can act on. That is almost exactly the skill of turning an underspecified goal into a spec an agent can execute, then checking the result against what was really being asked.

The floor rose

None of this means the data scientist is safe while everyone else is doomed – that is not the argument. The argument is that the floor rose. The technical toil that used to eat the day – the syntax, the boilerplate, the pixel-nudging – is getting cheap, and value moves up the stack to judgment, context, and communication. The three circles Conway drew in 2010 turn out to be a strikingly good description of who already has that judgment.

The danger zone has never been more crowded, now that anyone can hack. What keeps you out of it – all three circles at once – is the thing that has always defined the work. Data scientists did not have to retool for the AI era; the role was already shaped like the era’s demands. The job now is to lean into the part that was always the point: understand the problem, trust the output only as far as it has earned, and tell the story.

What Does il.cbs.muni Do?

The Israeli CBS publishes extensive municipal data covering cities, local councils, and regional councils. However, this data comes with challenges:

• Inconsistent formats across different years
• Different Excel structures for various data domains
• Multiple ID systems used by different government agencies
• Separate sheets for different municipality types (pre-2016)
• Complex headers that need careful processing

il.cbs.muni solves these problems by providing a consistent interface to read, process, and combine Israeli municipal data from different sources and years.

Key Features

library(il.cbs.muni)
library(dplyr)

# Read physical and demographic data for 2021
df1 <- read_cbs_muni(
  path = "p_libud_2021.xlsx",
  year = 2021,
  data_domain = "physical"
)

df1

df2 <- read_cbs_muni(
  "p_libud_2021.xlsx",
  2021,
  data_domain = "budget",
  cols = c(2, 50)
)

df2

df <- df1 |> 
  left_join(df2, join_by(`כללי_סמל הרשות` == `סמל הרשות`)) |> 
  relocate(contains("שירותים ממלכתיים: חינוך"))

df

# Combine data from both sheets
df_combined <- combine_cbs_muni(
  path = "2009.xls",
  year = 2009,
  cols_city = c(1:7, 11),
  cols_rc = c(1:7, 25),
  data_domain = "physical"
)

df_combined

df_combined[[10,1]]

[1] "באקה-ג'ת*"

df_combined <- df_combined |> 
  mutate(
    `שם הרשות` = clean_name(`שם הרשות`)
  )

df_combined[[10,1]]

[1] "באקה-ג'ת"

# Read yishuv data with selected columns
df_yishuv <- read_cbs_yishuv(
  path = "bycode2021.xlsx",
  cols = c(1, 2, 5, 13)
)

df_yishuv

df_yishuv <- df_yishuv |> 
  mutate(
    `סמל יישוב` = pad_yishuv_id(`סמל יישוב`)
  )

  df_yishuv

# Read socio-economic index at municipality level
df_ses <- read_cbs_index(
  path = "T02.xlsx",
  year = 2019,
  index_type = "ses",
  unit_type = "muni"
)

df_ses

df_ses <- df_ses |> 
  mutate(
    muni_id = modify_muni_id(`מעמד מוניציפלי_MUNICIPAL STATUS`, `סמל יישוב_CODE OF LOCALITY`),
    yishuv_id = pad_yishuv_id(`סמל יישוב_CODE OF LOCALITY`),
    .before = 1
  )

  df_ses

# Get IDs and names from multiple agencies
muni_ids <- read_muni_id(
  id_types = c("muni", "edu", "tax"),
  include_names = TRUE
)

muni_ids

Resources

• Documentation: https://matanhakim.github.io/il.cbs.muni/
• Source code: https://github.com/matanhakim/il.cbs.muni
• Bug reports: https://github.com/matanhakim/il.cbs.muni/issues
• CRAN page: https://cran.r-project.org/package=il.cbs.muni

Future Plans

I’m hoping to add:

• More helper functions for common municipal data transformations
• Support for additional CBS data sources
• Vignettes with detailed workflows
• A hex sticker!
• Anything that the Israeli open source community might be interested in

Acknowledgments

This package builds on the excellent work of the Israeli CBS in publishing municipal data. Special thanks to the tidyverse team for creating the tools that make this package possible. Specifically, this package was able to get submitted to CRAN with full tests and input validation thanks to Positron Assistant, which has been a huge blessing.

Happy analyzing! 📊

Introduction

Summer is when you have time in the NBA media and fandom world to relax for a second, and look for multi-year trends. I find specifically interesting the NBA discussion that is filled with talk about the “3-point revolution”, with additional references to “the death of the mid-range shot”. And so it seems, the familiar chart of the 3-pt shot frequency going up over time is seen everywhere you go, and considered common knowledge. More specifically, I encountered this discussion with many nuances in the Thinking Basketball podcast with Ben Taylor and Cody Houdek. Nevertheless, I think it is worthwhile to have a more focused look at the specific ranges where this change is occurring, and more specifically at the different positions and the variability between them.

This blog post will explore this idea, using the R programming language and data scraping, analysis and visualization techniques. If you’re interested in either NBA basketball or data, I think you might find it useful, interesting, and even fun!

Let’s get to it.

Code

1library(tidyverse)
2library(rvest)
3devtools::install_github("sfirke/janitor")
library(janitor)
4library(scales)
5library(ggthemes)
6library(ggiraph)
7library(conflicted)
conflicts_prefer(dplyr::filter())

Read data

We will be using the Basketball Reference website as our data source. Basketball Reference is the gold standard for NBA statistics, offering comprehensive shooting data broken down by distance ranges since the 1997-98 season.

Before actually scraping the data, we need to define our functions that would do so.

read_season_dist <- function(year) {
1  Sys.sleep(4)
  read_html(str_c("https://www.basketball-reference.com/leagues/NBA_", year, "_shooting.html")) |> 
    html_element("table") |> 
    html_table(header = FALSE) |> 
2    row_to_names(1:2) |>
    clean_names() |> 
3    mutate(across(everything(), parse_guess))
}

read_season_total <- function(year) {
  Sys.sleep(4)
  read_html(str_c("https://www.basketball-reference.com/leagues/NBA_", year, "_totals.html")) |> 
    html_element("table") |> 
    html_table() |> 
    clean_names()
}

df_dist_raw <- tibble(year = 1998:2025) |> 
  mutate(data = map(year, read_season_dist)) |> 
  unnest(data)
  
df_players_raw <-  tibble(year = 1998:2025) |> 
  mutate(data = map(year, read_season_total)) |> 
  unnest(data)

df_players <- df_players_raw |> 
1  distinct(player, year, .keep_all = TRUE) |>
2  select(year, player, fg, fga)

df_dist <- df_dist_raw |> 
  distinct(player, year, .keep_all = TRUE) |> 
3  select(!c(rk, awards))

df <- df_players |> 
  left_join(df_dist, join_by(player, year)) |> 
1  relocate(c(fg, fga), .after = mp)

Manipulate data

Now comes the fun part - transforming our data into a format suitable for analysis. We need to convert the wide format (where each distance range is a separate column) into a long format that’s easier to work with for visualization and analysis. Additionally, we will summarize the data into a frequency table with a single row for every year-position-distance.

df_freq <- df |> 
  select(year, pos, fg:fga, starts_with("percent_of_fga_by_distance"), dunks_percent_fga) |> 
1  pivot_longer(!c(year, pos:fga), names_to = "dist", values_to = "freq") |>
2  mutate(dist = str_remove_all(dist, "percent_of_fga_by_distance_|s_percent_fga")) |>
  mutate(
3    dist = fct_relevel(dist, "0_3", "3_10", "10_16", "16_3p", "3p") |>
      str_replace_all("_", "-"),
4    pos = fct_relevel(pos, "C", "PF", "SF", "SG", "PG")
  ) |>
5  filter(fga > 0, year > 1997) |>
  summarise(
    .by = c(year, pos, dist),
6    freq = weighted.mean(freq, fga)
  )

Let’s also create a league-wide summary without position breakdowns:

df_freq_tot <- df |> 
  select(year, fg:fga, starts_with("percent_of_fga_by_distance"), dunks_percent_fga) |> 
  pivot_longer(!c(year, fg:fga), names_to = "dist", values_to = "freq") |> 
  mutate(dist = str_remove_all(dist, "percent_of_fga_by_distance_|s_percent_fga")) |> 
  mutate(
    dist = fct_relevel(dist, "0_3", "3_10", "10_16", "16_3p", "3p") |> 
      str_replace_all("_", "-")
  ) |>
  filter(fga > 0, year > 1997) |> 
  summarise(
    .by = c(year, dist),
    freq = weighted.mean(freq, fga)
  )

Visualize

plot_total <- df_freq_tot |> 
1  filter(!(dist %in% c("2p", "dunk"))) |>
2  mutate(dist = fct_reorder2(dist, year, freq)) |>
  ggplot(aes(year, freq, color = dist, tooltip = dist, data_id = dist)) +
  geom_line_interactive(linewidth = 1) + 
  scale_y_continuous(labels = label_percent()) +
  scale_color_solarized() +
  labs(
    x = "",
    y = "Shot Frequency",
    color = "Distance (feet)",
    title = "Shot Frequency by Year and Distance from the Basket",
    subtitle = "The 3-point revolution meets the mid-range exodus",
    caption = "Data: basketball-reference.com. Analysis: Matan Hakim."
  )

girafe(ggobj = plot_total)

plot_position <- df_freq |> 
  filter(!(dist %in% c("2p", "dunk"))) |> 
  ggplot(aes(year, freq, color = pos, tooltip = pos, data_id = pos)) +
  geom_line_interactive(linewidth = 0.7) +
1  facet_wrap(~ dist, scales = "free_y") +
  scale_y_continuous(labels = label_percent()) +
  scale_color_solarized() +
  labs(
    x = "",
    y = "Shot Frequency",
    color = "Position",
    title = "Shot Frequency by Year, Position and Distance from the Basket",
    subtitle = "How the 3-point revolution affected different positions",
    caption = "Data: basketball-reference.com. Analysis: Matan Hakim."
  )

girafe(ggobj = plot_position)

Analysis

Conclusion

This deep dive into NBA shot selection reveals that the commonly discussed “3-point revolution” is far more complex and position-specific than the typical narrative suggests. We’re not just witnessing one trend, but rather multiple simultaneous transformations happening at different paces across different positions and distance ranges.

The data tells us several key stories: the complete extinction of long mid-range shots, the unexpected resilience of short floaters, the growing positional polarization at the rim, and the staggered adoption of 3-point shooting that has fundamentally redefined every position on the court.

Perhaps most remarkably, we can see how basketball has become more specialized yet more homogeneous. Centers and point guards now take shots from completely different areas of the court (specialization), but when it comes to avoiding the long mid-range, every position has reached the same conclusion (homogenization).

Looking ahead, the recent surge in 3-point attempts suggests this revolution may still have room to run. As defensive schemes continue to adapt and new analytical insights emerge, shot selection will undoubtedly continue evolving. The only certainty is that the mid-range shot from 16-24 feet - once a staple of NBA offense - appears to be permanently relegated to basketball history.

Thank you for tagging along on this journey through nearly three decades of NBA evolution!

Introduction

It’s hard to point on the specific moment where my passion for the NBA started. It might have been somewhere during the 1997-98 season, when I remember myself having a huge Michael Jordan poster hanging over my bed. I was in second grade, barely knowing what basketball even is, but I had to have MJ with me. It might have been during 8th grade, When my friend and I started playing NBA Live 2005 on our PCs, following that with backyard imitations of Kobe Bryant and Lebron James. It even might have been while watching the 2008 NBA Finals live (game started on 3AM Israel time, mind you), and being part of the amazing “Anything’s possible” moment from Kevin Garnett. But I guess the moment the NBA really captured me was during the 2014-15 season, the season that featured rise of Steph Curry and the Golden Warriors, alongside the new era of NBA analytics.

Since then, I am a regular follower of NBA content, specifically everything related to NBA data and analytics. For example, I’m a huge fan of the “Thinking Basketball” Youtube Channel by Ben Taylor, who is a master of bringing data and context together. Additionally, as an Israeli and a Hebrew speaker, I regularly follow the “7 Seconds or Less” Podcast (in Hebrew) by Yuval Oz and Oren Levi, who enjoy having discussions around evolving trends and shifts, rather than the latest “Woj Bomb”. These conversations tend to revolve around many topics, but one of the latest themes I found most interesting is NBA players’ height.

So, the question that guided me was “what are the trends regarding NBA players’ height over the years?”. Specifically, I was interested in the average height (a measure of central tendency) and the standard deviation of height (a measure of spread), and their evolution over time. Finally, I thought it was proper to analyse effective height, meaning the height of the players playing on the court, rather than the height of players on a team. For example, let’s compare Boban Marjanović and Trae Young. Marjanović, the tallest player in the 2022-23 NBA season standing at 7-3, played a total of 171 minutes for the Houston Rockets on that season. On the other hand, Young, standing at merely 6-1, played a total of 2,541 minutes for the Atlanta Hawks during the 2022-23 season - almost 15 times more minutes played than Marjanović. Obviously, we need to take these differences into account, and this is why I weighted the different statistics by minutes played.

This blog post describes both the process data collection and manipulation with the R programming language, and the final visuals with some explanatory text. Now, you might be here for either (or both!) of these reasons: data analysis with R, or NBA. If you’re not interested in R code involving web scraping, data manipulation and visualization, you can skip ahead to the NBA players’ height over the years section.

Let’s dive right in!

Code

library(tidyverse)
library(rvest)
library(janitor)
library(scales)
library(jtools)
library(gganimate)
library(conflicted)
conflicts_prefer(dplyr::filter())

read_season <- function(year) {
1  Sys.sleep(4)
  read_html(str_c("https://www.basketball-reference.com/leagues/NBA_", year, "_totals.html")) |> 
    html_element("table") |> 
    html_table() |> 
    clean_names() |> 
2    filter(str_detect(rk, "[:digit:]")) |>
    select(player, pos, age, mp)
}

df_seasons_raw <- tibble(year = 1952:2023) |>
  mutate(data = map(year, read_season)) |>
  unnest(data)

read_players_letter <- function(letter) {
1  Sys.sleep(4)
  read_html(str_c("https://www.basketball-reference.com/players/", letter)) |> 
    html_element("table") |> 
    html_table() |> 
    clean_names() |> 
2    filter(str_detect(from, "[:digit:]")) |>
3    mutate(player = str_remove_all(player, "\\*"))
}

letters_no_x <- letters[letters != "x"]
df_players_raw <- map(letters_no_x, read_players_letter) |>
  list_rbind()

df_players_raw |> 
  count(player, sort = TRUE) |> 
  head(n = 10) |> 
  knitr::kable()

df_players <- df_players_raw |> 
1  separate(ht, into = c("ht_feet", "ht_inches"), sep = "-", remove = FALSE) |>
  mutate(
2    ht_tot_inches = parse_double(ht_feet) * 12 + parse_double(ht_inches),
    ht_tot_cm = ht_tot_inches * 2.54,
3    birth_date = parse_date_time(birth_date, "bdy"),
    birth_year = year(birth_date),
4    birth_year_effective = if_else(
      month(birth_date) > 1,
      birth_year + 1,
      birth_year
    )
  )

df_seasons <- df_seasons_raw |> 
  mutate(
1    player = str_remove_all(player, "\\*"),
    birth_year_effective = year - age
  )

df_all <- df_seasons |> 
  left_join(df_players, join_by(player, birth_year_effective)) |> 
  mutate(
1    decade = (year - year %% 10) |>
      factor() |>
      str_c("'s")
  )

weighted_mean_by <- function(data, mean_var, by_vars, weights) {
  data |> 
1    group_by({{ by_vars }}) |>
    summarise(
      mean = weighted.mean({{ mean_var }}, {{ weights }}),
      sd = wtd.sd({{ mean_var }}, {{ weights }}),
      sd_upper = mean + sd,
      sd_lower = mean - sd,
      .groups = "drop"
    )
}

df_mean_years <- df_all |> 
  weighted_mean_by(ht_tot_inches, year, mp)

df_mean_decades <- df_all |> 
  weighted_mean_by(ht_tot_inches, decade, mp)

summarize_heigths <- function(data, var, by_vars, weights = NULL) {
  data |> 
    count({{ by_vars }}, {{ var }}, wt = {{ weights }}) |> 
    mutate(
      .by = {{ by_vars }},
      pct = n / sum(n),
    )
}

df_heights_years <- df_all |> 
  summarize_heigths(ht_tot_inches, by_vars = year, weights = mp)

df_heights_decades <- df_all |>
  summarize_heigths(ht_tot_inches, by_vars = decade, weights = mp)

inches_to_feet <- function(x) {
  str_c(
    x %/% 12,
    x %% 12,
    sep = "-"
  )
}

breaks_height <- (23:29) * 3
labels_height <- inches_to_feet(breaks_height)

p1 <- df_mean_years |> 
  ggplot(aes(year, mean)) +
  geom_line() +
  geom_point() + 
  scale_y_continuous(labels = inches_to_feet) +
  scale_x_continuous(breaks = seq(1950, 2020, 10)) +
  labs(
    title = "NBA players' average height has dropped over the last 20 years",
    subtitle = "2023 featured a spike in average height",
    x = "Year",
    y = "Mean Height (feet-inches)",
    caption = "Data: basketball-reference.com. Analysis: Matan Hakim."
  )

p2 <- df_mean_years |> 
  ggplot(aes(year, sd)) +
  geom_line() +
  geom_point() +
  scale_x_continuous(breaks = seq(1950, 2020, 10)) +
  labs(
    title = "Since 1995, NBA players are becoming more similar in their height",
    subtitle = "The early 1990's were the time with most variability in player heights",
    x = "Year",
    y = "Standard Deviation (inches)",
    caption = "Data: basketball-reference.com. Analysis: Matan Hakim."
  )

p3 <- df_heights_decades |> 
  ggplot(aes(ht_tot_inches, pct)) + 
  geom_line() + 
  geom_vline(data = df_mean_decades, aes(xintercept = mean), linetype = "dashed") +
  facet_wrap(~ decade, ncol = 1, strip.position = "right") +
  scale_x_continuous(breaks = breaks_height, labels = labels_height) +
  coord_cartesian(xlim = range(breaks_height)) +
  scale_y_continuous(labels = label_percent()) +
  labs(
    title = str_wrap(
      "Since the 2000's NBA players are becoming shorter and more similar in their height",
      width = 60
    ),
    subtitle = "Average height represented as dashed line",
    x = "Height (feet-inches)",
    y = "% of minutes played",
    caption = "Data: basketball-reference.com. Analysis: Matan Hakim."
  )

p4 <- df_heights_years |> 
  left_join(df_mean_years, join_by(year)) |> 
  ggplot(aes(ht_tot_inches, pct)) +
  geom_line() +
  scale_y_continuous(labels = label_percent()) +
  scale_x_continuous(breaks = breaks_height, labels = labels_height) +
  coord_cartesian(xlim = range(breaks_height)) +
  transition_time(year) +
  geom_vline(
    aes(xintercept = mean),
    linetype = "dashed"
  ) +
  labs(
    title = str_c(
      "Year: {frame_time}, Distribution of NBA Players' Height",
      "\n",
      "(Weighted by Minutes Played)"
    ),
    subtitle = "Average height represented as dashed line",
    x = "Height (feet-inches)",
    y = "% of minutes played",
    caption = "Data: basketball-reference.com. Analysis: Matan Hakim."
  ) +
    shadow_mark(past = FALSE, future = FALSE)

NBA players’ height over the years

If you’ve made it so far, it means you are really into NBA player’s heights! Let’s see how the average player height faired over the years (again, weighted by minutes played).

As we can see, we have 3 different eras for NBA heights. Since the 1950’s until 1987, the average height kept growing, going from just above 6-4 to just above 6-7. Between 1988 and 2004 we can see a small decrease following a small increase in average height, going around (and mostly a bit under) 6-7. Since then, we can see a slow and steady decrease until 2022, where the average NBA player height was 6-6.25, just like in 1979. Interestingly, 2023 featured a spike in average player height, compensating for a 3-years worth of decrease.

It seems that the decrease in average player height might be a part of the “small-ball” revolution, coinciding with an uptick in 3-point shots. Nevertheless, this recent spike in average height might hint towards the future trends and how NBA teams might utilize longer players to disrupt offenses with greater effect.

After looking at average height. let’s look ate the variability of heights. We’ll use the measure of standard deviation. A smaller standard deviation represents a year where heights were closer to the average, while a larger standard deviation represents a year where heights were less closer to the average.

This shows an interesting and different perspective. Since the 1950’s until 1976, height’s standard deviation remained fairly constant. beginning in 1977 and until 1995, height’s standard deviation rose constantly. finally, since 1996, height’s standard deviation was on a downwards trend, pretty much constantly. This shows us that during the last 25 years, NBA players became more and more similar in their heights. This means more players around 6-6 or 6-7, and less really short or really tall players. This might be explained by the increasing value of defensive switchability, outside shooting and all-around basketball skills.

We can combine these two ideas of average (central tendency) and standard deviation (spread) to look at the distribution of NBA heights throughout decades.

Here we can see the aforementioned trends: the dashed line moves to the right and then to the left, representing increasing and decreasing height; the bell-like curve turns from narrower to wider (1950’s-1970’s), remains about the same (1980’s-1990’s) and becoming narrower again (2000’s-present).

Finally, we can visualize these changes with a year-by-year animation, presenting the (again, weighted by minutes played) distribution of NBA player’s height and their average by year.

Conclusion

This blog post has shown the longstanding trends in NBA players’ height. Players are becoming shorter and more similar in their heights, Although we might see a beginning of a trend where higher players are more valuable. We have also seen some techniques in web scraping, data manipulation and visualization. I hope this helped you gain some insights on either of these topics.

Thank you for tagging along!

What is the Problem with Right-to-Left Languages?

As coders and R professionals, the main tool in our toolbox is code. We write R code to solve problems, and this is what sets us apart from other professionals, such as analysts using some type of a GUI program. Nevertheless, code has a very clear postulate - it is written in English, and therefore it is written left-to-right (from now on: LTR, or RTL for “right-to-left”). For the most of us, this is an obvious choice, as English is the lingua franca of the programming world. But for a lot of us, there is a need to use data and communicate results in other languages. This problem becomes much more troublesome when using an RTL language.

More than 2.3 billion people around the world are speaking a language that is written right-to-left.¹ Such languages include, for example, Arabic, Persian, Hebrew, and Fulani. I am myself a native Hebrew speaker, and this is what brought me to write this package. Code is not meant to be written right-to-left, and some problems arise when doing so. Let’s look at an example.

library(ggplot2)
# install.packages("rtlr")
library(rtlr)
theme_update(text = element_text(size = 20))
p <- ggplot(mtcars, aes(factor(cyl))) +
  geom_bar()
p

This is a very basic plot of the mtcars dataset, showing the number of cars with each number of pistons. Now, let’s think about the scenario where we communicate our analysis results to a non-English-speaking audience. Let’s say our audience’s native language is Arabic. Then, we should change the x-axis label:

p +
  labs(
    x = "اسطوانات"
    # `x` in `labs()` says "cylinders" in Arabic.
  )

This is fine, but let’s say we need to add more information to the x-axis label, such as the current year.

p +
  labs(
    x = "اسطوانات (2023)"
    # `x` in `labs()` says "cylinders (2023)" in Arabic.
  )

The code script correctly presents (2023) to the left of the word “اسطوانات” (meaning “pistons”), but our plot malfunctions: it shows (2023) to the right of “اسطوانات”. This is a common problem in RTL languages, as R interpretes all code as written LTR. This problem is also present when ending a sentence with a punctuation mark, such as . or ,.

rtlr - Solves Your RTL Problems

This is where rtlr steps in. Its main (and currently only) function, str_rtl(), wraps strings with the Unicode character \u202B, which tells R to handle everything in that string that comes after this character as RTL.² It does so as a wrapper around stringr::str_c().

p +
  labs(
    x = str_rtl("اسطوانات (2023)")
    # `x` in `labs()` says "cylinders (2023)" in Arabic.
  )

This solved our problem! The year parentheses are located correctly on the left side. A solution to this problem without using rtlr would look something like:

  x = paste0("\u202B", "اسطوانات (2023)")

This solution has two drawbacks: having to remember the exact Unicode character adds to your cognitive load, and when having to print multiple lines with \n you have to paste \u202B again.³ Let’s look at a common use case, where we want to add an informative caption to the plot, this time in Hebrew, but we have too much text for a single line:

p +
  labs(
    x = str_rtl("اسطوانات (2023)"),
    # `x` in `labs()` says "cylinders (2023)" in Arabic.
    caption = "מקור המידע: טבלה מובנית של נתוני מכוניות. עיבוד: מתן חכים. יש להקפיד על הצגת טקסט מימין לשמאל."
    # `caption` in `labs()` says in Hebrew:
    # "Source of data: a built-in data frame of car data. Analysis: Matan Hakim."
    # "You should insist on showing your text right-to-left."
  )

We have two problems here:

1. The caption gets cut off in the middle;

2. The dot in the end of the sentence is placed wrongly on the right side, while it should be on the left.

This can be easily solved after inserting each line as a separate string argument, and using the multiline = TRUE argument:

p +
  labs(
    x = str_rtl("اسطوانات (2023)"),
    # `x` in `labs()` says "cylinders (2023)" in Arabic.
    caption = str_rtl(
      "מקור המידע: טבלה מובנית של נתוני מכוניות. עיבוד: מתן חכים.",
      "יש להקפיד על הצגת טקסט מימין לשמאל.",
      multiline = TRUE
    )
    # `caption` says in Hebrew:
    # "Source of data: a built-in data frame of car data. Analysis: Matan Hakim."
    # "You should insist on showing your text right-to-left."
  )

Notice how the lines are split with no overflow, and how the final dot is placed on the left side, where it should be.

Further use cases might include printing to the console, where str_rtl() also comes in handy:

x <- "הנקודה צריכה להיות בסוף המשפט."
# `x` says in Hebrew:
# "the dot should come at the end of the sentance."

cat(x)

הנקודה צריכה להיות בסוף המשפט.

cat(str_rtl(x))

‫הנקודה צריכה להיות בסוף המשפט.

Again, notice how the dot ends up in the left side, where it should be. Additionally, since this blog post is ultimately rendered to HTML, the first time cat(x) is printed, it is all mixed-up backwards (meaning “tnirp” instead of “print”, but in Hebrew). This problem doesn’t exist when printing to the console, but it does when writing web-published code.

You can learn more on rtlr with printing ?str_rtl, visiting its pkgdown website, or browsing its Github repository.

Conclusion

This blog post has introduced the rtlr package for the R programming language which helps plotting and printing RTL languages correctly. It uses its main function, str_rtl(), to achieve this. rtlr has seen its initial release and its version is v0.1.0 - I welcome suggestions for further use cases or implementations. Feel free to open an issue on Github or email me.

I would like to thank Hadley Wickham and Jennifer Bryan for authoring R Packages (2e). Reading this book has been instrumental in the development of rtlr. Thanks to Hitsu Yonai and Micha Silver for providing helpful feedback for this blog post. Finally, Thanks to Adi Sarid, whose Stack Overflow question from 2018 motivated this package, and helpful comments gave me confidence in this project and his suggestions would be implemented in future versions of rtlr.