R & AI (& Testing)

RaukR 2026 • Data Science With R

Sebastian DiLorenzo

18-Aug-2026

What this sessions will cover

• A brief history of Posit data science agents
• Provider setup
• AI types
• Posit Assistant modes
• Some useful commands
• Prompt engineering 101
• Some LLM specific packages in R
• Testing, a toy example

A brief history of Posit data science agents

Positron Assistant

June 2025 (Preview)

A general-purpose coding assistant built into Positron. Aware of the live R/Python session state, it helped write, edit, and debug code — with agent mode for code execution. User needed their own model provider to use it.

Databot

August 2025

A exploratory data analysis agent for Positron. Unlike traditional code assistants, Databot connected directly to active R/Python sessions to inspect variables, data frames, plots, and console history in real time. User needed their own model provider to use it.

Posit Assistant

April 2026 (Preview feature) / July 2026 (General availability)

Combines the strengths of both predecessors into a full-spectrum data science agent. Runs in Positron and RStudio, powered by the Posit AI model service or other model provider. Features skills, plan mode, reasoning, and live session awareness.

How It Works Behind the Scenes Regardless of which LLM is processing your request, Posit Assistant wraps the model in a data-science-specific environment: Context Gathering: Posit Assistant grabs details from your active session (loaded packages, data frame schemas, workspace variables, console logs, and project instructions in AGENTS.md). 2 System Prompting & Skills: It injects domain-specific skills (like Quarto authoring, Shiny development, or R modeling patterns) into the call. 3 Execution: It hands all this off to your selected LLM, gets the instructions back, and performs the requested file edits or code execution.

Databot’s crucial weakness turned out not to be that it was too dangerous, but too narrowly scoped. EDA does not happen in a vacuum. It’s often done in service of some larger project, and insights derived from EDA typically immediately lead to the creation of a reproducible cleaning script, interactive Shiny app, automated data transformation job, etc. As an EDA-focused agent harness, Databot did not have the tools needed to do those tasks well.

More reading: https://opensource.posit.co/blog/2026-06-11_history-of-posit-data-science-agents/

Provider setup

• Enabled by default in positron since July, 2026.
• Enable in settings.json using assistant.enabled: true
• Enable model providers in Configure Language Model Providers.

AI types

Ask

You type a question, the model answers. No memory of previous messages, no back-and-forth. Like a search engine that understands natural language.

Use when: You need a quick explanation, definition, or one-off answer.

Chat

A conversation. The model remembers what was said earlier in the session and can refine answers based on follow-up questions.

Use when: You need to iterate, clarify, or build on previous answers — e.g. debugging a problem step by step.

Agentic

The model doesn’t just answer — it acts. It can write and execute code, read files, call tools, and loop through steps autonomously to complete a goal.

Use when: You want the LLM to carry out a multi-step task, not just describe how to do it.

Ask - early 2022. Even here we had already had autocomplete for a while.

Chat - late 2022, early 2023.

Agentic - late 2023, but reliable was in 2024. Multiple LLMs taking different roles and working as a team.

Generated by gemini: Analogy: > * Ask / Chat Mode is like having an exceptionally smart consultant standing at your desk. You give them a prompt, they hand you a document, and then they wait for you to read it and tell them what to do next.

Agentic Mode is like giving that consultant a goal (“Research our top three competitors, compile a summary table, and email it to the team”), handing them the keys to your tools, and letting them execute, troubleshoot, and finish the job while you do something else.

Posit Assistant modes

Mode	Purpose
Normal	Default mode — safe operations auto-approved, risky ones prompt for confirmation
Plan (/plan)	Read-only exploration and planning before any code changes are made
Auto (/auto)	A classifier reviews tool calls before they run, blocking actions outside the task scope
Cleaning (/clean)	Guided data audit and cleaning, with every judgment call routed to the user for approval

Sandbox (/sandbox): OS-level isolation for bash commands — restricts filesystem writes to the workspace, blocks network access, and protects sensitive paths like ~/.ssh. Off by default.

Normal mode (default)

• Purpose: Balanced day-to-day use. Low-risk operations (reading files, listing directories) are auto-approved; higher-risk ones (editing files, running shell commands) prompt for confirmation.
• Capabilities: Full access to all tools — code execution, file editing, session context, skills.
• Pros: Fast and fluid for most tasks; you only get interrupted when it matters.
• Examples: “Fix this bug”, “Explain this function”, “Build a Shiny app”.
• LIVE: Introduce verbose comments, fix a bug.

Plan mode (/plan)

• Purpose: Think before acting. The assistant explores the codebase using read-only tools, asks clarifying questions, and writes a step-by-step plan — but makes no code changes until you approve the plan.
• Capabilities: File reading, directory listing, grep/search. Writes only to the plan file. No code execution or edits.
• Pros: Great for complex or risky tasks where you want to align on approach first. Avoids surprises.
• Cons: Slower — adds an extra review step before anything happens.
• Examples: “Refactor the data pipeline across multiple files”, “Add a new feature to this package”, “Redesign the app layout”.
• LIVE: Create a function that does something cool.

Auto mode (/auto)

• Purpose: Autonomous operation with a safety net. A classifier model reviews each tool call before it executes, blocking actions that exceed the task scope or target unrecognized infrastructure.
• Capabilities: Same as Normal, but with an automatic classifier gating tool calls instead of user prompts.
• Pros: Hands-free for long-running tasks; the classifier catches scope creep automatically.
• Examples: Long exploratory analysis sessions, batch processing tasks, running in a CI-like context.

Auto mode (/auto) is a classifier-based tool approval mode. When enabled, Posit Assistant automatically decides whether each tool call should be approved or requires your confirmation, based on a built-in classifier that assesses the risk of the action.

It sits between the two extremes:

/yolo — approves all tool calls automatically, no prompts /restricted — requires approval for every tool call

Auto mode is appropriate when you want hands-off assistance for routine tasks (reading files, running code, installing packages) while still getting prompted for higher-risk actions (deleting files, running shell commands that could be destructive, etc.). It’s a good default for most development and analysis work where you trust the assistant but don’t want to rubber-stamp everything blindly.

Cleaning mode (/clean)

• Purpose: Structured, collaborative data cleaning. The assistant audits column names, types, distributions, duplicates, outliers, and data-entry errors — then surfaces every substantive decision to the user before acting.
• Capabilities: Reads data, generates summary statistics, produces visualizations, proposes cleaning steps, and ultimately generates a reusable documented cleaning script.
• Pros: Keeps the analyst in control of all judgment calls; produces reproducible output.
• Cons: Designed specifically for EDA/cleaning — not suitable for general coding tasks.
• Examples: “Clean this survey dataset”, “Check for data quality issues in df”, “Recode these categorical variables”.
• LIVE: library(messy), messy_penguins <- messy::messy(penguins, messiness = 0.2)

Sandbox

• Not a mode itself, but a security layer that can be toggled on top of any mode with /sandbox.
• Uses OS-level enforcement: Seatbelt on macOS, bubblewrap on Linux.
• Restricts: filesystem writes outside the workspace, network access, and reading of sensitive paths (~/.ssh, ~/.aws, ~/.gnupg).
• Use when: running untrusted code, working with sensitive data, or in shared/multi-user environments.

Some useful commands

Command	What it does
/report	Turns your conversation into a tidy Quarto document you can save and share
/notebook	Exports the conversation as a Jupyter notebook
/savememory	Saves information about your project so the assistant remembers it next time
/compact	Squishes the conversation history to save space when it gets too long
/clear	Starts a fresh conversation (keeps your current model settings)
/yolo	Lets the assistant do everything without asking for your OK each time
/restricted	Makes the assistant ask for your approval before doing anything

report and notebook can be handy for example when you have used plan mode and implemented a large change or feature and want to generate a report detailing what has been done.

/savememory Explore the codebase and answer: Understand overall project structure, purpose, key files/scripts, data files/schemas, and how components relate, in order to write a memory file (AGENTS.md) for future sessions.

Prompt engineering 101

• Be specific and clear about what you want
• Avoid ambiguity
• Give examples
• Break down complex tasks into smaller steps
• Follow best coding practices

“Create an image depicting a maestro directing a handful of robots that are building a machine. The maestro should have a blueprint rather than musical notes. The style should be realistic cartoonish, with a white background so it works well to insert on a white slide.”

https://docs.github.com/en/copilot/using-github-copilot/copilot-chat/prompt-engineering-for-copilot-chat

https://www.ibm.com/think/prompt-engineering#1580786329

“Be specific and clear”, this is one part where knowing R is very powerful. Your ability to explain technical concepts and use the language of R can greatly increase the chance that you get desired output. “avoid ambiguity” sounds like being specific and clear but it is more about avoiding vague terms like “fix this” or “make it better” without context. Instead, specify what needs to be fixed or improved and how. “Follow best coding practices”, AI will already do this to a certain extent with variable names, function names, formatting, but for example it may be up to the users prompt to make sure the scope of a requested function is narrow enough.

Bad example: Vague, no context, unclear goal

Good example: Specific, unambiguous, includes an example structure, scoped task, requests best practices

Prompt engineering 101

Bad prompt ❌

“Fix my RNA-seq code”

Good prompt ✅

“I have a data frame counts with genes as rows and samples as columns. Write an R function using DESeq2 to run differential expression between two groups defined by a condition column in coldata. Return a results table sorted by adjusted p-value. Follow tidyverse style conventions.”

Some LLM specific packages in R

ellmer — Chat with LLMs from R. Supports providers such as OpenAI, Anthropic, Google Gemini, and Ollama. Handles tool calling, streaming, and structured data extraction.

shinychat — Add a chat UI component to Shiny apps. Provides a ready-made chat interface that integrates with ellmer or any streaming LLM backend.

kuzco — Computer vision for R. Describe, classify, and extract structured information from images using multimodal LLMs.

Lots of other LLM adjacent packages, a pretty updated list: https://luisdva.github.io/llmsr-book/r-pkgs.html

You are not stuck using whatever is built into your IDE, you can use LLM functions directly from within R with some smart package usage!

Testing

Where: Your R package.

Writing tests might feel like extra work, but it pays off quickly:

• Fewer bugs — Tests catch mistakes early, before they silently corrupt results downstream.
• Better code structure — Testable code tends to be modular and well-defined; writing tests often reveals design problems.

Informal testing

Running code interactively to check that it does what you expect — printing outputs, inspecting objects, trying edge cases in the console. Fast and intuitive, but not repeatable: you have to re-run it manually every time something changes.

Automated testing

Storing tests as code so they can be re-run at any time. If you change a function and a test breaks, you know immediately. This makes refactoring safer and collaboration easier.

Automated testing

Setup

usethis::use_testthat()

1. Creates tests/testthat/
2. Adds testthat to Suggests in DESCRIPTION
3. Creates tests/testthat.R.

Create a test

usethis::use_test("foobar")

Creates and opens tests/testthat/test-foobar.R.

Run tests

Command	What it does
testthat::test_file("tests/testthat/test-foobar.R")	Run a single test file
devtools::test()	Run all tests in tests/testthat/
devtools::check()	Run all tests + full package check

Tip

Best is to use the Testing activity bar!

Parts of a test

R/foobar.R:

foobar <- function(x) {
  if (!is.numeric(x)) {
    stop("Input must be numeric")
  }
  x + 1
}

tests/testthat/test-foobar.R:

test_that("numeric values work", {
  expect_equal(foobar(2), 3)
  expect_equal(foobar(-2), -1)
})
test_that("non-numeric input throws an error", {
  expect_error(foobar("a"), "must be numeric")
})

Expectations

Function	Brief Description
expect_equal()	Equality with numeric tolerance
expect_identical()	Exact identity (strict type check)
expect_true()	Check if expression is TRUE
expect_false()	Check if expression is FALSE
expect_match()	Match character string against a regex
expect_type()	Check underlying type of object
expect_error()	Check if code throws an error
expect_message()	Check if code produces a message
expect_warning()	Check if code produces a warning

• A test file holds one or more test_that() tests.
• Each test describes what it’s testing: e.g. “multiplication works”.
• Each test has one or more expectations: e.g. expect_equal(2 * 2, 4).

Live Coding Example: 1. Short function & test file: - Write a short function add_one (or similar) in the package. - Run usethis::use_test() to create the test file. - Add tests using expect_equal() and expect_error(). 2. Expand the example: - Show/write a function returning a data frame. - Assert that the output data frame is the expected data structure. - Assert that columns have the correct types. 3. Show “Testing” Activity Bar: - Show the “Testing” panel in Positron. - Run the tests using the interactive UI and discuss.

Lab exercises

Start from the libminer package you created yesterday. If you need you can fork and use Jennys.

Use Posit Assistant, PA, to:

1. Ask which tests PA thinks are appropriate for your function. Don’t make edits.
   • Which mode did you use?
   • Do you agree with its suggestions?
2. Make and execute a /plan to add tests for your function. Try to prompt engineer the tests you want.
   • Did you get the output you expected or did you need to make some edits?
3. Save your progress so far in a /report. Afterwards /clear the conversation. Rule of thumb; /clear whenever you think it is a new conversation.
4. Pick an appropriate mode and create a new function for your package.
   • Then add all the other necessary pieces such as documentation, tests, new dependencies etc for your function.
   • At this point, do your tests pass? Does devtools::check() pass?
5. BONUS: Open a new workspace and create a quarto presentation. You can pick the subject to populate the slides with.
   • Can you add appropriate images to the slides? Did it work like you expected?
6. BONUS: Find a messy dataset and import it into R. Try to use /cleaning mode to clean it.

Thank you!

Questions?

2026 • SciLifeLab • NBIS • RaukR