Get Started with localLLM

localLLM provides an easy-to-use interface to run large language models (LLMs) directly in R. It uses the performant llama.cpp library as the backend and allows you to generate text and analyze data with LLMs. Everything runs locally on your own machine, completely free, with reproducibility by default.

Installation

Getting started requires two simple steps: installing the R package and downloading the backend C++ library.

Step 1: Install the R package

For current R releases, the standard CRAN installation is recommended:

install.packages("localLLM")
packageVersion("localLLM")

If CRAN gives you an older version

CRAN builds pre-compiled (binary) packages only for the current R release and the previous one. On an older R, install.packages() therefore installs the last binary that was built for that R version, which can be older than the current localLLM release. For example, on R 4.4, CRAN’s Windows and Apple Silicon binaries stopped at localLLM 1.2.1, so install.packages("localLLM") could not install 1.3.0 there.

If packageVersion("localLLM") is older than the version shown on CRAN, either update R (recommended), or install the current version from source:

options(download.file.method = "libcurl")
install.packages("localLLM", type = "source")
packageVersion("localLLM")

Installing from source compiles the package’s C++ code, so you need a compiler toolchain: Rtools on Windows, the Xcode Command Line Tools on macOS (xcode-select --install in the Terminal), or the usual build tools (build-essential or similar) on Linux.

From GitHub

To get a version that is not on CRAN yet, install the package from GitHub. The R package lives in the localLLM subdirectory of the repository, so pass subdir:

# install.packages("remotes")

# A specific release (recommended), e.g. v1.4.1
remotes::install_github("EddieYang211/localLLM@v1.4.1", subdir = "localLLM")

# The latest development version
remotes::install_github("EddieYang211/localLLM", subdir = "localLLM")

This also compiles from source and needs the same toolchain as above. Each package version downloads its matching backend in Step 2. If install_localLLM() cannot find the backend for a development version, the backend for that version has not been published yet; install a release tag instead.

A specific older version

The package version affects the output: a newer localLLM or backend can change results for the same model, prompt and seed. To reproduce an analysis that used an older version, install that exact version:

remotes::install_version("localLLM", version = "1.3.0")  # from the CRAN archive

Then run Step 2, which installs the backend matching that version. See the Reproducible Output vignette for details.

Step 2: Install the backend library

The install_localLLM() function installs the separate backend library used by the R package. It does not upgrade the R package itself. The function automatically detects your platform and downloads the appropriate pre-compiled library. GPU acceleration is selected automatically when a compatible GPU driver is detected:

Platform	GPU backend	Detection method
macOS (Apple Silicon)	Metal	always enabled
macOS (Intel)	Metal	always enabled
Windows (x86-64)	Vulkan	`vulkan-1.dll` present in System32
Linux (x86-64)	Vulkan	Vulkan loader + hardware ICD file present

On Windows and Linux, if no GPU driver is found, the CPU build is installed automatically.

library(localLLM)
install_localLLM()

# Force CPU build even when a GPU is detected
install_localLLM(force_cpu = TRUE)

# Reinstall after adding a GPU driver (re-runs detection)
install_localLLM(force_reinstall = TRUE)

Your First LLM Query

The simplest way to get started is with quick_llama():

library(localLLM)

response <- quick_llama("What is the capital of France?")
cat(response)

#> The capital of France is Paris.

quick_llama() is a high-level wrapper designed for convenience. On first run, it automatically downloads and caches the default model (Llama-3.2-3B-Instruct-Q5_K_M.gguf).

Text Classification Example

A common use case is classifying text. Here’s a sentiment analysis example:

response <- quick_llama(
  'Classify the sentiment of the following tweet into one of two
   categories: Positive or Negative.

   Tweet: "This paper is amazing! I really like it."'
)

cat(response)

#> The sentiment of this tweet is Positive.

Processing Multiple Prompts

quick_llama() can handle different types of input:

Single string: Performs a single generation
Vector of strings: Automatically switches to parallel generation mode

# Process multiple prompts at once
prompts <- c(

  "What is 2 + 2?",
  "Name one planet in our solar system.",
  "What color is the sky?"
)

responses <- quick_llama(prompts)
print(responses)

#> [1] "2 + 2 equals 4."
#> [2] "One planet in our solar system is Mars."
#> [3] "The sky is typically blue during the day."

Finding and Using Models

GGUF Format

The localLLM backend only supports models in the GGUF format. You can find thousands of GGUF models on Hugging Face:

Search for “gguf” on Hugging Face
Filter by model family (e.g., “gemma gguf”, “llama gguf”)
Copy the direct URL to the .gguf file

Loading Different Models

# From Hugging Face URL
response <- quick_llama(
  "Explain quantum physics simply",
  model_path = "https://huggingface.co/unsloth/gemma-3-4b-it-qat-GGUF/resolve/main/gemma-3-4b-it-qat-Q5_K_M.gguf"
)

# From local file
response <- quick_llama(
  "Explain quantum physics simply",
  model_path = "/path/to/your/model.gguf"
)

# From cache (name fragment)
response <- quick_llama(
  "Explain quantum physics simply",
  model_path = "Llama-3.2"
)

Managing Cached Models

# List all cached models
cached <- list_cached_models()
print(cached)

#>                                 name size_bytes            modified
#> 1 Llama-3.2-3B-Instruct-Q5_K_M.gguf 2322153920 2025-12-05 20:01:18
#> 2   gemma-3-4b-it-qat-Q5_K_M.gguf   2829698176 2025-12-14 19:21:11

# Delete a cached model by name
file.remove(cached$path[cached$name == "Llama-3.2-3B-Instruct-Q5_K_M.gguf"])