Purrrfectly parallel
Purrrfectly distributed

🔗 https://shikokuchuo-posit2025.share.connect.posit.cloud

Charlie Gao (Open Source, Posit)

Parallel purrr

Why now?

Parallel purrr

Why now?



Distributed purrr

Why + how to use

Ecosystem Evolution

Sept 2015

May 2018

Feb 2022




purrr x mirai ?

Ecosystem Evolution

Sept 2015

May 2018

Feb 2022

July 2025

purrr x mirai

Design

Use reliably in programming

mirai - Design Philosophy


⚙️ Modern Foundation

  • NNG | IPC / TCP / secure TLS | x-language data format support e.g. Arrow

⚡️ Extreme Performance

  • Scales to millions of tasks | 1,000x efficiency | zero-latency promises

🚀 Production First

  • Clear evaluation model | minimal complexity | no hidden state

🌐 Deploy Everywhere

  • Local, remote (SSH), cluster (Slurm, SGE, LSF, PBC) | modular compute profiles

mirai

mirai.r-lib.org

ecosystem

distributed purrr

  1. Replace local compute
  2. Extend local compute
  3. Differentiated compute

New in_parallel() adverb


Map:


mtcars |> map(\(x) mean(x))

New in_parallel() adverb


Map in parallel:


mtcars |> map(in_parallel(\(x) mean(x)))

Parallel purrr

Local compute

library(mirai)

daemons(url = local_url()) # 📡️

launch_local(6) # 🛰️

or

daemons(6)

Distributed purrr

Replace local compute

SSH to launch daemons on local network | cloud instance

config <- ssh_config("ssh://hostname") # or IP address

# daemons(url = local_url())
daemons(url = host_url()) # 📡️

#launch_local(6)
launch_remote(6, remote = config) # 🛰️

or

daemons(n = 6, url = host_url(), remote = config)

(host must be able to accept incoming connections)

SSH Tunneling

Avoids opening ports (firewall-compliant)

Connections are local-only, bridged by SSH tunnel

# config <- ssh_config("ssh://hostname")
config <- ssh_config("ssh://hostname", tunnel = TRUE)

# daemons(url = host_url())
daemons(url = local_url(tcp = TRUE)) # 📡️

launch_remote(6, remote = config) # 🛰️

or

daemons(n = 6, url = local_url(tcp = TRUE), remote = config)

Running remotely: 6 daemons

Distributed purrr

Extend local compute

Use both local and remote daemons at same time

config <- ssh_config("ssh://hostname")

daemons(url = host_url())

launch_remote(6, remote = config)

launch_local(6)


Total: 12 daemons

Distributed purrr

Differentiated compute

Driven by requirements e.g. GPU

Set up local + remote compute profiles:

daemons(6) # .compute = "default"

daemons(n = 6, url = host_url(), remote = config, .compute = "gpu")

Use with_daemons() to direct compute:

x |> map(in_parallel(\(x) mypkg::fn(x))) # run locally

with_daemons("gpu", {
  x|> map(in_parallel(\(x) mypkg::fn(x))) # run on remote
})

Distributed purrr

  1. Replace local compute
  2. Extend local compute
  3. Differentiated compute

 

Thank you.