rush

Package website: release | dev

rush is an R package for asynchronous and decentralized optimization. It uses a database-centric architecture in which workers communicate through a shared Redis database. To support high-throughput workloads, rush combines sub-millisecond per-task overhead with caching strategies that reduce database operations. The package integrates with the mlr3 ecosystem and serves as the backend for asynchronous optimization algorithms in bbotk and mlr3tuning.

Features

• Database-centric architecture for decentralized optimization without a central controller.
• Asynchronous communication between workers using Redis.
• Sub-millisecond per-task overhead for high-throughput workloads.
• Efficient caching mechanism that minimizes database read/write operations.
• Task queue support for centralized tasks distribution when needed.
• Fast data transformation from Redis to data.table.
• Scales to large remote worker networks via the mirai package.
• Worker logging directly into the Redis database using lgr.
• Minimal dependencies for lightweight integration.

Install

Install the latest release from CRAN.

install.packages("rush")

Install the development version from GitHub.

pak::pak("mlr-org/rush")

And install Redis.

Test with Redis

To test the package, set the RUSH_TEST_USE_REDIS environment variable to true. The test suite deletes the Redis database before execution, so never run it against a production server.

Sys.setenv(RUSH_TEST_USE_REDIS = "true")

Related Work

• future
• mirai
• batchtools
• crew
• rrq