☭ comrade

comrade is a Rust crate designed for managing compute work. It allows seamless management of shared work and functions even across machines.

Features

• Parallel Execution: Dispatch tasks to run concurrently and gather their results.
• Rally Execution: Run multiple tasks in parallel and return the result of the fastest one.
• Service Management: Manage background services with different operating modes (Decay, Daemon).
• Worker Unions: Delegate tasks using #[worker] annotations locally or as distributed task queues across multiple machines.
• Background Tasks: Seamlessly run background tasks without blocking the main logic of your program.

Core Concepts

Parallel Execution

comrade provides a simple interface for running tasks in parallel, perfect for independent tasks that can be processed concurrently.

let results: Vec<i32> = parallel(items, |item: &i32| {
    // ...
});

Rally Execution

The rally function allows you to run multiple tasks in parallel and return the result of the first task to finish. This is useful when you want to prioritize the first available result from several tasks (example: download from multiple HTTP mirrors).

let res: (i32, i32) = rally(items, |item: &i32| {
    // ...
});

Background Tasks

Easily run tasks in the background without blocking the main thread. This is useful for code that needs be run without waiting for a result.

fn handle() {
    background(|| {
        // Background task logic
        println!("This is a background task!");
    });
}

Service Management

comrade provides a way to manage persistent services with different modes. The Decay mode allows services to die, while the Daemon mode revives them and keeps them running indefinitely.

use comrade::service::ServiceManager;

fn run_services() {
    let mut manager = ServiceManager::new().mode(comrade::service::ServiceMode::Decay);

    // Register and start services
    manager = manager.register("my_service", |_| {
        // Service logic here
    });

    let thread_handle = manager.spawn();
    thread_handle.join().unwrap();
}

Worker Unions

You can annotate a function with #[worker] which gives them superpowers. These functions can be queued and dispatched by the system, and their results are returned when completed.

use comrade::{worker};

#[worker]
pub fn myfn(i: i32) -> i32 {
    i * 2
}

#[worker]
pub fn multiply(a: i32, b: i32) -> i32 {
    a * b
}

After initialization these functions can then be called anywhere and will be processed eventually by whatever worker picks it up.

fn main() {
    let mut manager = ServiceManager::new().mode(comrade::service::ServiceMode::Decay);

    // Init worker thread on `ServiceManager`
    manager = multiply_init(manager);
    let manager = manager.spawn();

    let res = multiply(2, 2);

    // Shutdown worker thread
    multiply_shutdown();
    manager.join().unwrap();
}

Future Plans: The current system works in-memory with a single worker thread processing tasks. In the future, we plan to extend this to support distributed task queues across multiple machines, enabling shared workloads and infinite scalability.