103 lines
3.2 KiB
Markdown
103 lines
3.2 KiB
Markdown
# ☭ comrade
|
|
`comrade` is a Rust crate designed for managing compute work. It allows seamless management of shared work and functions even across machines.
|
|
|
|
## Features
|
|
|
|
- **Parallel Execution**: Dispatch tasks to run concurrently and gather their results.
|
|
- **Rally Execution**: Run multiple tasks in parallel and return the result of the fastest one.
|
|
- **Service Management**: Manage background services with different operating modes (`Decay`, `Daemon`).
|
|
- **Worker Unions**: Delegate tasks using `#[worker]` annotations locally or as distributed task queues across multiple machines.
|
|
- **Background Tasks**: Seamlessly run background tasks without blocking the main logic of your program.
|
|
|
|
## Core Concepts
|
|
|
|
### Parallel Execution
|
|
|
|
`comrade` provides a simple interface for running tasks in parallel, perfect for independent tasks that can be processed concurrently.
|
|
|
|
```rust
|
|
let results: Vec<i32> = parallel(items, |item: &i32| {
|
|
// ...
|
|
});
|
|
```
|
|
|
|
### Rally Execution
|
|
|
|
The `rally` function allows you to run multiple tasks in parallel and return the result of the **first task to finish**. This is useful when you want to prioritize the first available result from several tasks (example: download from multiple HTTP mirrors).
|
|
|
|
```rust
|
|
let res: (i32, i32) = rally(items, |item: &i32| {
|
|
// ...
|
|
});
|
|
```
|
|
|
|
### Background Tasks
|
|
|
|
Easily run tasks in the background without blocking the main thread. This is useful for code that needs be run without waiting for a result.
|
|
|
|
```rust
|
|
fn handle() {
|
|
background(|| {
|
|
// Background task logic
|
|
println!("This is a background task!");
|
|
});
|
|
}
|
|
```
|
|
|
|
### Service Management
|
|
|
|
`comrade` provides a way to manage persistent services with different modes. The `Decay` mode allows services to die, while the `Daemon` mode revives them and keeps them running indefinitely.
|
|
|
|
```rust
|
|
use comrade::service::ServiceManager;
|
|
|
|
fn run_services() {
|
|
let mut manager = ServiceManager::new().mode(comrade::service::ServiceMode::Decay);
|
|
|
|
// Register and start services
|
|
manager = manager.register("my_service", |_| {
|
|
// Service logic here
|
|
});
|
|
|
|
let thread_handle = manager.spawn();
|
|
thread_handle.join().unwrap();
|
|
}
|
|
```
|
|
|
|
### Worker Unions
|
|
|
|
You can annotate a function with `#[worker]` which gives them superpowers. These functions can be queued and dispatched by the system, and their results are returned when completed.
|
|
|
|
```rust
|
|
use comrade::{worker};
|
|
|
|
#[worker]
|
|
pub fn myfn(i: i32) -> i32 {
|
|
i * 2
|
|
}
|
|
|
|
#[worker]
|
|
pub fn multiply(a: i32, b: i32) -> i32 {
|
|
a * b
|
|
}
|
|
```
|
|
|
|
After initialization these functions can then be called anywhere and will be processed eventually by whatever worker picks it up.
|
|
|
|
```rust
|
|
fn main() {
|
|
let mut manager = ServiceManager::new().mode(comrade::service::ServiceMode::Decay);
|
|
|
|
// Init worker thread on `ServiceManager`
|
|
manager = multiply_init(manager);
|
|
let manager = manager.spawn();
|
|
|
|
let res = multiply(2, 2);
|
|
|
|
// Shutdown worker thread
|
|
multiply_shutdown();
|
|
manager.join().unwrap();
|
|
}
|
|
```
|
|
|
|
**Future Plans**: The current system works in-memory with a single worker thread processing tasks. In the future, we plan to extend this to support distributed task queues across multiple machines, enabling shared workloads and infinite scalability.
|