✨ worker
This commit is contained in:
parent
5cb4facc48
commit
e3393f1e09
11 changed files with 841 additions and 48 deletions
103
README.md
Normal file
103
README.md
Normal file
|
@ -0,0 +1,103 @@
|
|||
# ☭ comrade
|
||||
`comrade` is a Rust crate designed for managing compute work. It allows seamless management of shared work and functions even across machines.
|
||||
|
||||
## Features
|
||||
|
||||
- **Parallel Execution**: Dispatch tasks to run concurrently and gather their results.
|
||||
- **Rally Execution**: Run multiple tasks in parallel and return the result of the fastest one.
|
||||
- **Service Management**: Manage background services with different operating modes (`Decay`, `Daemon`).
|
||||
- **Worker Unions**: Delegate tasks using `#[worker]` annotations locally or as distributed task queues across multiple machines.
|
||||
- **Background Tasks**: Seamlessly run background tasks without blocking the main logic of your program.
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Parallel Execution
|
||||
|
||||
`comrade` provides a simple interface for running tasks in parallel, perfect for independent tasks that can be processed concurrently.
|
||||
|
||||
```rust
|
||||
let results: Vec<i32> = parallel(items, |item: &i32| {
|
||||
// ...
|
||||
});
|
||||
```
|
||||
|
||||
### Rally Execution
|
||||
|
||||
The `rally` function allows you to run multiple tasks in parallel and return the result of the **first task to finish**. This is useful when you want to prioritize the first available result from several tasks (example: download from multiple HTTP mirrors).
|
||||
|
||||
```rust
|
||||
let res: (i32, i32) = rally(items, |item: &i32| {
|
||||
// ...
|
||||
});
|
||||
```
|
||||
|
||||
### Background Tasks
|
||||
|
||||
Easily run tasks in the background without blocking the main thread. This is useful for code that needs be run without waiting for a result.
|
||||
|
||||
```rust
|
||||
fn handle() {
|
||||
background(|| {
|
||||
// Background task logic
|
||||
println!("This is a background task!");
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
### Service Management
|
||||
|
||||
`comrade` provides a way to manage persistent services with different modes. The `Decay` mode allows services to die, while the `Daemon` mode revives them and keeps them running indefinitely.
|
||||
|
||||
```rust
|
||||
use comrade::service::ServiceManager;
|
||||
|
||||
fn run_services() {
|
||||
let mut manager = ServiceManager::new().mode(comrade::service::ServiceMode::Decay);
|
||||
|
||||
// Register and start services
|
||||
manager = manager.register("my_service", |_| {
|
||||
// Service logic here
|
||||
});
|
||||
|
||||
let thread_handle = manager.spawn();
|
||||
thread_handle.join().unwrap();
|
||||
}
|
||||
```
|
||||
|
||||
### Worker Unions
|
||||
|
||||
You can annotate a function with `#[worker]` which gives them superpowers. These functions can be queued and dispatched by the system, and their results are returned when completed.
|
||||
|
||||
```rust
|
||||
use comrade::{worker};
|
||||
|
||||
#[worker]
|
||||
pub fn myfn(i: i32) -> i32 {
|
||||
i * 2
|
||||
}
|
||||
|
||||
#[worker]
|
||||
pub fn multiply(a: i32, b: i32) -> i32 {
|
||||
a * b
|
||||
}
|
||||
```
|
||||
|
||||
After initialization these functions can then be called anywhere and will be processed eventually by whatever worker picks it up.
|
||||
|
||||
```rust
|
||||
fn main() {
|
||||
let mut manager = ServiceManager::new().mode(comrade::service::ServiceMode::Decay);
|
||||
|
||||
// Init worker thread on `ServiceManager`
|
||||
manager = multiply_init(manager);
|
||||
let manager = manager.spawn();
|
||||
|
||||
let res = multiply(2, 2);
|
||||
|
||||
// Shutdown worker thread
|
||||
multiply_shutdown();
|
||||
manager.join().unwrap();
|
||||
}
|
||||
```
|
||||
|
||||
**Future Plans**: The current system works in-memory with a single worker thread processing tasks. In the future, we plan to extend this to support distributed task queues across multiple machines, enabling shared workloads and infinite scalability.
|
Loading…
Add table
Add a link
Reference in a new issue