✨ valkey task distribution
This commit is contained in:
parent
e827faaa3f
commit
f381f30e27
7 changed files with 866 additions and 26 deletions
56
README.md
56
README.md
|
@ -33,7 +33,7 @@ let res: (i32, i32) = rally(items, |item: &i32| {
|
|||
|
||||
### Background Tasks
|
||||
|
||||
Easily run tasks in the background without blocking the main thread. This is useful for code that needs be run without waiting for a result.
|
||||
Easily run tasks in the background without blocking the main thread. This is useful for code that needs to be run without waiting for a result.
|
||||
|
||||
```rust
|
||||
fn handle() {
|
||||
|
@ -85,6 +85,7 @@ pub fn multiply(a: i32, b: i32) -> i32 {
|
|||
```
|
||||
|
||||
After initialization these functions can then be called anywhere and will be processed eventually by whatever worker picks it up.
|
||||
Additionally there are new functions derived from your function. See the below example:
|
||||
|
||||
```rust
|
||||
fn main() {
|
||||
|
@ -94,12 +95,63 @@ fn main() {
|
|||
manager = multiply_init(manager);
|
||||
let manager = manager.spawn();
|
||||
|
||||
// works like the original function
|
||||
let res = multiply(2, 2);
|
||||
|
||||
// async
|
||||
let e = take_time_async(1500);
|
||||
|
||||
println!("This will run right after!");
|
||||
// ...
|
||||
|
||||
// is OUR value ready?
|
||||
println!("the value is {}", e.wait());
|
||||
|
||||
// Shutdown worker thread
|
||||
multiply_shutdown();
|
||||
manager.join().unwrap();
|
||||
}
|
||||
```
|
||||
|
||||
**Future Plans**: The current system works in-memory with a single worker thread processing tasks. In the future, we plan to extend this to support distributed task queues across multiple machines, enabling shared workloads and infinite scalability.
|
||||
These tasks can now be distributed with Valkey.
|
||||
|
||||
Make sure you have a Valkey server running and the `$VALKEY_URL` environment variable is set for your application:
|
||||
```yml
|
||||
services:
|
||||
valkey:
|
||||
image: valkey/valkey
|
||||
ports:
|
||||
- 6379:6379
|
||||
```
|
||||
|
||||
Then you can spawn worker threads like that:
|
||||
|
||||
```rust
|
||||
fn main() {
|
||||
let mut s = ServiceManager::new().mode(comrade::service::ServiceMode::Decay);
|
||||
s = multiply_init_union(s);
|
||||
s = myfn_init_union(s);
|
||||
let s = s.spawn();
|
||||
|
||||
log::info!("Spawned workers. Working for 1 minute");
|
||||
std::thread::sleep(Duration::from_secs(60));
|
||||
|
||||
myfn_shutdown();
|
||||
multiply_shutdown();
|
||||
|
||||
s.join().unwrap();
|
||||
}
|
||||
```
|
||||
|
||||
When workers are running, you can use them like:
|
||||
|
||||
```rust
|
||||
fn main() {
|
||||
// Register workers in union
|
||||
myfn_register_union();
|
||||
|
||||
// Will be computed somewhere else
|
||||
let x = myfn(50);
|
||||
println!("x is {x}");
|
||||
}
|
||||
```
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue