📜 Website Archive
- Introduce URI graph data model and layered indexer pipeline - Create indexers for blob metadata, header metadata, graph, text, and documents - Implement new API endpoints for graph queries, full-text search, and header metadata - Add migrations for new tables - Refactor code to support streaming HTTP serving and chunk-based blob processing |
||
|---|---|---|
| .woodpecker | ||
| docs | ||
| migrations | ||
| src | ||
| .dockerignore | ||
| .gitignore | ||
| build.sh | ||
| Cargo.lock | ||
| Cargo.toml | ||
| cog.toml | ||
| config.yml | ||
| devenv.lock | ||
| devenv.nix | ||
| docker-compose.yml | ||
| flake.lock | ||
| flake.nix | ||
| README.md | ||
| renovate.json | ||
| zensical.toml | ||
WebArc
webarc is a local-first web archiving system designed to capture, preserve, and replay HTTP content in an extensible way.
Unlike traditional crawlers that fetch pages in isolation, WebArc focuses on recording real HTTP traffic, storing it in an archive, and making that archive usable through multiple interfaces.
See the documentation.