WebArc

webarc is a local-first web archiving system designed to capture, preserve, and replay HTTP content in an extensible way.

Unlike traditional crawlers that fetch pages in isolation, WebArc focuses on recording real HTTP traffic, storing it in an archive, and making that archive usable through multiple interfaces.

See the documentation.