diff --git a/technology/applications/cli/gron.md b/technology/applications/cli/gron.md new file mode 100644 index 0000000..e01a080 --- /dev/null +++ b/technology/applications/cli/gron.md @@ -0,0 +1,51 @@ +--- +obj: application +repo: https://github.com/tomnomnom/gron +--- + +# gron +gron transforms [JSON](../../files/JSON.md) into discrete assignments to make it easier to grep, [ripgrep](ripgrep.md) for what you want and see the absolute 'path' to it. + +## Usage +Transform [JSON](../../files/JSON.md) (from a file, URL, or stdin) into discrete assignments to make it greppable +Usage: `gron [OPTIONS] [FILE|URL|-]` + +### Options +| Option | Description | +| ------------------ | ------------------------------------------------------------------------------ | +| `-u, --ungron` | Reverse the operation (turn assignments back into [JSON](../../files/JSON.md)) | +| `-v, --values` | Print just the values of provided assignments | +| `-c, --colorize` | Colorize output (default on tty) | +| `-m, --monochrome` | Monochrome (don't colorize output) | +| `-s, --stream` | Treat each line of input as a separate [JSON](../../files/JSON.md) object | +| `-k, --insecure` | Disable certificate validation | +| `-j, --json` | Represent gron data as [JSON](../../files/JSON.md) stream | +| `--no-sort` | Don't sort output (faster) | + +## Example +Gron turns the following json file: +```json +{ + "key": "value", + "num": 42, + "b": true, + "obj": { + "sub": "key" + }, + "lst": [1,2,3] +} +``` + +into this: +``` +json = {}; +json.b = true; +json.key = "value"; +json.lst = []; +json.lst[0] = 1; +json.lst[1] = 2; +json.lst[2] = 3; +json.num = 42; +json.obj = {}; +json.obj.sub = "key"; +``` \ No newline at end of file diff --git a/technology/applications/cli/ripgrep-all.md b/technology/applications/cli/ripgrep-all.md new file mode 100644 index 0000000..d4e79f3 --- /dev/null +++ b/technology/applications/cli/ripgrep-all.md @@ -0,0 +1,119 @@ +--- +obj: application +repo: https://github.com/phiresky/ripgrep-all +--- + +# ripgrep-all +rga is a line-oriented search tool that allows you to look for a [regex](../../tools/Regex.md) in a multitude of file types. rga wraps the awesome [ripgrep](ripgrep.md) and enables it to search in [pdf](../../files/PDF.md), docx, [sqlite](../../dev/programming/SQLite.md), jpg, movie subtitles ([mkv](../../files/media/Matroska.md), mp4), etc. + +## USAGE: + +> rga \[RGA OPTIONS\] \[RG OPTIONS\] PATTERN \[PATH \...\] + +## FLAGS: + +| Option | Description | +| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| --rga-accurate | Use more accurate but slower matching by [mime](../../files/MIME.md) type
By default, rga will match files using file extensions. Some programs,
such as sqlite3, don't care about the file extension at all, so users
sometimes use any or no extension at all. With this flag, rga will try
to detect the [mime](../../files/MIME.md) type of input files using the magic bytes (similar
to the [`file`](system/file.md) utility), and use that to choose the adapter. Detection is
only done on the first 8KiB of the file, since we can't always seek on the input (in archives). | +| --rga-no-cache | Disable caching of results
By default, rga caches the extracted text, if it is small enough, to a
database in ${XDG_CACHE_DIR-~/.cache}/ripgrep-all on Linux,
~Library/Caches/ripgrep-all on macOS, or C:\\Users\\username\\AppData\\Local\\ripgrep-all on Windows. This way,
repeated searches on the same set of files will be much faster. If you
pass this flag, all caching will be disabled. | +| --rga-list-adapters | List all known adapters | +| --rga-print-config-schema | Print the [JSON Schema](../../tools/JSON%20Schema.md) of the configuration file | +| --rg-help | Show help for [ripgrep](ripgrep.md) itself | +| --rg-version | Show version of [ripgrep](ripgrep.md) itself | +| -V, --version | Prints version information | + +## OPTIONS: + +| Option | Description | +| ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| --rga-adapters=\... | Change which adapters to use and in which priority order (descending)
"foo,bar" means use only adapters foo and bar. "-bar,baz" means use all default adapters except for bar and baz. "+bar,baz" means use all default adapters and also bar and baz. | +| --rga-cache-compression-level=\ | ZSTD compression level to apply to adapter outputs before storing in
cache db
Ranges from 1 - 22 \[default: 12\] | +| --rga-config-file=\ | | +| --rga-max-archive-recursion=\ | Maximum nestedness of archives to recurse into \[default: 5\] | +| --rga-cache-max-blob-len=\ | Max compressed size to cache
Longest byte length (after compression) to store in cache. Longer
adapter outputs will not be cached and recomputed every time.
Allowed suffixes on command line: k M G \[default: 2000000\] | +| --rga-cache-path=\ | Path to store cache db \[default: /home/user/.cache/ripgrep-all\] | +| -h | Shows a concise overview | +| --help | Shows more detail and advanced options | + + +## Available Adapters +rga works with _adapters_ that adapt various [file formats](../../files/File%20Formats.md). It comes with a few adapters integrated: + +``` +rga --rga-list-adapters +``` + +Adapters: +- **pandoc** + Uses pandoc to convert binary/unreadable text documents to plain [markdown](../../files/Markdown.md)-like text + Runs: pandoc --from= --to=plain --wrap=none --markdown-headings=atx + Extensions: .epub, .odt, .docx, .fb2, .ipynb, .html, .htm + +- **poppler** + Uses pdftotext (from poppler-utils) to extract plain text from [PDF](../../files/PDF.md) files + Runs: pdftotext - - + Extensions: .pdf + Mime Types: application/pdf + +- **postprocpagebreaks** + Adds the page number to each line for an input file that specifies page breaks as ascii page break character. + Mainly to be used internally by the poppler adapter. + Extensions: .asciipagebreaks + +- **ffmpeg** + Uses [ffmpeg](../media/ffmpeg.md) to extract video metadata/chapters, subtitles, lyrics, and other metadata + Extensions: .mkv, .mp4, .avi, .mp3, .ogg, .flac, .webm + +- **zip** + Reads a [zip](../../files/ZIP.md) file as a stream and recurses down into its contents + Extensions: .zip, .jar + Mime Types: application/zip + +- **decompress** + Reads compressed file as a stream and runs a different extractor on the contents. + Extensions: .als, .bz2, .gz, .tbz, .tbz2, .tgz, .xz, .zst + Mime Types: application/gzip, application/x-bzip, application/x-xz, application/zstd + +- **tar** + Reads a [tar](compression/tar.md) file as a stream and recurses down into its contents + Extensions: .tar + +- **sqlite** + Uses [sqlite](../../dev/programming/SQLite.md) bindings to convert [sqlite](../../dev/programming/SQLite.md) databases into a simple plain text format + Extensions: .db, .db3, .sqlite, .sqlite3 + Mime Types: application/x-sqlite3 + +The following adapters are disabled by default, and can be enabled using '--rga-adapters=+foo,bar': + +- **mail** + Reads mailbox/mail files and runs extractors on the contents and attachments. + Extensions: .mbox, .mbx, .eml + Mime Types: application/mbox, message/rfc822 + +## Configuration +In addition to the command-line flags, you can configure rga via the config file. The config file is located in `~/.config/ripgrep-all/config.jsonc` or your OS-equivalent. The file is in the json-with-comments format. + +### Custom adapters +Since version 1.0, you can specify custom adapters that invoke external preprocessing scripts in the config file. + +See [Community Adapters](https://github.com/phiresky/ripgrep-all/discussions/categories/show-your-adapter) for a list of adapters that other people are using. + +For example, the integrated PDF-to-text adapter would look like the following in the config file: +```json +"custom_adapters": [ + { + "name": "poppler", + "version": 1, + "description": "Uses pdftotext (from poppler-utils) to extract plain text from PDF files", + + "extensions": ["pdf"], + "mimetypes": ["application/pdf"], + + "binary": "pdftotext", + "args": ["-", "-"], + "disabled_by_default": false, + "match_only_by_mime": false, + "output_path_hint": "${input_virtual_path}.txt.asciipagebreaks" + } +] +```