tgrep

Trigram-indexed grep with a client/server architecture for fast regex search in large codebases.

tgrep is integrated into GitHub Copilot CLI to power fast grep searches across large repositories.

Install tgrep, then start a server:

tgrep serve .           # builds the index if needed; runs in the foreground

In another terminal:

tgrep -- "fn main" .    # automatically connects to the server
tgrep status .          # show indexing and refresh status

Without a server, run tgrep index . to build an on-disk index. Searches use the server when available, then the local index, or scan the filesystem if no usable index exists. Indexes are stored in .tgrep/ by default; add this directory to .gitignore.

Indexes can lag filesystem changes. The server updates asynchronously; without a server, rerun tgrep index . after changes. Use --no-index when a search must read the current files. Queries scan during an initial build until complete coverage is available.

For coding agents, see AGENTS.md. To install MCP search tools and startup hooks for Codex or pi, run bash install-agent.sh from this checkout (Linux/macOS, Python 3.11+). See the agent integration guide.

A trigram index narrows each query to files that could match, then verifies those files with the regex engine in parallel. Patterns without useful trigrams may still require searching every indexed file.

In the August 24, 2026 benchmark sweep, tgrep was faster than ripgrep in 17 of 18 repo/platform combinations, with speedups from 0.93x to 51.9x. These measure client/server search latency with the index already built, not indexing time. Results depend on the query, repository, storage, and match volume. See BENCHMARKS.md for measurements and methodology.

tgrep index .                        # index current directory
tgrep index /path/to/repo             # index a specific repo
tgrep index . --index-path /tmp/idx    # custom index location
tgrep index . --exclude vendor --exclude third_party  # skip directories

Each build reports elapsed time and, where available, peak memory:

Index built successfully at /tmp/idx
Indexed in 22.6s using external strategy (peak memory 160.1 MiB)

Windows reports peak private committed bytes; Linux samples anonymous resident memory. Both exclude file-backed mappings and report the working set separately when it is substantially larger. macOS reports resident memory instead.

Like ripgrep, tgrep applies .gitignore only inside a Git repository. It warns when a root .gitignore is present but not applied. Use --no-require-git on indexing, serving, and searching to apply those rules outside Git:

tgrep index . --no-require-git
tgrep serve . --no-require-git
tgrep --no-require-git -- "pattern" .

When Git's core.ignorecase is enabled, tgrep applies an additional case-insensitive exclusion pass for the repository's root .gitignore and .git/info/exclude. Git-tracked paths are exempt from this additional pass, but not from ordinary traversal, binary, or size filtering.

Nested .gitignore files and global ignore rules do not get this additional pass. --no-ignore disables it along with the normal ignore rules.

Use the same --no-require-git, --no-ignore, --max-filesize (or --no-max-filesize), and --exclude settings for index and serve. Startup reconciliation removes indexed files excluded by the server's current settings. For example, serving an uncapped index with --max-filesize 8M removes files larger than 8 MiB from that index.

Pass the same size policy, --no-require-git, and any custom --index-path to searches too. --exclude is only available on index and serve; --no-ignore on a search forces a filesystem scan.

tgrep index . --max-filesize 8M
tgrep serve . --max-filesize 8M
tgrep --max-filesize 8M -- "pattern" .

tgrep index defaults to an external merge sort with a 64 MiB posting buffer. When the buffer fills, sorted segments spill to disk and are merged into the index. This bounds posting-buffer memory, not total process memory: file tables, worker buffers, and decoded content also consume memory.

tgrep index .                          # external, 64 MiB posting buffer
tgrep index . --index-buffer 16        # smaller buffer, lower peak
tgrep index . --index-strategy=memory  # sort entirely in RAM

--index-buffer is in MiB and applies only to the external strategy. --index-strategy=memory avoids temporary spill files but holds all postings in RAM. Both strategies still need a writable index directory.

tgrep serve uses the external builder with the default buffer for new indexes; it does not accept --index-strategy or --index-buffer. Large files can be memory-mapped, but files needing decoding or UTF-8 repair require heap buffers. Use --max-filesize to limit admitted file sizes. See index-build benchmarks.

tgrep serve .                         # auto-build index if missing
tgrep serve . --index-path /tmp/idx   # custom index location
tgrep serve . --watch-mode poll       # poll without native subscriptions
tgrep serve . --poll-interval 60      # polling cadence after fallback
tgrep serve . --watch-budget 4096     # lower this process's native watch ceiling
tgrep serve . --no-watch              # disable all automatic refresh
tgrep serve . --exclude node_modules  # exclude directories from indexing

The server builds the index in the background if none exists and resumes incomplete builds. Clients scan the filesystem until the full corpus is ready; tgrep status reports indexing progress and hidden-file coverage. Legacy indexes are upgraded by startup reconciliation. Multiple clients can connect simultaneously.

On Windows, replaced index generations stay in .retired while readers have them memory-mapped. Cleanup retries after publication, every minute, and on startup; uncommitted backups are preserved for recovery. This does not reclaim storage already stranded in NTFS's $Deleted namespace by older versions.

These tuning options apply only to tgrep serve:

Fresh external builds use the default 64 MiB posting buffer and global Rayon pool, not --max-memory or --max-cpu. If external bootstrap fails, the server falls back to an in-memory build where these settings apply.

The server checks for pending saves once a minute. It saves at the mutation threshold, when filename-only membership changes, or when content changes remain unsaved for at least ten minutes since startup or the last successful save. Active builds and flushes defer this check.

Automatic refresh has two modes, configured on tgrep serve:

In auto mode, a watch-budget or native-registration failure releases this process's watches and switches it to polling until restart. Status reports the reason. Explicit poll mode creates no native subscriptions.

On Linux/Android, the budget counts admitted directories; ignored subtrees do not consume watches. The OS inotify quota is shared with other processes, so registration can fail before this budget is reached. Raising --watch-budget does not raise the OS quota. Windows and macOS use one recursive root subscription and filter ignored events after delivery.

Polling checks filesystem metadata for additions, changes, and deletions. The next poll waits --poll-interval seconds after the previous reconciliation finishes; scan time and ongoing builds/saves add to the delay. Queries do not defer polling. Queue overflows and native rescan notifications also request reconciliation; read-only access events are discarded.

Native mode reconciles about once an hour to recover missed notifications. It waits for a two-minute gap in queries, deferring no longer than four hours. --poll-interval does not change this native safety cadence.

No-change scans do not rewrite the index; changed merges may rewrite it. Metadata-preserving changes can be missed, and reconciliation is not an atomic filesystem snapshot. Failures appear in status. Use --no-index for current file contents.

--no-watch permits the initial build/startup reconciliation but disables later refresh. It conflicts with explicitly supplied --watch-mode, --poll-interval, and --watch-budget. --watch-mode poll also rejects an explicit --watch-budget.

tgrep -- "pattern" .                    # regex search
tgrep -- "pattern" file1.rs file2.rs     # multiple files/paths
tgrep -- "TODO|FIXME" .                 # alternation
tgrep -- '\w+(?!_test)' .               # backtracking-engine fallback
tgrep -i -- "error" .                   # case-insensitive
tgrep -F -- "Vec<T>" .                  # literal string
tgrep -l -- "MyStruct" .                # filenames only
tgrep -c -- "pattern" .                 # matching lines per file
tgrep -m 5 -- "pattern" .               # at most 5 matching lines per file
tgrep -g "*.rs" -g "*.toml" -- "pattern" .  # multiple globs (OR)
tgrep -t rust -C 3 -- "pattern" .       # Rust files, 3 lines of context
tgrep -e "pattern" -e "also_this" .     # multiple patterns (OR)
tgrep --json -- "pattern" .             # JSON stream
tgrep --vimgrep -- "pattern" .          # editor jump targets
tgrep --stats -- "pattern" .            # query plan and timing
tgrep --no-index -- "pattern" .         # read current files, bypass index
tgrep -U -- 'first\nsecond' .           # multiline match
tgrep -q -- "pattern" .                 # exit code only
tgrep --files .                        # list admitted paths, including binary files
tgrep --files -t rust .                # list Rust files
tgrep --type-list                      # show file types

With the default traversal rules, --files reads the live server or the local index instead of walking the repository. The local result is an index snapshot; use --no-index to inspect the filesystem as it exists now. Flags that change traversal membership or the file-size policy also fall back to a walk.

tgrep status .

Server status for /src/my-monorepo
  PID:        37980
  Port:       51043
  Files:      152
  Trigrams:   12265
  Cache:      2/50000
  Watcher:    active
  Watch mode: native (requested: auto)
  Watch budget: 8192
  Poll interval: 120s (after completion)
  Reconcile:  idle
  Last successful reconcile: 2m ago
  Reconcile pending: no
  Reconcile overdue: no
  Last reconcile duration: 42ms
  Indexing:   complete
  Hidden coverage: complete

Watcher refers to native notifications, so it is normally inactive in polling mode. Refresh fields report fallback reasons, errors, pending work, and elapsed deadlines. Older servers may omit these fields.

Indexing: complete and Hidden coverage: complete describe index readiness, not freshness. An existing index can be queried while startup reconciliation is still running. Without a server, status shows on-disk metadata.

tgrep count-files .              # count candidate text files (no server needed)
tgrep count-files /path/to/repo  # scan a specific repo

Counts files admitted by the walk's ignore, visibility, extension, and default size rules, without reading their contents. The reported "text files" can therefore include files containing NUL bytes. Prints the count to stdout and details to stderr:

284957
284957 text files (47516 binary skipped, 0 too large, 0 errors) in 1200ms

These tables describe search flags. Use tgrep index --help and tgrep serve --help for subcommand options.

Pattern matching

--engine auto (the default) uses Rust's regex crate, with fallback to fancy-regex for lookaround and backreferences. -P and --engine pcre2 select fancy-regex, not the PCRE2 library; they do not promise full PCRE2 syntax compatibility.

Output formatting

Encoding

By default, a UTF-8/UTF-16LE/UTF-16BE BOM selects decoding; otherwise files are treated as UTF-8. A BOM overrides a named encoding, but not -E none. Any non-auto encoding bypasses the index and server. Invalid UTF-8 is still repaired, including with -E none; this is not raw-byte matching.

File walking

Accepted for compatibility

--mmap/--no-mmap (tgrep chooses memory mapping automatically), --crlf/--no-crlf (a trailing \r is always stripped), --no-config (tgrep reads no config file), and --colors <SPEC> (colors are not yet configurable) are accepted and ignored so ripgrep command lines keep working. --debug/--trace imply --stats.

For indexed content searches, --stats reports Raw candidates before path, visibility, glob, and type filters, and Candidates after those filters but before size checks, reads, or matching. no index narrowing means the raw set covers the entire nonempty index. (via server) describes transport, not index effectiveness. Older servers may omit candidate statistics.

-z/--search-zip is not supported and exits with code 2 rather than silently reporting no matches in compressed files.

Note: -L means --follow (as in ripgrep). Use the long --files-without-match for the non-matching-files listing.

Without -e/-f, the first positional argument is the pattern and the rest are paths. As soon as -e or -f supplies a pattern, every positional becomes a path, matching ripgrep:

tgrep -e needle .            # searches for "needle" under .
tgrep -e needle -e other .   # both patterns, still just one path

Use -- before a positional pattern that starts with - or is a subcommand name, such as serve or index. Quoting alone does not prevent subcommand parsing. Put all flags before --:

tgrep -F -- serve .

tgrep matches ripgrep's context-dependent defaults rather than fixed ones:

• Line numbers are on only when stdout is a terminal. Piping to another command drops them, so tgrep needle . | cut -d: -f1 behaves the same as it does with ripgrep. --column, --vimgrep and -p turn them back on; -b and -A/-B/-C do not.
• Filenames are shown unless you named exactly one file. A directory argument always shows them. -H/-I override either way.
• Paths are printed by appending onto the argument you typed: the argument survives verbatim and only the appended part uses the platform separator. So tgrep needle src/ prints src/main.rs while tgrep needle src prints src\main.rs on Windows. --path-separator rewrites every separator.

-m/--max-count limits matching lines, not individual matches. All matches on an admitted line are reported. With -U, the unit is a contiguous block of lines covered by matches, so a multiline match is not cut short.

tgrep reads a whole-file buffer even with -m; its bytes_searched statistic can therefore exceed ripgrep's.

-U/--multiline allows matches across line boundaries and prints every covered line. --vimgrep reports one row per match, on its starting line. Unlike ripgrep, tgrep reports columns relative to each printed line rather than repeating the starting column on continuation lines.

A file is treated as binary if its decoded content contains a NUL byte:

• Binary files found by walking a directory are skipped silently — they appear in neither the output, -l, -c, nor --files-without-match.
• A binary file named explicitly on the command line reports a note: bin.dat: binary file matches (found "\0" byte around offset 7).
• --binary promotes traversal to the explicit behaviour, so binary files are searched and summarised with that note.
• -a/--text disables binary detection entirely and prints matches as text.
• --json has no note. As in ripgrep, the matching lines are emitted as ordinary match events and the file's end message carries binary_offset — the offset of the first NUL — so a consumer can still tell a binary hit apart from a text one. stats.bytes_searched stops at that offset rather than counting the whole file.

tgrep also skips known binary extensions during directory content searches and indexing, unlike ripgrep. For searches, --binary and -a lift this restriction and bypass the index; they do not change what index or serve indexes. --files lists binary paths too.

index and serve include hidden, non-ignored files by default. --hidden is accepted on either command but is redundant. Ordinary queries still hide hidden files and directories; -./--hidden includes them using the index or server, for content searches and --files. Visibility is relative to the requested search root and includes Windows hidden attributes. Ignore rules inside hidden directories remain active. The configured index directory, including its staging and retired generations, is always excluded from indexing and query filesystem walks.

Flags that widen or re-interpret the indexed corpus still walk the tree: non-auto -E/--encoding, -a/--text, --binary, and ignore-disabling flags such as --no-ignore. Explicit file arguments also bypass the index.

For content searches, --follow, --one-file-system, --ignore-file, and --ignore-file-case-insensitive do not trigger fallback: indexed searches ignore them. Add --no-index to apply them. --files falls back to walking for these options automatically. index and serve reject these traversal options, --max-depth, and individual --no-ignore-* discovery switches.

Both positive and negative --glob/--iglob patterns filter the indexed corpus when a compatible index or server is available, for content searches and --files. Positive globs such as --glob '*.ts' do not reinclude ignored files in indexed mode. Use --no-index when those matches are needed: filesystem scans retain ripgrep-style glob overrides, which can reinclude ignored files. Negative globs, such as --glob '!.git', exclude entire matching directory subtrees. --hidden never disables ignore rules.

Incomplete indexes and legacy indexes without hidden-file coverage fall back to scanning. Rebuild with tgrep index . or let a current server reconcile them. Older servers without coverage support also cause fallback.

Older binaries reject the new content-index format. To downgrade, rebuild with the older binary, preferably at a separate --index-path.

ripgrep can search raw bytes; tgrep searches decoded text, replacing invalid UTF-8 sequences with U+FFFD. A pattern can match those replacement characters. For UTF-8 repair, text-output columns and byte offsets are mapped back to the source bytes.

JSON output uses lines.text with replacements and submatch offsets into that text. ripgrep instead emits base64 lines.bytes for invalid UTF-8. Transcoded encodings use decoded-text offsets rather than original byte positions; offsets also exclude stripped BOM bytes.

Directory searches and indexing skip files larger than 64 MiB by default; ripgrep has no default limit. These files are absent from the index and its filename listing, so their matches will not appear.

Use --max-filesize to choose another bound, or --no-max-filesize to remove it. Rebuild or serve the index with the same setting; an uncapped query cannot recover paths that a capped index never recorded. Use --no-index --no-max-filesize to scan without either restriction.

A directly named file ignores the inherited cap, but respects an explicit --max-filesize: tgrep -- pattern ./huge.log searches that file regardless of its size.

Same as ripgrep:

A match plus an error yields 2, unless -q is set, which yields 0.

An ignore file that fails to parse is not one of these errors. Like ripgrep, tgrep reports it on stderr, skips the offending rule and carries on, leaving the exit code determined by the search alone. Suppress the message with --no-ignore-messages, or with --no-messages, which covers it as well.

1. Indexing walks the repository with ignore rules, including root-level p4ignore.ini, and filters by size and binary extension. It decodes files, checks the first 8 KiB for NUL bytes, and extracts overlapping three-byte trigrams in parallel. Sorted postings map trigrams to candidate files.
2. Querying decomposes regex literals into trigram lookups, intersects or unions posting lists, and verifies candidates with the full regex engine. Inline case flags participate in planning; unsupported constructs use conservative plans rather than excluding possible matches.
3. Serving combines a memory-mapped IndexReader with a mutable LiveIndex overlay in HybridIndex. Updates take precedence over disk entries. Clients use JSON-RPC 2.0 over newline-delimited TCP on loopback, with one thread per connection. Shared read locks allow concurrent queries but can contend with writers.

The server's content cache is limited to 50,000 entries and 1 GiB of decoded content, with a 64 MiB per-entry limit. --files combines content-index paths with a filename-only sidecar for admitted paths without searchable content.

cargo build --release --locked  # build optimized binary
cargo test --workspace          # run tests
make check                      # check formatting and run clippy
make install                    # install to ~/.cargo/bin (Unix)

git clone https://github.com/microsoft/tgrep.git
cd tgrep
cargo install --path tgrep-cli --locked

brew install tgrep

Download from GitHub Releases for Linux, macOS (Intel & Apple Silicon), and Windows.

Download the archive for your architecture, extract it to a temporary directory, then copy the executable into a directory on PATH.

# Linux (x86_64)
tmpdir="$(mktemp -d)"
gh release download --repo microsoft/tgrep -p '*x86_64-unknown-linux-musl.tar.gz' -D "$tmpdir"
tar xzf "$tmpdir"/tgrep-*-x86_64-unknown-linux-musl.tar.gz -C "$tmpdir"
install -Dm755 "$tmpdir/tgrep" "$HOME/.local/bin/tgrep"
rm -rf "$tmpdir"

# macOS (Apple Silicon)
# For Intel, replace aarch64 with x86_64.
tmpdir="$(mktemp -d)"
gh release download --repo microsoft/tgrep -p '*aarch64-apple-darwin.tar.gz' -D "$tmpdir"
tar xzf "$tmpdir"/tgrep-*-aarch64-apple-darwin.tar.gz -C "$tmpdir"
mkdir -p "$HOME/.local/bin"
install -m755 "$tmpdir/tgrep" "$HOME/.local/bin/tgrep"
rm -rf "$tmpdir"

# Windows x64 (PowerShell); for ARM64, replace x86_64 with aarch64.
$download = Join-Path $env:TEMP ("tgrep-dl-" + [guid]::NewGuid())
New-Item -ItemType Directory -Path $download | Out-Null
gh release download --repo microsoft/tgrep -p '*x86_64-pc-windows-msvc.zip' -D $download
Get-ChildItem $download -Filter '*.zip' | ForEach-Object {
    Expand-Archive -LiteralPath $_.FullName -DestinationPath "$HOME\.cargo\bin" -Force
}
Remove-Item -LiteralPath $download -Recurse

Ensure $HOME/.local/bin (Unix) or $HOME\.cargo\bin (Windows) is on PATH.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

MIT