DskDitto: Ultra-fast, parallel duplicate-file detector
dskDitto: High-Performance Parallel Duplicate Finder
dskDitto is an exceptionally rapid, parallelized utility designed to locate duplicate files. It features interactive menus that transform the tedious process of cleaning up redundant data into a seamless experience.
🚀 Core Capabilities
The tool is engineered for speed and flexibility. Here is a breakdown of its primary features:
- Extreme Velocity: Utilizes parallel processing to scan massive drives almost instantly.
- Dual Interfaces:
- Default: A sophisticated Terminal User Interface (TUI) built with
Bubble Tea. - Experimental: A graphical user interface (GUI) powered by
Raylib.
- Default: A sophisticated Terminal User Interface (TUI) built with
- Risk-Averse Management: Safely purge duplicates or swap them for symlinks via confirmation prompts.
- Targeted Search: Hash a single file to find every identical copy across your entire system.
- Cryptographic Flexibility: Supports both
SHA-256(standard) andBLAKE3for content verification. - Advanced Filtering:
- Filter by file size or directory depth.
- Ignore hidden files, symlinks, or virtual filesystems.
- Respect filesystem boundaries.
- Data Export: Generate reports in
CSV,JSON, or plain text formats. - Hard-link Intelligence: Recognizes Unix hard-links to prevent false-positive duplicate reports.
Logic Workflow
🛠 Installation
To install the tool, you will need Go 1.22+.
Standard CLI/TUI Version:
go install github.com/jdefrancesco/dskDitto/cmd/dskDitto@latest
The binary will be located at $(go env GOPATH)/bin/dskDitto (usually ~/go/bin).
GUI-Enabled Version:
Ensure you have CGo and Raylib installed on your system, then run:
go install -tags gui github.com/jdefrancesco/dskDitto/cmd/dskDitto@latest
📖 Usage & Configuration
Run the tool using the following syntax:
dskDitto [options] PATH ...
Command Line Flags
| Flag | Description |
|---|---|
--version | Display version and exit. |
--no-banner | Suppress the initial startup greeting. |
--gui | Launch the experimental Raylib GUI (requires GUI build). |
--profile file | Generate a CPU profile and save to file. |
--time-only | Scan and exit immediately, showing only the duration. |
--min-size bytes | Ignore files smaller than this value. |
--max-size bytes | Ignore files larger than this (Default: 4 GiB; 0 for unlimited). |
--all-sizes | Shortcut for --max-size 0. |
--hidden | Include hidden files and directories. |
--exclude path | Skip specific paths and their children (can be used multiple times). |
--no-symlinks | Do not follow or scan symbolic links. |
--empty | Include files with 0 bytes. |
--include-vfs | Scan virtual filesystems (e.g., /dev, /proc). |
--one-file-system / --xdev | Stay within a single filesystem device. |
--dir-concurrency int | Set manual limit for directory reads (0 = auto). |
--no-cache | Prevent the OS from caching files during hashing. |
--current | No recursion; scan only the provided paths. |
--depth levels | Limit how deep the recursion goes. |
--dups count | Only display groups with at least count files. |
--text / --bullet | Output results as text without opening the TUI. |
--remove keep | Keep keep files per group and delete the rest. |
--link | Used with --remove; replaces duplicates with symlinks. |
--file path | Find duplicates of one specific file. |
--name-only | Group by filename only (ignores size/content). |
--file-shallow path | Find files with the same name as path (shallow mode). |
--fuzzy | Use similarity detection rather than exact matches. |
--fuzzy-threshold pct | Minimum similarity % for fuzzy mode (Default: ). |
--fuzzy-same-ext | In fuzzy mode, only compare files with matching extensions. |
--hash algo | Choose sha256 or blake3. |
--csv-out file | Export results to a CSV file. |
--json-out file | Export results to a JSON file. |
--fs-detect path | Identify the filesystem type of the given path. |
--color-safe | Use a basic TUI theme for compatibility. |
--no-confirm | Bypass confirmation prompts for deletion/linking. |
Note: You can terminate any active scan by pressing Ctrl+C.
🛡 Safety and File Manipulation
The philosophy of dskDitto is safety first. It will never delete your files automatically.
Operational Modes:
- Read-Only (Default): When using
--text,--bullet, or the standard TUI, the tool only identifies and reports duplicates. - Destructive Mode: Actual changes only occur if the
--removeflag is utilized.- Deletion:
--remove keepdeletes all but the specified number of files in each group. - Symlinking: Adding
--linkto the remove command converts the redundant files into symlinks pointing to the preserved original.
- Deletion:
Confirmation Logic:
Unless the --no-confirm flag is active, the TUI and GUI will always prompt the user for a confirmation code before executing mass deletions or link conversions to prevent catastrophic accidents.
Fuzzy Matching Math
In --fuzzy mode, the tool calculates similarity based on a threshold: