Remove Duplicate Files

Duplicate File Detective

Monday, November 30, 2009

Create Shortcuts to Moved (or Removed) Duplicate Files

Duplicate File Detective v3.0 contains a large number of new features that are the direct result of user feedback.

One of the most important of these is the ability to create shell shortcuts when moving or removing (deleting) duplicate files. The idea here is quite simple - if you have (for example) a group of four duplicate files, it should be possible to move or delete three of them and create shell shortcuts in their place - those shell shortcuts will then resolve (or point) to the first unmarked file in the same duplicate group.

You can also create shell shortcuts when archiving duplicate files (e.g. adding them to a zip file), as long as you elect to remove the original duplicate files as they are added to the archive (the original file must no longer exist in order for its replacement with a shell shortcut to make sense).

Let's take a look at how these options are presented within Duplicate File Detective v3.0. Please click the screen shot below for a slightly larger image.

delete, move duplicate files with shell shortcuts

When creating shell shortcuts to moved or deleted duplicate files, the prerequisites are as follows:

  1. Duplicate files must be moved or deleted to make room for their replacement with shell shortcuts.
  2. At least one member of each duplicate group being processed must be unmarked. This is the target file to which duplicate shortcuts will resolve.
  3. If any of the above requirements are not met, shell shortcut creation will not be attempted for that duplicate group.

The benefit of this approach is clear - if you're working to dedupe a large document repository, for example, you can archive a series of duplicate files into a single large zip file and still allow users to access those files through (newly created) shell shortcuts.

In future releases of Duplicate File Detective, we'll be expanding upon this concept even further. Please stay tuned for details.

Wednesday, November 25, 2009

Calculating File Hashes (Checksums) in Duplicate File Detective v3

Today I'd like to briefly illustrate some of the improvements made to the file hash calculator tool in Duplicate File Detective v3.0.

First, and perhaps most importantly, the file hash calculator has been re-designed to allow for the generation of multiple file hash types for multiple file paths - all in a single pass.

file hash / checksum generation with Duplicate File Detective v3As you can see in the adjacent screen shot, it is now possible to generate CRC32, ADLER32, MD5, SHA1, SHA256, and SHA512 file checksums with only a single read of each target file.

The output of the file hash / checksum operations has also been re-engineered to display results in a hierarchical (tree like) fashion. For each file hashed, a series of child nodes show the hash value for each type (e.g. CRC32, MD5, etc.) selected in the area above.

It is also possible to copy file hash results to the Windows clipboard, in which case they are "flattened" into a series of plain-text lines.

Here's a quick sample of what the hash results shown above look like on the Windows clipboard (folder names and hash values are truncated for easier viewing in this limited space):

C:\MoveTest\Sample Pictures\Lighthouse.jpg (CRC32): A3A2B75C
C:\MoveTest\Sample Pictures\Lighthouse.jpg (MD5): 8969288F4245120E7C3870287CCE...
C:\MoveTest\Sample Pictures\Lighthouse.jpg (Adler32): A6EA1A41
C:\MoveTest\Sample Pictures\Lighthouse.jpg (SHA1): 1B4605B0E20CECCF91AA278D10E...
C:\MoveTest\Sample Pictures\Lighthouse.jpg (SHA256): FF86372CE43519D675B8D8D29...
C:\MoveTest\Sample Pictures\Lighthouse.jpg (SHA512): 9BDD0C215A9BE94F6F677F8AD...
C:\MoveTest\Sample Pictures\Hydrangeas.jpg (CRC32): 03632F46
C:\MoveTest\Sample Pictures\Hydrangeas.jpg (MD5): BDF3BF1DA3405725BE763540D66...
[...]


The actual file hash / checksum generation process is extremely fast and efficient, and can also be canceled if needed.

Finally, it is also now possible to right-click any number of selected files within the main Duplicate File Detective result report and elect to generate hash values for them. This combination makes it very easy to generate and compare hash values of arbitrary strength (e.g. bit length) as needed.

Tuesday, November 24, 2009

Duplicate File Detective 3 is Released

Key Metric Software is proud to announce the release of Duplicate File Detective v3.0, a major upgrade to our professional-grade duplicate file management solution.

Duplicate File Detective v3.0 provides a number of new features that focus on extending the scalability and reach of the product. New features include full Unicode support, new duplicate file processing options (such as the ability to replace moved or deleted duplicates with shell shortcuts), duplicate search logging, support for Windows 7 and Server 2008, and scalability improvements that allow the product to report upon (and process) millions of duplicate file entries in a single session.

- More Info on What's New in v3
- Full Release Notes
- Download Duplicate File Detective v3.0 Now

We're extremely proud of this release and, as always, welcome your thoughts and feedback.