AntiDupl.NET

[JohnW]

[Moderator note: this is the primary AntiDupl.NET program thread. View database entry]

---

AntiDupl.NET program will help you find duplicate images.

It can find and display duplicate images in the main graphic formats: JPEG, GIF, TIFF, BMP, PNG, EMF, WMF, EXIF, and ICON.
The comparison is based on the contents of the files, so the program can find not only almost identical, but similar images.
In addition, the program can find images with some types of defects.

I used it on a folder containing about 4000 images. It found many duplicates with different names and file sizes. All in under 2 minutes. It was not infallible but I was very impressed.

It does require .NET Framework and this still seems to upset quite a few PFC members.
(Doesn't bother me!)

But it works really well

More information and download from ...

http://antidupl.narod.ru/net/data/help/ ... index.html

[webfork]

Ghacks
http://www.ghacks.net/2010/03/13/detect ... idupl-net/

[I am Baas]

AntiDupl-2.x series do not require .net framework.

http://antidupl.narod.ru/english/index.html

[webfork]

AntiDupl-2.x series do not require .net framework.

http://antidupl.narod.ru/english/index.html

Where did you see that?

Note: If you have not installed on your computer Windows Vista (or later), then for this series required Microsoft.NET Framework 2.0 (22 Mb) or a later version. (http://antidupl.narod.ru/english/download.html)

[I am Baas]

AntiDupl-2.x series do not require .net framework.

http://antidupl.narod.ru/english/index.html

Where did you see that?

Download page @ http://antidupl.narod.ru/english/download.html

[webfork]

Oh okay -- now that I look at it more carefully, I'm not sure what the word "this" in that message is referring to. To me that seems to refer to the programs below it (the 2.x series). The author really needs to be more clear.

[billon]

Latest version is 2.3.0 (07/05/2013)

2.3.0
There are following changes in the current version:
Added support for DDS(DirectDraw Surface) and TGA(Truevision Graphics Adapter) image formats.
A search profiles was added.
Added optional limitation to the maximum size of the image when searching.
Added saving/recovering of the position of the main window.
The program will recommend to remove more new pictures, when all other things being equal.
Also bugs were fixed:
The program opens an invalid directory when renaming of pictures.
The program does not move a picture to another disk when renaming.

2.2.11
There are following changes in the current version:
Added translation into Ukrainian.
Added translation into German.
Also bugs were fixed:
Directory "user" is created in program directory even if a user set another path to user directory.

2.2.10
There are following changes in the current version:
Added optional checking the database of mistakes when loading.
Added the display of the process of load and save the database of mistakes.
Website of the program changed its address.

2.2.9
There are following changes in the current version:
Added the partial loading of image database.
Added support of different versions of the database of images, the database of mistakes and the file of results.
By default, user files are stored in a separate directory now.
Added minimal time of displaying of start window (3 seconds).
Added translation into Simplified Chinese.

2.2.8
The following error was fixed in the current version:
Incorrect image comparison.

2.2.7
The program has open-source code starting with this version.

2.2.6
There are following changes in the current version:
The system of version numbering was changed (the last number was removed, because it did not carry the semantic load).
Checking results at loading was made optional.
The ability to update the database of images manually was added.
The automatic verification of a database of images was removed.
Also bugs were fixed:
A memory leak during the search.
Incorrect interrupt the loading process of database with pictures.
Incorrect state of the status bar during of the search.

2.2.5.531
The following error was fixed in the current version:
The program gets the non-zero difference between absolutely identical images.

2.2.4.528
There are following changes in the current version:
Belarusian translation was added.
Traditional Chinese translation was added.
Also bugs were fixed:
Some of the images in JPEG format were mistakenly identified as defective.
Help was not called when the program path contains spaces.
Pictures were drawn incorrectly after resizing of the program window.
The program does not display images that have marked as "read only".

2.2.3.520
The following critical error was fixed in the current version:
The pictures are not deleted and not renamed.

2.2.2.517
There are following changes in the current version:
The algorithm for image loading was accelerated.
The ability to disable the hot keys was added.
Now the priority of an application is decreased while running in the background mode.
The maximum number of threads of comparing and loading was increased.
Added the ability to call help from dialog windows.
Also bugs were fixed:
Images blink when current result has been changed.
Sometimes the memory is not freed after the end of search.
Error displaying some files with GIF-animations.

2.2.1.503
There are following changes in the current version:
Now, search and other long-term processes can be interrupted by pressing Escape key.
Instead of a one database of pictures, is now used separate databases for each value of the parameter "normalized size of images" that allows you to save memory.
The mode of program, when the database was constantly placed in memory, was removed.
Now the parameter "Normalized image size" can has value 128.
Also bugs were fixed:
Incorrect sorting of results.
Incorrect loading of damaged and non-standard images in Jpeg2000 format.
In Windows 7, the current picture was not selected when opening of the parent directory.
The program does not display images with long file names.
Search was interrupted when the user pressed the Space key.

2.2.0.495
There are following changes in the current version:
Add support for JP2(Jpeg2000) and PSD image formats.
In the column "Dimensions" results are now sorted by the image area, rather than its width.
Now, after searching the program sorts the results in the same manner as they have been sorted before.
Optionally, the program can not compare images with one another within the same search path.
When comparing the images, the program can optionally ignore the edge of the picture.
An hot keys were added into Paths Window: (Control-C, Control-V, Control-X and Delete).
Also bugs were fixed:
The program crashes his job, if the user is first deleted the last result in the table, and then tried to start the selection results by pressing Shift.
In some case the program incorrect group the results.
Grammatical errors in the Russian version of Help.

2.1.1.466
There are following changes in the current version:
Support for Drag & Drop to add files and directories into Paths Window was added.
Now, a Progress Window does not appear in the case of short (less than 200 ms) operation.
The program adds file extension to the renamed image, if the user forgot to do it himself.
Translation into Simplified Chinese was added.
Also bugs were fixed:
The program erroneously was using data about the pictures which collected by the previous version of the program.
The user could enter incorrect values in options window.
The file of results and mistake database were saved incorrectly, if the user renames or replaces pictures into the program.
Renaming or moving files resulted in loss of data about these files into the database of mistakes.

2.1.0.452
There are following changes in the current version:
The algorithm of image comparison was changed (to increase the accuracy of the program).
Now you can optionally limit the number of search results (for resource savings).
Now you can adjust the number of operations of undoing (to save resources and to accelerate the work program).
The parameter 'Threshold Difference' was placed at the main toolbar.
The algorithm of deleting temporary files was changed.
Limitation to the user to stop some operations was added (for data integrity).
A window of progress was added for the operations with the current result.
The type of image was added to the information placed under image preview.
If a pair of images have a different types, it will be highlighted in red now.
Renaming of the image is now possible by means of double-clicking on the name of pictures on the preview panel.
There was added translation into German.

2.0.0.431
There are following changes in the current version:
The support for 64 bit OS was added (now the program can scan a collection of images containing more than 1 million images).
Now you can undo/redo the last 10 user actions.
Now you can rename and move images inside the program.
Now a pair of images can optionally be displayed according to their relative size.
Now optionally small images are not stretch to the size of the preview panel.
The difference between a fully identical images is highlighted in green now.
The possibility of simultaneous replacing of multiple pairs of images was removed (since it could lead to uncertain results).
Also help was updated and bugs were fixed:
The program incorrectly displays the percentage of performed work.
The program do not correctly displays results, if it is minimized during the searching at Windows 7.
The program sometimes crashes when searching.

1.8.5.379
There was added translation into Korean.

1.8.4.375
There was added translation into German.

1.8.3.372
There are following changes in the current version:
Added translation into Simplified Chinese.
Also help was updated and bugs were fixed:
The program shows a few times the same search results.
When changing the current result, the preview panel and the table of results are flickered.

1.8.2.365
The following critical error was fixed in the current version:
The pictures are not deleted.

1.8.1.362
There are following changes in the current version:
The program icon was improved.
Added translation into Traditional Chinese.
Also help was updated and bugs were fixed:
Incorrect annotation of remaining time during search.
Animated GIF is not drawn.

1.8.0.354
There are following changes in the current version:
A window to display the progress of long processes was added (processing of the results, downloading and saving of files, closing of program).
The button "Refresh results" was added.
The form of storage of the results, as well as algorithms for their processing were changed.
Now the program when opening a directory with the picture by using of external program at once selects this picture.
Now you can customize the load process a database with pictures.
Added translation into Italian and Spanish.
Also help was updated and bugs were fixed:
The program does not remove and does not display defective images.
Option "Delete to recycle bin" does not apply until start the search.
Group is not displayed for defective images.
Slow updating of the table of results.
Incorrect program hint, if the difference of images is zero.
The result table is not cleared before the search.

1.7.1.319
The following bugs were fixed in the current version:
The main window is not minimized when user click "Minimize"in the window of progress of the search.
Low performance during loading of the program and during the beginning of the search.
The program crash during searching in very large collections of images.
Incorrect recommendations of program for the automatic processing of images.
Checking for defect images is performed even if appropriate option was disabled.
Incorrect stopping of search (results are not shown after interruption of the search by the user).

1.7.0.311
There are following changes in the current version:
Now you can minimize the program to the taskbar or system tray during searching.
Now a pictures are loaded into several streams during searching.
The data about defects are saved in a database of images now.
The option "Remember the picture" was removed (it is always enabled now).
The amount of memory used by the program was reduced about 2 times.
The design of search progress window was changed.
The command line options were added (you can set your path for storage of service files of the program).
The system of numbering versions was changed (the latter figure is matching the revision number now).
Also help was updated and bugs were fixed:
Unhandled exception in the case of out of memory when working with very large images.
Incorrect localization of a self-extracting archive.

New site - http://antidupl.sourceforge.net
Now open-source
Download self-extracting exe, extract, run AntiDupl.NET.exe. Requires .NET 2.0. Settings in application folder
Tested (on XP): Portable and stealth

[I am Baas]

AntiDupl.NET-2.x series does not require .net framework. The latest version (AntiDupl.NET-2.2.6.exe, 09/27/2012) is portable.

[__philippe]

AntiDupl is my preferred "similar images" search engine, holding its own even against some commercial products.

AD proves amazingly fast at processing additional pictures, once the initial scanning of an entire image collection is completed, and the results indexed into its own database.
(for instance, comparing about 50 new pictures against an indexed DB of 30,000 entries takes under 60 sec)

Another tool in the same category, imgSearch, offers one really nifty feature for displaying its search results:

Unlike AD which displays pairs of similar pictures in succession, imgSearch goes to the trouble of displaying multiple similar images side by side as a group,
while preserving the pictures relative aspect ratio, which is very convenient when vetting pictures to keep or to delete from a group.

On the down side, imgSearch is MUCH slower than AD for scanning large collection of files (say > 2000),
and, unfortunately, version 3.0.8 has not been updated since March 2007.

imgSearch on PFC: http://www.portablefreeware.com/index.php?id=1507

Cheers,

__philippe

[__philippe]

Latest AntiDupl.Net released: 2.3.1 as of 02-AUG-2013

http://antidupl.sourceforge.net/english ... nload.html

2.3.1 Changelog:

• Added translation into Byelorussian.
  Added translation into German.

Bugs fixed:

• The menu items "Open search profile" and "Save search profile as" was duplicated on the toolbar.
  Prevent saving a search profile with zero number of results.
  The threshold difference in the toolbar was not updated when loading the search profile.
  Sometimes the program settings were not saved.
  The program could not load results containing defective images.

__philippe

[__philippe]

AntiDupl.NET-2.3.5 released as of 04-MAR-2105
Still running happily under XP...

Bin/Sourcecode archive : http://antidupl.sourceforge.net/english ... nload.html

2015 Changelog history so far (verbatim) :

04.03.2015 - AntiDupl.NET-2.3.5
- Speed up calculating SSIM by means of SSE2.

25.02.2015 - AntiDupl.NET-2.3.4
- Added menu items to a context menu for the selected files: "To replace", "Rename as" and "Move to".
- Added to a context menu of images entering into groups, the containing more than two files,
possibility of group renaming and movement of files of group.
- Some leakages of memory when reading EXIF of information are eliminated.
- The algorithm of calculation of bluring is changed.

04.02.2015 - AntiDupl.NET-2.3.3
- Added new algorithm of comparing of images by index of structural similarity - SSIM.
It allows to find duplicates among strongly changed pictures, and also images with a big uniform background are better compared.
- Added reading and display in the tooltip EXIF information on the main fields.
- Fixed crash during the processing of big images and a lack of memory, now in case of a lack of memory these files are passed.

12.01.2015 - AntiDupl.NET-2.3.2
- Added detection of a blockiness and blur of the image.
- Added menu items "Rename like neighbour" and "Move and rename image to neighbour".
- Possibility of specifying of search in subdirectories for each directory in a window of a choice of ways is added

[Midas]

Thanks for the update, __philippe.

Here's a screenie:

On a related query, have you tried AllDup (http://www.portablefreeware.com/?id=1791)?

[__philippe]

@Midas

After briefly browsing AllDup documentation, it seems their duplicates search engine is lacking one crucial function, namely a mechanism to detect duplicates by "contents similarity", via some adjustable threshold parameter, which is why I view AntiDupl a better choice (for my current usage, anyway).

Actually, the most recent AntiDupl release now offers two distinct "similarity search" algorithms, with various speed of processing:

• Standard "mean-square difference" faster algorithm, calculates a mean square deviation of brightness for each couple of images
• Newer "SSIM" (Structural SIMilarity index) slower algorithm, better at detecting strongly changed pictures

see "Images Comparison Algorithms" and "Threshold Parameters" options detailed explanation here:
http://antidupl.sourceforge.net/data/he ... tions.html

Thanks for the pix, anyway !

__philippe

[Midas]

Many thanks for the breakdown, __philippe.

[I am Baas]

AntiDupl.NET v2.3.6 is available (02.04.2015):

• Added the option "Compare images inside one directory".

• Menu items "Refresh database of image" exchanged on "Delete irrelevant records from a database of image". Whether now it checks there are files for which sketches are stored in a database on a former place. If isn't present, then irrelevant record are deleted. Allows to reduce the database size after operations of deleting or relocation.

• The column of display of the size of group is added (number of pictures in group).

• Added to a context menu of images items of adding of its or its directory in the list of the ignored paths.

http://antidupl.sourceforge.net/english ... nload.html