How to select and delete batch copies of files in Windows?

[HeyItsChris]

How to select and delete batch copies of files in Windows?

 

I tried Beyond Compare the application is sort of difficult to use though for that task but I think it's possible on that application.

 

Thanks

[ENTERPRISE]

Are you trying to find file duplicates and then removing them?

[HeyItsChris]

Yes Sir!  Thank you so much for replying.

[ENTERPRISE]

I have a great utility for this. I will have to have a dig in my software folder and I will upload it. (Free Open-Source software)

[HeyItsChris]

Thank you so much buddy!  It's for my friends music collection.  After converted his music from .M4A to .MP3, he some how made duplicates within nearly every artist folder.

 

Thanks again

[ENTERPRISE]

I need to re-download it, however the resource is here : 

 

Go to the marked solution

 

This utility does a multitude of duplicate finding, so it is not limited to just images as that thread may suggest.

[HeyItsChris]

Thanks man!  I'll try it out now.

[HeyItsChris]

For some reason it's not working.  I added the entire folder of all the folders containing the artists.  Which is only like 7 folders, from which only a couple contain duplicate music files which looks like this "song example.mp3 & song example (1).mp3".  I need to get rid of the (1) versions of the same song duplicates.

 

I tried individual folders as well and it still didn't work.

 

*Update* 

I reset the configuration & now it's working but how do I delete the duplicates by a batch operation & not individually?  I guess I will try & figure it out.

Edited April 28, 2022 by HeyItsChris

[HeyItsChris]

I proceeded to search as name & it found nothing but Hash did find a bunch & so I selected them by adding *(1)* and deleted them but they are all still there.  The application can't find the filenames with the (1) on them in the specified folder...  Or maybe I'm doing something wrong (most likely) hahaha

 

I made a backup first so I didn't mess anything up so I can fix it.  Idk what to do now if I cannot specify to search for all filenames with (1) in them in this specified folder then idk what to do...

 

*update*

Testing Music Duplicates now 

 

*Update 2* 

Still nothing.  Music Duplicates freezes up while accessing 1604 songs on a Samsung EVO 970 Plus nvme SSD on a 4.5Ghz Ryzen 3700X with 32GB RAM.  Should I wait 10-30 minutes or end task?  It seems to be doing nothing in the task manager.

 

 

 

 

 

Edited April 28, 2022 by HeyItsChris

[ENTERPRISE]

Strange, it worked fine for me. I won't be able to help until im home later. If I recall, you have to install a prerequisite (cannot recall the file) prior to using the utility. 

 

I know you can definitely select all duplicates in one click for removal, definitely do not need to go through and click one by one. 

 

I would leave it running prior to ending it. I found it doesn't use a huge amount of resources, so telling whether or not it is doing anything via Task Manager may be misleading.

[schuck6566]

ccleaner has a duplicate file cleaner app built in it's tools section but you have to run it then ck each file you want to have deleted. Or ck the box at top to remove all.

 

Edited April 28, 2022 by schuck6566

[HeyItsChris]

Thanks man but I have 1,604 files to be checked & I'm sure there is about 50% of them duplicates so 802 files would take forever lol

 

I appreciate the guidance though, I'm learning something new every single day!

[HeyItsChris]

Well in the end I just ended up using Windows Search & typed in (1) & it found all the files with (1) in them & I deleted them all, took a few seconds but it's done now.

 

Since all my duplicates just simply had a (1) at the end of the filename.

 

Thanks again everyone

Edited April 28, 2022 by HeyItsChris

[tictoc]

Late to the show, but this is the kind of task that can be quickly accomplished from the CLI. More than likely this is what those GUI apps are using under the hood.

Since you're on windows we can use PowerShell with a few cmdlets.  *Note* I am running this from PowerShell on Linux, but everything should be the same on Windows.

 

Here is a test scenario I created to accomplish this from the CLI.

List directories, sub-directories, and files in the current path:

Spoiler

PS /home/tictoc/temp/testing/powershell> Get-ChildItem -Recurse               

    Directory: /home/tictoc/temp/testing/powershell

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
drwxr-xr-x tictoc           tictoc              4/28/2022 19:00           4096 music
drwxr-xr-x tictoc           tictoc              4/28/2022 19:02           4096 music2

    Directory: /home/tictoc/temp/testing/powershell/music

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 17:37              0 test (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:33              0 test.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:37              0 test1 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:36              0 test1.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test2 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test2.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test3 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test3.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test4 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test4.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test5 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test5.mp3

    Directory: /home/tictoc/temp/testing/powershell/music2

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test1 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test1.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test2 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test2.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test3 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test3.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test4 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test4.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test5 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test5.mp3

 

 

List all the files with "(1)" somewhere in the middle of the file name.

PS /home/tictoc/temp/testing/powershell> Get-ChildItem -Include *`(1`)* -Recurse

    Directory: /home/tictoc/temp/testing/powershell/music

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 17:37              0 test (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:37              0 test1 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test2 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test3 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test4 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test5 (1).mp3

    Directory: /home/tictoc/temp/testing/powershell/music2

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test1 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test2 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test3 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test4 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test5 (1).mp3

 

Find the files and delete them:

PS /home/tictoc/temp/testing/powershell> Get-ChildItem -Include *`(1`)* -Recurse | RemoveItem

or the short and sweet one-liner with aliases:

PS /home/tictoc/temp/testing/powershell> gci -i *`(1`)* -r | ri

 

Final result:

PS /home/tictoc/temp/testing/powershell> Get-ChildItem -Recurse

    Directory: /home/tictoc/temp/testing/powershell

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
drwxr-xr-x tictoc           tictoc              4/28/2022 19:14           4096 music
drwxr-xr-x tictoc           tictoc              4/28/2022 19:14           4096 music2

    Directory: /home/tictoc/temp/testing/powershell/music

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 17:33              0 test.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:36              0 test1.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test2.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test3.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test4.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test5.mp3

    Directory: /home/tictoc/temp/testing/powershell/music2

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test1.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test2.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test3.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test4.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test5.mp3

 

 

Random aside, running PowerShell on Linux always feels like some whacky voodoo.

 

Edited April 29, 2022 by tictoc
typos

[ENTERPRISE]

9 hours ago, tictoc said:

Late to the show, but this is the kind of task that can be quickly accomplished from the CLI. More than likely this is what those GUI apps are using under the hood.

Since you're on windows we can use PowerShell with a few cmdlets.  *Note* I am running this from PowerShell on Linux, but everything should be the same on Windows.

 

Here is a test scenario I created to accomplish this from the CLI.

List directories, sub-directories, and files in the current path:

  Reveal hidden contents

PS /home/tictoc/temp/testing/powershell> Get-ChildItem -Recurse               

    Directory: /home/tictoc/temp/testing/powershell

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
drwxr-xr-x tictoc           tictoc              4/28/2022 19:00           4096 music
drwxr-xr-x tictoc           tictoc              4/28/2022 19:02           4096 music2

    Directory: /home/tictoc/temp/testing/powershell/music

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 17:37              0 test (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:33              0 test.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:37              0 test1 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:36              0 test1.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test2 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test2.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test3 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test3.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test4 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test4.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test5 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test5.mp3

    Directory: /home/tictoc/temp/testing/powershell/music2

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test1 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test1.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test2 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test2.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test3 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test3.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test4 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test4.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test5 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test5.mp3

 

 

List all the files with "(1)" somewhere in the middle of the file name.

PS /home/tictoc/temp/testing/powershell> Get-ChildItem -Include *`(1`)* -Recurse

    Directory: /home/tictoc/temp/testing/powershell/music

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 17:37              0 test (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:37              0 test1 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test2 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test3 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test4 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:39              0 test5 (1).mp3

    Directory: /home/tictoc/temp/testing/powershell/music2

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test1 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test2 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test3 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test4 (1).mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test5 (1).mp3

 

Find the files and delete them:

PS /home/tictoc/temp/testing/powershell> Get-ChildItem -Include *`(1`)* -Recurse | RemoveItem

or the short and sweet one-liner with aliases:

PS /home/tictoc/temp/testing/powershell> gci -i *`(1`)* -r | ri

 

Final result:

PS /home/tictoc/temp/testing/powershell> Get-ChildItem -Recurse

    Directory: /home/tictoc/temp/testing/powershell

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
drwxr-xr-x tictoc           tictoc              4/28/2022 19:14           4096 music
drwxr-xr-x tictoc           tictoc              4/28/2022 19:14           4096 music2

    Directory: /home/tictoc/temp/testing/powershell/music

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 17:33              0 test.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:36              0 test1.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test2.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test3.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test4.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 17:38              0 test5.mp3

    Directory: /home/tictoc/temp/testing/powershell/music2

UnixMode   User             Group                 LastWriteTime           Size Name
--------   ----             -----                 -------------           ---- ----
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test1.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test2.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test3.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test4.mp3
-rw-r--r-- tictoc           tictoc              4/28/2022 19:02              0 test5.mp3

 

 

Random aside, running PowerShell on Linux always feels like some whacky voodoo.

 

 

That is awesome man, though I have to say I am a basic man at times and just prefer GUI as I am being lazy lol.

[Diffident]

On 29/04/2022 at 01:36, tictoc said:

Random aside, running PowerShell on Linux always feels like some whacky voodoo.

 

At least on Linux it's using the proper forward slash instead of the uncomfortable backslash.

[tictoc]

26 minutes ago, Diffident said:

 

At least on Linux it's using the proper forward slash instead of the uncomfortable backslash.

 

PowerShell is crazy in that it will display the path "properly" on Linux, but you can do whatever you want from the command line. The aggressive path normalization can break all sorts of things in a mixed Unix/Windows environment.

 

From the command line these are all the same and the shell doesn't care:

PS /> gci -Path home/tictoc/temp/testing/powershell -i *`(1`)* -r | ri

PS /> gci -Path home\tictoc\temp\testing\powershell -i *`(1`)* -r | ri

PS /> gci -Path home\tictoc/temp\testing/powershell -i *`(1`)* -r | ri