Duplicate File Finder (File Comparrison, File Compare)

Tell us your most wanted features from QNAP products.
smitty123
Know my way around
Posts: 148
Joined: Tue Apr 16, 2013 2:13 pm

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by smitty123 » Sat Jan 09, 2016 6:26 pm

it's real, but your suggestion is interesting

looking at https://www.qnap.com/i/useng/app_center ... jump_win=1 it says linux-station needs 4gb ram.

so looks like it won't work on my nas (arm, 256mb ram).

still hoping someone at Qnap will port this in QTS since it's also a linux base.
--------
Qnap TS-212 QTS 4.3... , 2x3TB WD Red.
Windows 10 x64

User avatar
Don
Guru
Posts: 12007
Joined: Thu Jan 03, 2008 4:56 am
Location: Long Island, New York

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by Don » Sun Apr 23, 2017 10:47 pm

Spam meaaages deleted
Read the Online Manuals and use the forum search feature before posting.

It is a recommended to use RAID and have external backups. RAID will protect you from disk failure, keep your system running, and data accessible while the disk is replaced and the RAID rebuilt. Backups will allow you to recover data that is lost or corrupted, or from system failure. One does not replace the other.

Submit bugs and feature requests to QNAP via their Helpdesk app.

NAS: TVS-882BR | F/W: 4.3.6.0895 | 40GB | 2 x M.2 SATA RAID 1 (System/VMs) | 4 x M.2 NMVe QM2-4P-384A RAID 5 (Cache) | 5 x 4TB HDD RAID 6 (Data) | 1 x Blu-ray
NAS: TVS-663 | F/W: 4.4.2.1320 | 16GB | 2 x M.2 NMVe QM2-2P RAID 1 (Cache) | 4 x 4TB RAID 5
Apps: Boinc, Squid, DNSMasq, PLEX, iDrive, QVPN, QLMS, MP3fs, HBS, Entware, DLstation, +others

User avatar
OneCD
Ask me anything
Posts: 7955
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by OneCD » Mon Apr 24, 2017 2:09 am

@smitty123: a couple of posts were removed from this topic as they were in breach of the forum rules regarding advertising - and they fit the spambot profile. ;)

QNAP admin wrote:By participating in the Forums, you agree not to:
  • post any spam messages.
  • post advertisements, chain letters, junk email or bulk communications.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

Blisk
Know my way around
Posts: 109
Joined: Fri Jul 08, 2011 5:54 am

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by Blisk » Mon Feb 26, 2018 5:14 am

after two years still no software for duplicated files?

User avatar
Trexx
Ask me anything
Posts: 5294
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by Trexx » Tue Feb 27, 2018 12:00 am

There are many way to accomplish this (dupeguru in Docker, Qfiling, VM options, Entware, etc.)
Paul

Model: TS-877-1600 FW: 4.4.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.3.x Test/Backup Box
Model:TS-228a FW: 4.4.3.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

User avatar
parkerc
Easy as a breeze
Posts: 296
Joined: Thu Oct 16, 2014 7:36 am
Location: Near London, England
Contact:

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by parkerc » Sun Nov 17, 2019 4:55 am

This maybe one of my old posts, but I thought I’d update it to say that I’m currently using fdupes to find duplicate files. I have it installed as a .qpkg thanks to the QNAP club.

https://www.qnapclub.eu/en/qpkg/418

You can tweak the published .php file to change the command line used. I changed mine to ignore hidden files (as QNAP has so many) and group by size.

Code: Select all

 fdupes -ASr /share > /share/Public/fdupes.output
Model Name : QNAP TS-253be (16GB) & TS-453D (16GB)
Firmware Version:: QTS Latest
OS Version : Apple iOS (iPad/Safari Browser)
Number & Type of HDDs : 2 x 3TB WD Reds, Raid 1 & 4 x 4TB WD Reds, Raid 5
Website (When I have time) : http://www.nodecentral.co.uk

shooms99
Starting out
Posts: 23
Joined: Thu Aug 11, 2016 12:44 pm

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by shooms99 » Fri Dec 06, 2019 12:03 pm

Trexx wrote:
Tue Feb 27, 2018 12:00 am
There are many way to accomplish this (dupeguru in Docker, Qfiling, VM options, Entware, etc.)
How do I use QFiling to look for duplicate files? I don't have a specific file in mind, but I just want to generate a list of all ducplicate documents (photos, videos, files, etc.) so I can free up space on the QNAP.

User avatar
Xoffis
Starting out
Posts: 10
Joined: Fri Nov 29, 2019 9:01 pm
Location: Sweden
Contact:

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by Xoffis » Thu Dec 12, 2019 7:45 pm

I also installed QFiling/Qsirch just for dupe search, don't seem to have that function?

So I do want to +1 on this!
Sorry, English is not my native language, plz have oversight
And i have autism, so try not to answer with irony and so on as i can have large trouble understand it and may take it personally

A NAS n00b with my first real NAS a TS-231P2
4Gb RAM
2x6TB IronWolf drives

shooms99
Starting out
Posts: 23
Joined: Thu Aug 11, 2016 12:44 pm

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by shooms99 » Thu Dec 12, 2019 8:08 pm

If you are comfortable with SSH, I was able to follow this walkthrough as a guide:

https://blog.linuxserver.io/2017/09/17/ ... s-on-qnap/

instead of using TVHeadEnd, I chose a random dupeguru image from the repository, which was created by another user.

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9242
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by Moogle Stiltzkin » Thu Jan 09, 2020 9:28 am

shooms99 wrote:
Thu Dec 12, 2019 8:08 pm
If you are comfortable with SSH, I was able to follow this walkthrough as a guide:

https://blog.linuxserver.io/2017/09/17/ ... s-on-qnap/

instead of using TVHeadEnd, I chose a random dupeguru image from the repository, which was created by another user.
i couldn't figure out how to get dupeguru docker running, so i instead ran dupeguru from windows pc and pointed to the QNAP share and that seems to work, albeit there is the 1gbe bottle neck :(
https://dupeguru.voltaicideas.net/

I guess using win10 Virtual station and using dupeguru from there works as well, that part i know how. but if all you require is just dupeguru, then docker is much more efficient if you don't require full blown vm.
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by storageman » Fri Jan 10, 2020 12:31 am

Maybe Hero will be available for you guys

rober1967
First post
Posts: 1
Joined: Wed Jun 14, 2017 4:33 pm

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by rober1967 » Wed Aug 12, 2020 9:45 pm

Hi, I've a ts251. Is there some tool that can be installed on nas and that can list all duplicated files ? I installed qfiling, qsirch, qumagie, but I didn't got how to use to search duplicate.
Thanks.

Robert.

plip1978
New here
Posts: 4
Joined: Mon Feb 15, 2016 3:02 am

Re: Duplicate File Finder (File Comparrison, File Compare)

Post by plip1978 » Fri Sep 11, 2020 9:47 pm

+1 on this too. Especially for photos!

Post Reply

Return to “Features Wanted”