QTS Hero... ZFS? What? When?

Interested in our products? Post your questions here. Let us answer before you buy.
Post Reply
User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9389
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: QTS Hero... ZFS? What? When?

Post by Moogle Stiltzkin » Wed Dec 25, 2019 7:23 pm

NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

P3R
Guru
Posts: 12375
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: QTS Hero... ZFS? What? When?

Post by P3R » Wed Dec 25, 2019 9:30 pm

I gave a correct answer to your question and threw in a little joke as a bonus.

To see my answers I'm afraid you have to deal with that sometimes I'll give you a little more than you asked for.

If you don't like my posts then please feel free to add me as a foe, or deal with the whole content.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
dolbyman
Guru
Posts: 20398
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: QTS Hero... ZFS? What? When?

Post by dolbyman » Wed Dec 25, 2019 11:10 pm

x51 and x53 units in the list..what happened to "mid range" units or higher?

wonder how well they cope
Last edited by dolbyman on Thu Dec 26, 2019 11:38 am, edited 1 time in total.

adamyh
New here
Posts: 5
Joined: Wed Feb 01, 2017 11:04 pm

Re: QTS Hero... ZFS? What? When?

Post by adamyh » Thu Dec 26, 2019 4:45 am

I think ZFS will be fine. There is less overhead for snapshots and scrubs compared to ext4. The dedup requires lots of memory through. The challenge is the ZIL. That’s where I/O is key.

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9389
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: QTS Hero... ZFS? What? When?

Post by Moogle Stiltzkin » Thu Dec 26, 2019 7:21 am

wait, do we will still need to do raid scrubs with zfs ?

at least they did say for file system check, that should no longer be required for zfs? so i am assuming they may remove that if you install qts hero?
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

P3R
Guru
Posts: 12375
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: QTS Hero... ZFS? What? When?

Post by P3R » Thu Dec 26, 2019 7:55 am

Moogle Stiltzkin wrote:
Thu Dec 26, 2019 7:21 am
wait, do we will still need to do raid scrubs with zfs ?
It's an integral part of ZFS and one key to the reliability.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

P3R
Guru
Posts: 12375
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: QTS Hero... ZFS? What? When?

Post by P3R » Thu Dec 26, 2019 8:31 am

adamyh wrote:
Thu Dec 26, 2019 4:45 am
The dedup requires lots of memory through. The challenge is the ZIL. That’s where I/O is key.
Yes and with almost all the TS-X5X (desktop) models being limited to 4 bays or less there is no help to expect from SLOG SSDs (except in the very few PCIe models). I don't know but add limited amounts of (non-ECC) RAM and I doubt that the TS-X5X user experience will be as positive as you think.

The early adopters are so sold on the ZFS-idea that the disadvantages will not matter to them. It will be more interesting to see how the second wave of users, that aren't ZFS-fans, will react to the possible performance limitations, not to mention the very limited RAID expansion/migration possibilities.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9389
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: QTS Hero... ZFS? What? When?

Post by Moogle Stiltzkin » Thu Dec 26, 2019 8:38 am

P3R wrote:
Thu Dec 26, 2019 7:55 am
Moogle Stiltzkin wrote:
Thu Dec 26, 2019 7:21 am
wait, do we will still need to do raid scrubs with zfs ?
It's an integral part of ZFS and one key to the reliability.
ah i c. ty :'

P3R wrote:
Thu Dec 26, 2019 8:31 am
Yes and with almost all the TS-X5X (desktop) models being limited to 4 bays or less there is no help to expect from SLOG SSDs (except in the very few PCIe models). I don't know but add limited amounts of (non-ECC) RAM and I doubt that the TS-X5X user experience will be as positive as you think.

The early adopters are so sold on the ZFS-idea that the disadvantages will not matter to them. It will be more interesting to see how the second wave of users, that aren't ZFS-fans, will react to the possible performance limitations, not to mention the very limited RAID expansion/migration possibilities.
when the time comes we can add a poll and see how users feel about it. i just hope that it is rock stable..... :?

and yes you are right about the limited raid expansion. i think if you want to expand, you need to start from scratch. for me... thats ok.... i don't do this often, and i keep backups. but for people that want to slot in a new hdd and expand the current raid, they'd probably be annoyed.


i just notice rob released a demo video of the qts hero :'
https://www.youtube.com/watch?v=gCxtX4qwSus


the part i was looking at is that pre compression and post compression storage sizes are both displayed. this is important because if your destination is not using a zfs compression, then you will need to look at pre compression disk space usage to check that it will fit when backing up. (check the qnap response about this in this thread somewhere explaining how this works)

did anyone else spot what other hurdles there may be with this qts hero :'


data reduction statistics.... it doesn't seem as if it would show you how much space was saved based on filters like directory locations, media type etc..... it just show you a general statistics.... not sure if possible, but if it could inform the user what filtered media where compression is benefiting the most, that would be useful to know.

atm how i check on this is perhaps using qsirch (or the windows tree size app) to then sort by highest storage usage files in order, then replace the ones eating up the most space. I get a lot of savings by replacing x264 videos with x265 for an almost similar level of quality but a lot of space savings.

i would also usually notice that some videos have flac audio, so i'd probably replace that with AAC or opus versions which use less space, but still acceptable quality for most people.


@18:20 into the video, rob mentions that snapshots seem to be more faster. but does it also have better space savings for creating snapshots? or is that similar to the current snapshots in that aspect?


a funny pic analogy i found that sums up wut qts hero does for you. before and after :mrgreen:
Image
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

User avatar
Trexx
Ask me anything
Posts: 5304
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

QTS Hero... ZFS? What? When?

Post by Trexx » Sun Jan 05, 2020 5:00 am

NASWorld has a QTS Hero in-line compression test video up

https://youtu.be/YPBk6-ryNnE


Sent from my iPhone using Tapatalk
Paul

Model: TS-877-1600 FW: 4.4.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.3.x Test/Backup Box
Model:TS-228a FW: 4.4.3.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

User avatar
Moogle Stiltzkin
Ask me anything
Posts: 9389
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: QTS Hero... ZFS? What? When?

Post by Moogle Stiltzkin » Sun Jan 05, 2020 8:17 am

so for qts hero do you suggest we select thin provisioning as shown in that video demo you linked?

or can i still use thick as usual? or is that not a good setting for zfs?
NAS
[Main Server] QNAP TS-877 w. 4tb [ 3x HGST Deskstar NAS (HDN724040ALE640) & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A w. 5x 2TB Samsung F3 (HD203WI) EXT4 Raid5
[Backup] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) single disks.
[^] QNAP TS-659 Pro II
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-228
[^] QNAP TS-128
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Asus AC68U Router|100dl/50ul MBPS FTTH Internet | Win10, WC PC-Intel i7 920 Ivy bridge desktop (1x 512gb Samsung 850 Pro SSD + 1x 4tb HGST Ultrastar 7K4000)


Guides & articles
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin

P3R
Guru
Posts: 12375
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: QTS Hero... ZFS? What? When?

Post by P3R » Sun Jan 05, 2020 9:32 am

Trexx wrote:
Sun Jan 05, 2020 5:00 am
NASWorld has a QTS Hero in-line compression test video up...
I can record a 5 minute video with dramatic music that show you that a string is exactly 1 meter or 3.28084 feet long. Unfortunately that will not tell you how long strings are...

Without knowing exactly what data was compressed that video is just another Youtube waste of 5 minutes and 34 seconds of the viewers life (hopefully most at least skip through most of it if they feel the urge to watch). To save everyone else wasting their time, the claim made in the video is that 143.93 GB of totally unknown data can be compressed by 28.98 GB or 19.72 %... :S

Different (loss-less) compression algorithms can of course be slightly more or less effective but the differences are usually small and depend on what data is compressed. Encrypted or already compressed data will not really benefit from compression. Most media file formats today are already compressed.

Use Zip or any other compression tool on your own data sets to get an idea of how much you will be able to gain from inline ZFS compression. That will be much, much more useful than this YT-video.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

occamsrazor
Know my way around
Posts: 234
Joined: Tue Mar 30, 2010 8:30 pm

Re: QTS Hero... ZFS? What? When?

Post by occamsrazor » Mon Jan 06, 2020 12:55 pm

I'm confused... is Hero available yet or not? This link doesn't work for me.. https://stage-software.qnap.com/qts-hero-edition.html
TS-451 [4 x 10TB WD Reds in Raid-5]
TS-239 Pro II [2 x 3TB in Raid-0]
pfSense router and Ubiquiti Unifi switches
Mac Minis, MacBook Pro, iPhones

P3R
Guru
Posts: 12375
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: QTS Hero... ZFS? What? When?

Post by P3R » Mon Jan 06, 2020 9:20 pm

occamsrazor wrote:
Mon Jan 06, 2020 12:55 pm
I'm confused... is Hero available yet or not?
As far as I know not as a separate software yet. It may be bundled on the new Special Edition Qnap models named TS-hXXXX if they're already available.
It's probably a test site that for a short time was available outside the organization by mistake. It was only a (non-working I believe) shop for buying the license, not the actual firmware.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!

User avatar
storageman
Ask me anything
Posts: 5511
Joined: Thu Sep 22, 2011 10:57 pm

Re: QTS Hero... ZFS? What? When?

Post by storageman » Tue Jan 07, 2020 12:44 am

The QTS Hero will be available to buy as OS for a select number of models.
I would recommend minimum 32GB of RAM regardless of what Qnap say.

User avatar
qrusher
Getting the hang of things
Posts: 95
Joined: Thu Mar 15, 2018 9:51 pm
Location: RUserious
Contact:

Re: QTS Hero... ZFS? What? When?

Post by qrusher » Fri Feb 07, 2020 5:51 am

i think qts-h mode currently available on some series fw quite time
if some one can boot device on a last fw wo hdd there you can see init screen that let chng device init mode to qts-h

i think

https://www.qnap.com/en/how-to/faq/arti ... as-devices

after reinit from installed qts

x77u
x72u
x83u
You do not have the required permissions to view the files attached to this post.
Last edited by qrusher on Fri Feb 07, 2020 5:52 pm, edited 3 times in total.
crush qnaps like a thanos crushing planets
Updates are compromise between old bug patches and new bugs introduced. 8)
do you backup..then kill it with fire
stuff:mikrotik Crs309-1g-8s+in & s+rj10 x4

Post Reply

Return to “Presales”