TS-473 10Gbe write speeds slow with Mac mini. Great read speeds! Help!

Questions about using NAS on Mac OS.
Post Reply
ChrisSchmid1988
New here
Posts: 4
Joined: Fri Aug 21, 2020 2:46 pm

TS-473 10Gbe write speeds slow with Mac mini. Great read speeds! Help!

Post by ChrisSchmid1988 » Fri Aug 21, 2020 3:01 pm

Hello!

I bought a new Mac mini with the 10Gbe option built in. My NAS configuration is as follows:

- TS-473 NAS with four new 4 TB Seagate Ironwolf Pro drives
- QNAP QXG-10G1T card
- 2x Samsung MZ-N6E500BW 860 EVO M.2 M.2 SSDs configured as a cache.

The Mac mini has a fresh install of Catalina.

I tried everything, but I can't seem to get my write speeds up. I'm getting around 150 to max. 200 MB/s write no matter if I'm using SMB, AFP or NFS. SMB signing is disabled. It does not matter if I'm connecting the NAS directly to the Mac mini or to my Switch (Netgear S3300 52X-PoE+). The read speeds change a little bit with the different constellations, but they always stay at around 900 to 1000 MB/s, which is great.

Jumbo Frames are set to 9000 at the NAS, the Mac and the Switch. I tried setting them all to 1500, but that again only affects the read speeds, not the write speeds. They always stay at around 150 MB/s.

I don't know what to do next :(

I'd be forever thankful to anyone who has an idea!

User avatar
Trexx
Ask me anything
Posts: 5304
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: TS-473 10Gbe write speeds slow with Mac mini. Great read speeds! Help!

Post by Trexx » Sat Aug 22, 2020 11:37 am

ChrisSchmid1988 wrote:Hello!

I bought a new Mac mini with the 10Gbe option built in. My NAS configuration is as follows:

- TS-473 NAS with four new 4 TB Seagate Ironwolf Pro drives
- QNAP QXG-10G1T card
- 2x Samsung MZ-N6E500BW 860 EVO M.2 M.2 SSDs configured as a cache.

The Mac mini has a fresh install of Catalina.

I tried everything, but I can't seem to get my write speeds up. I'm getting around 150 to max. 200 MB/s write no matter if I'm using SMB, AFP or NFS. SMB signing is disabled. It does not matter if I'm connecting the NAS directly to the Mac mini or to my Switch (Netgear S3300 52X-PoE+). The read speeds change a little bit with the different constellations, but they always stay at around 900 to 1000 MB/s, which is great.

Jumbo Frames are set to 9000 at the NAS, the Mac and the Switch. I tried setting them all to 1500, but that again only affects the read speeds, not the write speeds. They always stay at around 150 MB/s.

I don't know what to do next :(

I'd be forever thankful to anyone who has an idea!
1st I would disable the cache is generally it is more trouble for most use cases than help.

Start simple with running the following commands to get base storage performance (ignore NW for now).

Code: Select all

qcli_storage -T

qcli_storage -t
Also what QTS version, how did you configure your 4 drives (static volume, raid, etc.)?


Sent from my iPad using Tapatalk
Paul

Model: TS-877-1600 FW: 4.4.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.3.x Test/Backup Box
Model:TS-228a FW: 4.4.3.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

ChrisSchmid1988
New here
Posts: 4
Joined: Fri Aug 21, 2020 2:46 pm

Re: TS-473 10Gbe write speeds slow with Mac mini. Great read speeds! Help!

Post by ChrisSchmid1988 » Fri Aug 28, 2020 1:21 am

Trexx wrote:
Sat Aug 22, 2020 11:37 am
1st I would disable the cache is generally it is more trouble for most use cases than help.

Start simple with running the following commands to get base storage performance (ignore NW for now).

Code: Select all

qcli_storage -T

qcli_storage -t
Also what QTS version, how did you configure your 4 drives (static volume, raid, etc.)?


Sent from my iPad using Tapatalk
Thank you so much for the reply! I thought I needed the cache to get the full speed out of the system... Is qtier a better option in your opinion?

I'm using my four drives in a RAID 5 configuration with one thick volume. I'm using the newest version of QTS available which is 4.4.3.1400.

Where can I type in your commands? Do I have to connect to the NAS via SSH? I'm sorry for being such a noob. I thought this would be easier...

User avatar
Trexx
Ask me anything
Posts: 5304
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: TS-473 10Gbe write speeds slow with Mac mini. Great read speeds! Help!

Post by Trexx » Fri Aug 28, 2020 8:42 am

Yes - you have to SSH into your NAS to run the commands that I listed.

What are you planning to use the NAS for? And what is your expectation re: full speed?
Paul

Model: TS-877-1600 FW: 4.4.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x 500GB Evo 860
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
GPU: EVGA GTX 1060 6GB
UPS: CP AVR1350

Model:TVS-673 32GB FW: 4.4.3.x Test/Backup Box
Model:TS-228a FW: 4.4.3.x Test/Backup Box
-----------------------------------------------------------------------------------------------------------------------------------------
NAS RAID Rebuild Times | Live QTS Videos | | QNAP NAS Guide | Information needed when you ask for HELP | QNAP Links, Tutorials, etc.
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq

ChrisSchmid1988
New here
Posts: 4
Joined: Fri Aug 21, 2020 2:46 pm

Re: TS-473 10Gbe write speeds slow with Mac mini. Great read speeds! Help!

Post by ChrisSchmid1988 » Fri Aug 28, 2020 1:22 pm

I've ran the commands. Here's the output of the first one... Doesn't look very promising.

Code: Select all

fio test command for physical disk: /sbin/fio --filename=test_device --direct=1 --rw=read --bs=1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
fio test command for RAID: /sbin/fio --filename=test_device --direct=0 --rw=read --bs=1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
Start testing!
Performance test is finished 100.000%...
Enclosure  Port  Sys_Name      Throughput    RAID        RAID_Type    RAID_Throughput   Pool  
NAS_HOST   1     /dev/sdb      ?@           /dev/md2    RAID 1                        256   
NAS_HOST   2     /dev/sda      ?@           /dev/md2    RAID 1                        256   
NAS_HOST   3     /dev/sde      ?@           /dev/md1    RAID 5                        1     
NAS_HOST   4     /dev/sdf      ?@           /dev/md1    RAID 5                        1     
NAS_HOST   5     /dev/sdc      ?@           /dev/md1    RAID 5                        1     
NAS_HOST   6     /dev/sdd      ?@           /dev/md1    RAID 5                        1     
And here's the second one:

Code: Select all

fio test command for LV layer: /sbin/fio --filename=test_device --direct=0 --rw=read --bs=1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 &>/tmp/qcli_storage.log
fio test command for File system: /sbin/fio --filename=test_device/qcli_storage --direct=0 --rw=read --bs=1M --runtime=15 --name=test-read --ioengine=libaio --iodepth=32 --size=128m &>/tmp/qcli_storage.log
Start testing!
Performance test is finished 100.000%...
VolID   VolName             Pool     Mapping_Name            Throughput      Mount_Path                    FS_Throughput
1       DataVol1            1        /dev/mapper/cachedev1   ?@             /share/CACHEDEV1_DATA         627.45 MB/s
2       HybridMount1        1        /dev/mapper/cachedev2   ?@             /share/CACHEDEV2_DATA         533.33 MB/s
I will be using the NAS in my recording studio for music production. I want to be able to record directly onto it. Sometimes up to 16 tracks of full quality wav files for drum recording at the same time.

I image write speeds will be important there.

As of the speeds I'm expecting... I've seen people with write speeds anywhere from 500 - 800 MB/s with 10gbe. So that must be achievable, right?
Last edited by OneCD on Fri Aug 28, 2020 2:23 pm, edited 1 time in total.
Reason: added code tags

Bob Zelin
Been there, done that
Posts: 937
Joined: Mon Nov 21, 2016 12:55 am
Location: Orlando, FL.
Contact:

Re: TS-473 10Gbe write speeds slow with Mac mini. Great read speeds! Help!

Post by Bob Zelin » Sat Aug 29, 2020 1:05 am

I apologize for "yelling" at you. I just had an email from an editor in LA, that just bought a TS-932X, to complain about it.
I build QNAP systems for professional video editors (and I guess you are in that group, because you are a ProTools guy, or an Apple Logic guy) - and you have done what so many people
have done. You hear about QNAP, you hear about getting 500 - 800 MB/sec, and then you go out and buy the CHEAPEST possible QNAP you can find, and then you complain about crappy performance.

You should have purchased a TVS-872XT. And if you are a single user, you should have purchased a TVS-672XT. And if you were REALLY on a budget, you should have purchased a TVS-472XT. But with only 4 drives, you will never exceed 450 MB/sec. You want 800 MB/sec - you get a TVS-872XT (Intel i5 with 16 Gig of RAM), you hook this up to a 10G port on your Mac, and NOW, you will get 800 MB/sec.
You do NOT get this wonderful killer performance, when you buy the cheapest QNAP you can find. You do NOT get this wonderful killer performance when you only put in 4 drives in your system.
I see people telling me "hey - I bought the QNAP TS-453BT3, and I can't get anything faster than 450 MB/sec". THAT IS CORRECT - you buy a bigger faster QNAP, and you get faster performance.

I saw your post when you first put it up. I am only responding to you know, becuase I see that you are doing professional music production, and using it in your recording studio. You own the wrong QNAP. You will never be happy.
the new QNAP TS-h686 is $1249. But it's still a 4 bay. But it will be faster than what you have now (and you need a 10G card like the QXG-10G1T, which is $89)
the TVS-472XT is also $1249 but comes with the 10G card. Also a 4 bay
The TVS-672XT is $1600
and the TVS-872XT is $2100 complete with 10G card and 16 Gig of RAM.
and for reference, the TS-h886 6 bay which is also a ZFS box that will be faster is $1549, but once again, for either of the 2 small QNAP ZFS boxes, you need a QXG-10G1T 10G card.

SO - what are you going to do with your $650 QNAP TS-473 ? I have no idea.

What would I do if I was you ? You already own the QXG-10G1T. Buy the TS-h886, buy 2 1TB or 500 Gig 2 1/2" SSD's for the QuTS operating system, buy 2 more drives so you will have 6 matching
drives now. NOW you will have a working system. What about the $650 you spent on the TS-473 ? No idea.

Bob Zelin
Bob Zelin / Rescue 1, Inc.
http://www.bobzelin.com

ChrisSchmid1988
New here
Posts: 4
Joined: Fri Aug 21, 2020 2:46 pm

Re: TS-473 10Gbe write speeds slow with Mac mini. Great read speeds! Help!

Post by ChrisSchmid1988 » Sat Aug 29, 2020 2:50 am

Bob Zelin wrote:
Sat Aug 29, 2020 1:05 am
I apologize for "yelling" at you. I just had an email from an editor in LA, that just bought a TS-932X, to complain about it.
I build QNAP systems for professional video editors (and I guess you are in that group, because you are a ProTools guy, or an Apple Logic guy) - and you have done what so many people
have done. You hear about QNAP, you hear about getting 500 - 800 MB/sec, and then you go out and buy the CHEAPEST possible QNAP you can find, and then you complain about crappy performance.

You should have purchased a TVS-872XT. And if you are a single user, you should have purchased a TVS-672XT. And if you were REALLY on a budget, you should have purchased a TVS-472XT. But with only 4 drives, you will never exceed 450 MB/sec. You want 800 MB/sec - you get a TVS-872XT (Intel i5 with 16 Gig of RAM), you hook this up to a 10G port on your Mac, and NOW, you will get 800 MB/sec.
You do NOT get this wonderful killer performance, when you buy the cheapest QNAP you can find. You do NOT get this wonderful killer performance when you only put in 4 drives in your system.
I see people telling me "hey - I bought the QNAP TS-453BT3, and I can't get anything faster than 450 MB/sec". THAT IS CORRECT - you buy a bigger faster QNAP, and you get faster performance.

I saw your post when you first put it up. I am only responding to you know, becuase I see that you are doing professional music production, and using it in your recording studio. You own the wrong QNAP. You will never be happy.
the new QNAP TS-h686 is $1249. But it's still a 4 bay. But it will be faster than what you have now (and you need a 10G card like the QXG-10G1T, which is $89)
the TVS-472XT is also $1249 but comes with the 10G card. Also a 4 bay
The TVS-672XT is $1600
and the TVS-872XT is $2100 complete with 10G card and 16 Gig of RAM.
and for reference, the TS-h886 6 bay which is also a ZFS box that will be faster is $1549, but once again, for either of the 2 small QNAP ZFS boxes, you need a QXG-10G1T 10G card.

SO - what are you going to do with your $650 QNAP TS-473 ? I have no idea.

What would I do if I was you ? You already own the QXG-10G1T. Buy the TS-h886, buy 2 1TB or 500 Gig 2 1/2" SSD's for the QuTS operating system, buy 2 more drives so you will have 6 matching
drives now. NOW you will have a working system. What about the $650 you spent on the TS-473 ? No idea.

Bob Zelin
Hi Bob,

I'm not feeling yelled at :)

Your opinion is valued. Your posts came up a lot when I searched this forum and you really seem to know your stuff. There seems to be something off with QNAPs marketing then. The TS-473 is marketed as a middle class NAS. They even call it an "affordable high end" device on the product page and advertise sequential throughput speeds of 2347 MB/s read and 1764 MB/s write with 2x10GbE cards. Seems like false advertising to me then.

I get that I would need more drives to get 800 MB/s and that's fine. Your video guys obviously need a lot more throughput than I ever will for audio. You talk about 450 MB/s as a mark you could achieve even with a lower end "consumer grade" 10gbe NAS like the TS-453BT3. And I'm nowhere near that! 150 - 200 MB/s seems abysmal even for the TS-473 don't you agree?

Some of the devices you mentioned, like the TVS-472XT, the TVS-672XT seem to be almost the same device as the TS-473/673 but with added Thunderbolt which I neither need nor want. Thunderbolt always comes with a HEFTY premium. Are these devices really that different from my NAS?

Is the processor even that relevant? I'm not using any virtualisation, media functions or anything. The processor and RAM in my NAS are not even sweating even if I'm pushing files, so how would a faster processor help me?

Thank you for your input Bob!

Bob Zelin
Been there, done that
Posts: 937
Joined: Mon Nov 21, 2016 12:55 am
Location: Orlando, FL.
Contact:

Re: TS-473 10Gbe write speeds slow with Mac mini. Great read speeds! Help!

Post by Bob Zelin » Sat Aug 29, 2020 3:51 am

People always ask me "gee Bob - how do you know all that stuff ? " . It's simple. I DONT know all that stuff. I make mistakes all the time. I just install a lot of QNAP's. As for the Annapurna product line, how do I know it's ** ? Because I OWN ONE. I bought a TS-832X when it came out, having installed countless TVS-871T's, and I said "what on earth is going on here - this product is terrible". Since that time, I have had the honor of installing other horrible QNAP products like the TS-1635 and TS-1635ax (because after all - it looks just like the TS-1685, which is double the price). But the TS-1635ax is
1/10th the performance of a TS-1685. There are GREAT QNAP's and there are terrible QNAP's. I have no idea of how the marketing team places certain products. I now see the TS-h886 and the TS-h686 in the Enterprise page of the QNAP website - it's great - but it's not an enterprise product. And I see the QNAP TS-167XU-RP, which is an AMAZING 16 bay rackmount QNAP with killer performance, and it's listed under the SMB section, as if it's an inferior product. But it's the same product as the QNAP TS-1683XU-RP, but it's AMD Ryzen (which is great), as opposed to Intel Xeon (which is also great).
Both of these are ENTERPRISE products - I don't care what's on the QNAP website.

All my experience, all my comments are all based on installing these products, and features. Sometimes things work, sometimes they don't. The Annapurna series of CPU's - they are just garbage.
When I first started to do all of this, it was with the QNAP TVS-863+ (the first 10G product that was and 8 bay) and then soon after the TVS-871T, which was an 8 bay with dual 10G ports. The Thunderbolt 2 ports NEVER worked as advertised - not QNAP's fault - Intel told me directly at an NAB show that you could never edit off of Thunderbolt 2 - but both Apple and QNAP misinterpreted this, and I learned the hard way. When they discontinued the TVS-871T, the TVS-873e came out. I had to put a 10G card and 16 Gig of RAM into this to make it the equivalent of the TVS-871T, but it still was never equivalent to the TVS-871T. It worked - it was cheap, but it was not the TVS-871T (the TVS-873e is similar to the Synology DS1819+ - it works, it's ok, but it's not a great product). When the TVS-872XT came out, it was a game changer. It was a true improvement of the TVS-871T. It's really fast, it has 16 Gig of RAM, the Intel i5, and the built in 10G port. I NEVER use the Thunderbolt ports for anything, other than expansion. On the old TVS-871T, those T2 ports were great for the QNAP TX-800P expanders. QNAP will allegedly have a T3 expander again in the future - but who knows if that will ever happen. Thunderbolt 3 connectivity to a Mac is certainly dramatically better than Thunderbolt 2 was, but it's still not as reliable and consistant at a 10G connection, even though the speed is "slower". And I am not at the mercy of a 2 meter cable.

So when I first saw the TVS-1282T, and the TS-1685, I said "ooh - look - 4 SSD slots for caching - great" - and it did nothing, because I tried it, and it did not improve performance. My clients who have that, and who spend the money for those four SSD's, use it as a true "cache" volume, or separate data volume for graphics, audio, etc - not as QNAP describes caching. Certainly not for these applications.

The TS-453BT3 was another product advertised as a video post production product. Well, if you are doing proxy footage with multiple users - then maybe, but to get even 2 editors to do 4K editing with this, is a real problem. You want performance, you buy a larger system, with more drives, and with a better CPU.

I have recently been demoing the new ZFS TS-h886, which is a 6 bay. The SSD's are used to hold the QuTS ZFS operating system. The product seems absolutely amazing, but it only comes with 2.5 Gig ethernet ports - useless to me - so I have to add a 10G card - QNAP QXG-10G1T, which is only $89 to get the performance. SO - how does this 6 bay compare to a full 8 bay TVS-872XT in real life ?
I have no idea ! I am sitting here with 2 Mac computers, doing basic testing. It "seems great" - but until it's at a facility, I have no idea of how it's actually going to perform.

So when you see me talking about the TVS-872XT, and it's similar sister products like the 672, etc, it's got nothing to do with the Thunderbolt 3 ports. I never use those. I use 10G, and these products KICK A** - especially the TVS-872XT.

I think, as a single user, who is not using video bandwidth, but doing 16 tracks of audio, I have to ASSUME that the TS-h886 will be more than sufficient for your application (provided you stick a
10G QNAP QXG-10G1T) into this product.

So the bottom line here is - should you be getting more than 100 - 200 MB/sec on a TS-473 ? Yea, sure. A TS-873e with the 10G card will do 800 MB/sec so you "should" be able to get 400 MB/sec, but I never tried it on this model. As for their "silly specs" on the website - they are showing 2310 MB/sec using 4 Intel SSD's. I have built all the enterprise QNAP systems - 16 bay, 24 bay, and win a Win 10 PC and a QXG-10G1T, I have NEVER EVER seen more than 1100 MB/sec, even on an empty RAID system that is optomized. So I don't know where those numbers come from.

here is the TVS-872XT spec page
https://www.qnap.com/en-us/product/tvs-872xt
and it shows 1171 MB/sec. using EIGHT SSD's in a RAID 5 configuration. And that is QNAP's own spec. So how anyone ever recorded 2310 MB/sec on an inferior box with only 4 SSD's -
well - you got me there !!!

Bob Zelin
Bob Zelin / Rescue 1, Inc.
http://www.bobzelin.com

Post Reply

Return to “Mac OS”