Fix for Qsirch and Spotlight

j-beda
Starting out
Posts: 12
Joined: Thu Feb 10, 2011 10:44 am

Re: Fix for Qsirch and Spotlight

Post by j-beda » Wed Oct 21, 2020 10:33 pm

Samba seems to have had hooks for it for quite a while, see https://wiki.samba.org/index.php/Spotlight for either "Elasticsearch" or "Gnome Tracker"

It would be nice if this was just a checkbox we could select in the QNAP Samba settings.

mousebat
Starting out
Posts: 23
Joined: Wed Feb 12, 2020 10:39 pm

Re: Fix for Qsirch and Spotlight

Post by mousebat » Wed Oct 21, 2020 10:42 pm

Spotlight support has been with Samba for years using Gnome Tracker. This is news however because Samba has migrated to using ElasticSearch and FSCrawler as the engine.

I'm pretty sure QNAP will compile Samba exactly how they want it, they might be a little obstinate about some things but they will be aware of this.

Mojave support hmm... Hold my beer...

j-beda
Starting out
Posts: 12
Joined: Thu Feb 10, 2011 10:44 am

Re: Fix for Qsirch and Spotlight

Post by j-beda » Wed Oct 21, 2020 10:48 pm

With info gleaned from https://care.qumulo.com/hc/en-us/articl ... and-Qumulo I just started testing to see if I could get my local machines creating local indexes of the remotely mounted volumes:

You can check the index status of a volume with the following command:

Code: Select all

sudo mdutil -s /Volumes/volume_name
Output of the command:
/Volumes/volume_name:
Indexing enabled.
Enable Spotlight indexing on a volume

Code: Select all

sudo mdutil -i on /Volumes/volume_name
Output of the command:
/Volumes/volume_name:
Indexing enabled.
Disable indexing of a volume

Code: Select all

sudo mdutil -i off /Volumes/volume_name
Output of the command:
/Volumes/volume_name:
Indexing disabled.

Delete an index file and force a re-indexing of a volume

Code: Select all

sudo mdutil -E -i off /Volumes/volume_name && sudo mdutil -i on /Volumes/volume_name

mousebat
Starting out
Posts: 23
Joined: Wed Feb 12, 2020 10:39 pm

Re: Fix for Qsirch and Spotlight

Post by mousebat » Wed Oct 21, 2020 10:58 pm

Indexing the drives manually will work but if you work in a networked environment with multiple client machines connecting to the drive it will be a nightmare to keep up with.

I've added Mojave Support to the QsirchMac. It's on Github now :)

mousebat
Starting out
Posts: 23
Joined: Wed Feb 12, 2020 10:39 pm

Re: Fix for Qsirch and Spotlight

Post by mousebat » Wed Oct 21, 2020 11:12 pm

Whoops - forgot to change build target. Have updated GitHub again.

j-beda
Starting out
Posts: 12
Joined: Thu Feb 10, 2011 10:44 am

Re: Fix for Qsirch and Spotlight

Post by j-beda » Wed Oct 21, 2020 11:13 pm

Wonderful! I will check out the Mojave option.

I only have one person complaining about the search being broken, so if I can keep them happy I might not need to solve it for everyone (but maybe they just don't know enough to complain....)

mousebat
Starting out
Posts: 23
Joined: Wed Feb 12, 2020 10:39 pm

Re: Fix for Qsirch and Spotlight

Post by mousebat » Wed Oct 21, 2020 11:25 pm

:lol: All my staff complain incessantly if there is an interruption to their workflow. It caused me so much of a headache that I decided to learn objective C, create the app and shut them up.

A while ago I trialed using a box running Ubuntu Server with Samba and Gnome Tracker for Spotlight support (after solving the DBUS issue) but it was never quite right and in the (hopefully unlikely) event of me dying, no one would know how to administrate it or fix the bleeding thing if it fell over. At least with an off the shelf NAS they can RTFM after they've put me in the ground.

j-beda
Starting out
Posts: 12
Joined: Thu Feb 10, 2011 10:44 am

Re: Fix for Qsirch and Spotlight

Post by j-beda » Wed Oct 21, 2020 11:27 pm

Can I hold that beer again?
Screen Shot 2020-10-21 at 11.26.02 AM.png
You do not have the required permissions to view the files attached to this post.

mseewald
Starting out
Posts: 20
Joined: Sat Nov 11, 2017 9:38 pm

Re: Fix for Qsirch and Spotlight

Post by mseewald » Wed Oct 21, 2020 11:30 pm

mousebat wrote:
Wed Oct 21, 2020 11:25 pm
:lol: All my staff complain incessantly if there is an interruption to their workflow. It caused me so much of a headache that I decided to learn objective C, create the app and shut them up.
:lol:

Please let me ask back regarding one of your earlier post: You mentioned you could fix the issue by adding a user "_spotlight". I tried this also on my system but could not get it to work. Can you confirm that this ultimately did not turn out to be robust?

mousebat
Starting out
Posts: 23
Joined: Wed Feb 12, 2020 10:39 pm

Re: Fix for Qsirch and Spotlight

Post by mousebat » Wed Oct 21, 2020 11:33 pm

Yes - scratch that. It seemed to work at the time but have no idea why it worked at the time. Looking back in hindsight it was wishful thinking, there is no _spotlight user on the qnap

mousebat
Starting out
Posts: 23
Joined: Wed Feb 12, 2020 10:39 pm

Re: Fix for Qsirch and Spotlight

Post by mousebat » Wed Oct 21, 2020 11:51 pm

Ok - I've jumped the gun a bit with building the app for Mojave compatibility. :ashamed:

I thought it would be an easy job but there is actually a lot of problems to fix to make that work.

I'm sorry but without me sitting down and rewriting it ain't going to happen any time soon.

Re-uploading the Catalina only app. Anyone on a pre2012 mac, please accept my sincere apologies for getting your hopes up.

j-beda
Starting out
Posts: 12
Joined: Thu Feb 10, 2011 10:44 am

Re: Fix for Qsirch and Spotlight

Post by j-beda » Thu Oct 22, 2020 12:04 am

Not a problem.

mseewald
Starting out
Posts: 20
Joined: Sat Nov 11, 2017 9:38 pm

Re: Fix for Qsirch and Spotlight

Post by mseewald » Thu Oct 22, 2020 5:15 am

mousebat wrote:
Wed Oct 21, 2020 10:42 pm
I'm pretty sure QNAP will compile Samba exactly how they want it, they might be a little obstinate about some things but they will be aware of this.
It's actually very interesting to reflect on what QNAP are doing here ..

We know or we can find out looking at the operating system, that Qsirch includes elasticsearch, tesseract (OCR), and tika (metadata extraction). QNAP's youtube video with the "Spotlight demo" is showing that samba shares are supposed to work. So, overall, it seems highly likely that QNAP's Qsirch is essentially a custom version of elasticsearch / tika / tesseract (OCR) / ... plus some kind of web search interface .. and it's supposed to deliver search results to Spotlight via the native samba/ elasticsearch interface.

And the reason that it currently does not reliably work, may be because samba supports the elasticsearch backend only starting with v4.12 .. and QTS distributes samba v4.10.18. So either, the support for Spotlight is missing in that version or it is included but doesn’t work in a robust fashion.

Is anyone brave enough to try moving QNAP's samba from 4.10.18 to 4.12.x? There's a good chance that Spotlight may start to work.

mousebat
Starting out
Posts: 23
Joined: Wed Feb 12, 2020 10:39 pm

Re: Fix for Qsirch and Spotlight

Post by mousebat » Thu Oct 22, 2020 5:54 am

There is a gap in my knowledge of the process. I don’t know how samba translates from elasticsearch to whatever native api spotlight is expecting. I’ll have to trawl through the source to find out.

It’s still a kick in the teeth that QNAP can get away with selling there products as working with Finder search integration when it is well and truly borked!

mseewald
Starting out
Posts: 20
Joined: Sat Nov 11, 2017 9:38 pm

Re: Fix for Qsirch and Spotlight

Post by mseewald » Thu Oct 22, 2020 7:59 pm

mousebat wrote:
Thu Oct 22, 2020 5:54 am
I don’t know how samba translates from elasticsearch to whatever native api spotlight is expecting.
I am not sure you would even need to go there. If Qsirch puts metadata into elasticsearch in the format as samba is expecting it, then we would just need to a newer version of samba and a few more lines in smb.conf
Over weekend, I should have time to pull some data from QNAP's elasticsearch. I have a working setup for samba/fscrawler/elasticsearch to compare with...

Post Reply

Return to “Qsirch & Qfiling”