Update: one day after. I have updated several contents of this post. It may be better to read it again in full !
Dear all,1) For keyword search in files content - Locally on my PC
I was using Copernic Desktop Search (CDS) for years (since V2 I think). A month ago I was still using their V3.7 released a long time ago.
I have bought their V4 more than one year ago but even recently it was too buggy for me. Note when V3 was released years ago, I remember that I had to wait one year so that it was not buggy ! Well, I don't think I can say they care about their customers (note: it costs a one time fee of about $50. And 50% coupon when a new version was released)!
Now I am testing X1 Search V8
for the last 10 days (I remember having tested unsuccessfully X1 when it was with yahoo. Apparently they changed the full code of their program around 2010 which I have missed!). So far I quite happy overall especially since it displays faster in the preview pane xls, pdf and html files
The only problem I had is that it doesn't display correctly accents in the preview pane html files (see http://forums.x1.com....php?f=68&t=9638
With CDS V3.6 size of the index was 85 Go with about 2,000,000 files indexed (Note:
In one hdd drive I even hit the NTFS limit : too much files to handle ! edit: see https://www.donation....msg373167#msg373167
) . It took about 15 days to complete 24/24 7/7.
I still haven't finished indexing all my contents (mainly xls doc ppt mdb csv zip pdf eml msg files) with X1. So far speed of indexing and size of the index seems to be about the same.
edit1: Apparently I have some locally saved email (.eml) that were not indexed. I need to try to find why ! Edit2: I have tried several thing but it won't index some eml files ! I also checked and discovered that some subfolders were not indexed even if X1 says all the index is up to date. I have tried 3 times to re-index and the last time it found alone some yearly folders that it had missed the first time ! I have still some yearly subfolders not indexed ! But I'll try to update that info here. Update : Done. In my case with the same computer and the same data content (about 2,000,000 files) to be indexed, X1 did it in 10 days (5 days less than CDS V3.6) and the size of the index is 52 Go (less 33Go than CDS) !
See edits above! Note;: I had some strange problems yesterday. X1 was stuck and then suddenly it started to use twice the same amount of the size index ! Grrr I moved the index on another HDD from a SSD. I let it finish the indexing and moved back to the SSD where the index size is again the normal size (52Go)! Now I need to test extensively to see if everything is ok with X1 ! I'll try to update again about it here !
I haven't tested dtsearch yet (so I don't know if it is better/faster. Dtsearch index size seems to be 15% the size of the indexed content. http://support.dtsea....com/faq/dts0206.htm
so my guess is that the index will be bigger than with CDS or X1). Price seems high $200 but X1 is $50 +$25 each year.
So maybe I'll give it a try.
In fact after watching some videos about it,
I won't try it
will try it even if I don't use regex for searching keywords, and because the interface seems not very enough user friendly (I don't want to click many times just to do a keyword search !).1 bis) For keyword search in files content - online in the cloud
I have also uploaded my data in Google Drive
(see my experiments here https://www.donation....msg373077#msg373077
). There, it also does index the content of the files with some limits
: for instance :
- index only the first 100 pages of pdf files - But if you open a 1000 pages pdf files and do a keyword search in it, it will find the keyword !
- index only the first million characters of any file (https://support.goog...e/answer/37603?hl=en
- may not be able to open very large xls files (note: I already created Google Drive Sheets close to the 2,000,000 cells limit but it consumed close to the 2Go RAM limit of Firefox 32Bits !). In reality I stay closer to less than 20 MB for xls files.
- doesn't display small extracts of the files (like we see when we do a keyword search in google.com). So you have to open each files to see if it is the document you were looking for !
- you have to wait a few seconds for the UI to preview or open the file.
- doesn't display correctly html files. In fact it displays the html code only !! This is strange as gmail can do so properly without images when you send a joined html file in an email !
If you have millions of small files (html plus their related gif etc files..) it may very difficult (it creates easily orphaned files without telling about it...https://productforum...ns/drive/qM_Wdt6ElRQ
) ! My next goal is to convert my html files in txt so I can do searches on it inside google drive. But I hit another problem : Google drive folders are not folders like in your pc but labels (see https://productforum...ns/drive/qM_Wdt6ElRQ
). It can take ages when you want to upload such many thousands files in Google Drive. It loads somewhere in a server memory all the google drive files, before adding new ones. Plus you can't search yet easily in a folder. 2) For keyword search in filenames only
I also use Everything for file searching. I like their folder filters shortcut (search only in some pre-defined folders).
I also use Listary files searching when I have opened a folder with many files. It can search inside very fast (I use it also for selecting very fast folders that I have previously bookmarked).
Well, as someone wrote hereafter, I think I am closed to limits for indexing all my data ! But day after day, it becomes better !
My computer : win 8.1 64bits french +4cores + hdd + ssd +16Go ram.