Search Speed in PDF-XChange Editor

Forum for the PDF-XChange Editor - Free and Licensed Versions

Moderators: TrackerSupp-Daniel, Tracker Support, Vasyl-Tracker Dev Team, Sean - Tracker, Paul - Tracker Supp, Chris - Tracker Supp, Tracker Supp-Stefan, Ivan - Tracker Software

Post Reply
rah
User
Posts: 10
Joined: Tue Jul 03, 2012 6:17 pm

Search Speed in PDF-XChange Editor

Post by rah » Tue Nov 10, 2015 3:07 pm

I've got a number of large PDFs that I need to search through regularly. The largest so far is 22,000 pages long. I have found when I search it using PDF-XChange Editor, it takes over 2 minutes to find something on the last page if I'm currently on page 1.

By comparison, if I perform the exact same search using PDF-XChange Viewer, it takes about 20 secs.

Why not incorporate the same search function in PDF-XChange Editor that's used in Viewer?

User avatar
Tracker Supp-Stefan
Site Admin
Posts: 13654
Joined: Mon Jan 12, 2009 8:07 am
Location: London
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Tracker Supp-Stefan » Tue Nov 10, 2015 4:55 pm

Hello Rah,

Thanks for the post.
The Editor allows you to do a much more advanced search - and it can search in a lot more of the components of a PDF file (e.g. attachments/document info) - and those extra features will inevitably have negative impact on the search speed, but this should only apply the first time you search that file, consecutive searches (in the same Editor session) should be faster.
Also - this search speed is still 180+ pages a second :)

Regards,
Stefan

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Thu Jan 21, 2016 12:22 am

Tracker Supp-Stefan wrote: and those extra features will inevitably have negative impact on the search speed, but this should only apply the first time you search that file, consecutive searches (in the same Editor session) should be faster.
First let me say that the search speed is excellent and I pose the question more of curiosity and surely not as a complaint. :wink:

Is it possible that the search mechanism regards an Editor session as finished when the Editor is open but I do not use the search for a while?

Here the reason for my question:

I use "Search" (= the function with the advanced search pane) for searching in a file with some thousand pages.
During the first search there is a certain delay as explained by Stefan (usually about 30 seconds). During the following searches there is practically no delay at all if I make them immediately after the first one or within the next 2-3 minutes. The results appear in a second or less than a second. 8)

Then I decide to view a document outside of the XChange Editor (for example a doc file in Word) or to write in this forum ( :wink: ) and therefore I minimize ("close") the XChange Editor to the tray by clicking on the "X" in the upper right corner. (In the preferences I have chosen the option "Close to System Tray"). The pdf file remains open too, the search pane the same. When I return to the Editor (bringing it back from the tray) and I start the next search (in the same file), there is again the delay (perhaps this time a little bit smaller than the first one, but I am not sure about this): I can see from the progress bar that the program searches again all the document from the start to the beginning. This happens only when I did not use the search for about 5 or 6 or 7 minutes I would say (or longer of course). It is like the session would have been interrupted automatically after some (quite short) time.

Has this new delay some inevitable technical reasons or is this a bug?

(The issue has no connection with minimizing the Editor into the tray. As I have seen just now it occurs also when I leave the Editor open on the screen without using it for the time mentioned.)

If the information is of importance:
I do not use for this search any advanced criterion. And in the Options are checked only the elements "Case Sensitive", "Whole Words" (sometimes) and "Include Page Text".
XChange Editor: version 5.5., build 316.0

User avatar
David.P
User
Posts: 869
Joined: Thu Feb 28, 2008 8:16 pm
Location: Germany

Re: Search Speed in PDF-XChange Editor

Post by David.P » Thu Jan 21, 2016 10:40 am

Peter 123 wrote:During the first search there is a certain delay as explained by Stefan (usually about 30 seconds). During the following searches there is practically no delay at all if I make them immediately after the first one or within the next 2-3 minutes. The results appear in a second or less than a second.

Then I decide to view a document outside of the XChange Editor. [...] When I return to the Editor (bringing it back from the tray) and I start the next search (in the same file), there is again the delay [...].

Has this new delay some inevitable technical reasons or is this a bug?
Hi,

I am seeing the same behaviour, PDF-XChange Editor after a while "forgets" the cached information (pre-rendered pages and thumbnails as well as text to be searched). I gather that this is intended, normal application behaviour in Windows. However, for special purposes and/or power users, maybe there should be an option to switch this behavior off, and another option that pre-renders the entire document including thumbnails and search text in a background thread, as soon as the document is opened.

This would speed up the (first) browsing through a document, as well as any (first) search operation considerably, with large documents.

There is a related discussion here:
Preload pages in PDF XChange Viewer

PDF-XChange Editor RAM usage when working with a large document:
Image

Best regards
David.P
David.P
PDF-XChange Pro

User avatar
Tracker Supp-Stefan
Site Admin
Posts: 13654
Joined: Mon Jan 12, 2009 8:07 am
Location: London
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Tracker Supp-Stefan » Fri Jan 22, 2016 4:17 pm

Hi All,

Yes - after a period of inactivity the Editor will release some memory that is no longer actively needed - this is a "good practice" among windows applications. Of course there will always be the power user like yourselves that want to push the program to it's limits :)
I will check with our devs if there are any plans on adding extra options/features to the current behaviour.

Regards,
Stefan

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Fri Jan 22, 2016 9:51 pm

Tracker Supp-Stefan wrote: I will check with our devs if there are any plans on adding extra options/features to the current behaviour.

Thanks, Stefan. :D That would be a great help because whenever you use for example a reference book in form of a pdf file, naturally there will be a lot of consecutive searches but not always within the small search "limit" of about 2 or 3 or 4 minutes. If you are "too late" with your next search - exceeding this limit - the "big search procedure" starts again. Perhaps it is technically possible at least to increase the period of inactivity, let's say to 15 minutes or something like that. That would already be a certain relief for frequent searching. (And the more this period could be increased, the better it would be, of course. :wink:)

Lzcat - Tracker Supp
Site Admin
Posts: 714
Joined: Thu Jun 28, 2007 8:42 am

Re: Search Speed in PDF-XChange Editor

Post by Lzcat - Tracker Supp » Mon Jan 25, 2016 8:11 am

Hi.
Current "period of inactivity" is 10 minutes, so I don't think that 15 minutes will be much better. We will think about some extra options to tune such editor behavior.
Victor
Tracker Software
Project manager

Please archive any files posted to a ZIP, 7z or RAR file or they will be removed and not posted.

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Mon Jan 25, 2016 2:30 pm

Lzcat - Tracker Supp wrote:We will think about some extra options to tune such editor behavior.
:D Excellent. Thanks a lot.

User avatar
Tracker Supp-Stefan
Site Admin
Posts: 13654
Joined: Mon Jan 12, 2009 8:07 am
Location: London
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Tracker Supp-Stefan » Mon Jan 25, 2016 2:34 pm

Pleasure!

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Thu Mar 24, 2016 2:44 am

I tried just now the new version of the PDF-XChange Editor 6.0 (build 317.0).

I am sorry about what I have to say about it:

Concerning the search speed the new version is not only a huge disappointment. I would call it a fiasko.

I have two PDF files for which I use the XChange Editor regularely. This is the search speed within the files (concerning the first search after opening them in the Editor):

File 1:
speed with the old version (5.5., build 316.1): 30 seconds
speed with the new version (6.0., build 317.0): 1 minute and 35-40 seconds!

File 2:
speed with the old version: 7 seconds
speed with the new version: 35 seconds!

Whatever you may have changed in the new version in order to improve the Editor in general - to my mind it cannot justify such an enormous deterioration in the search speed.

The conclusion for me is sad but simple: I will never upgrade to the new version as long as this horrible delay exists. Searching is an essential part of my work with the pdf files.

It is still true that consecutive searches (within the same session of the Editor and without closing in the meantime the PDF file again) are amazingly fast. But this is a merit which offers the old version too.

As I said before: I am sorry that I have to comment the new version in such a negative way, especially because I appreciate that all your team here in this forum is always very responsive, friendly and willing to help. But unfortunately this cannot compensate this horrible step backwards that happened with version 6.0 concerning the search speed. (There are some other disappointments too but this is the most severe one.)

User avatar
Will - Tracker Supp
Site Admin
Posts: 6819
Joined: Mon Oct 15, 2012 9:21 pm
Location: London, UK
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Will - Tracker Supp » Thu Mar 24, 2016 8:35 am

Hi Peter,

Thanks for the post - I'm sorry to hear that and would definitely like to further into this. Can you send the same files that you're searching, so that we can have the same test-basis? If you don't feel comfortable uploading here, please send directly to support@tracker-software.com and provide a link to this topic? Feel free to send to my attention (i.e. RE: Search Speed - Attn: Will). Also, if you could please advise on the specific search terms used and whether you see any difference between Find (Ctrl + F) and the Advanced Search (Ctrl + Shift + F), please let me know.

Thanks,
If posting files to this forum, you must archive the files to a ZIP, RAR or 7z file or they will not be uploaded.
Thank you.

Best regards

Will Travaglini
Tracker Support (Europe)
Tracker Software Products Ltd.
http://www.tracker-software.com

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Thu Mar 24, 2016 10:35 am

Hi Will,

thanks for your friendly answer and your interest in the issue. :D
Will - Tracker Supp wrote: Can you send the same files that you're searching, so that we can have the same test-basis?
Attaching them here was not possible as their size is more than 5 MB. Please download them from these URLs:

"File 1" (the bigger one):
https://onedrive.live.com/redir?resid=A ... file%2czip

"File 2":
https://onedrive.live.com/redir?resid=A ... file%2czip
Will - Tracker Supp wrote:Also, if you could please advise on the specific search terms used
These are my standard search options, with which I also tested the new version: Checked are only "Case Sensitive" (most of the time) and "Include Page Text":

Image

I think you can try with any search term you like (e.g. in File1 with "DOPPELT" and in File 2 with "reckon" [no capitals] in case that you you don't have greek letters on your computer). The Editor anyway performs a complete search from the beginning to the end of the file - and this is the procedure to which I refer with the numbers of seconds mentioned in my above posting (= the time it takes to perform the complete search even if the search term is already found earlier).
Will - Tracker Supp wrote:and whether you see any difference between Find (Ctrl + F) and the Advanced Search (Ctrl + Shift + F), please let me know.
At the moment I cannot tell you exactly about differences because I am back again at the old version while I used the new one only after virtualization. I tried the new version with the Advanced Search (which has become my standard search method). I think one time I used the Find feature and if I remember correctly there was also a big delay (but I did not watch the seconds).

Thanks again for your support.

User avatar
Bhikkhu Pesala
User
Posts: 1772
Joined: Tue May 29, 2007 9:29 am
Location: East London
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Bhikkhu Pesala » Sat Mar 26, 2016 12:37 pm

If the PDF file is large enough the speed difference is very obvious. On a file of 1180 pages it is 21 seconds in the Editor compared to 1 second for the Viewer.

For this smaller file: http://www.aimwell.org/The%20Debate%20o ... ilinda.pdf it is 3 seconds for the Editor and less than 1 second (0 seconds) for the Viewer. (I searched for "Milinda"). The Viewer found 88 entries instead of 87 for the Editor (the extra entry is apparently in Document Information).
Windows 10 64-bit • AMD A10-6800K, 8 Gbyte RAM
Review: http://www.softerviews.org/PDF-XChange.html

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Sat Mar 26, 2016 5:37 pm

Concerning the number of pages I should clarify something for the Tracker team in case that they compare the speed for several PDF files of various sources:

In the 2 PDF-Files I uploaded, the page hight is stretched (in order to reduce the number of page brakes). That means that the number of pages in these files is only about the half it would be with the usual page hight (A 4): e.g. 2500 pages in these PDF files normally would be (roughly calculated) about 5000 pages (!).

This would explain the probably longer search time compared for example with files and page numbers given by Bhikkhu Pesala (above and in the interesting tests reported in the other thread: http://www.tracker-software.com/forum3/ ... 232#p99232 )
(I assume that you tested with "normal" pages in A 4 format.)

User avatar
Bhikkhu Pesala
User
Posts: 1772
Joined: Tue May 29, 2007 9:29 am
Location: East London
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Bhikkhu Pesala » Sat Mar 26, 2016 7:48 pm

My 1180 page test document is simply 10 copies of a 118 page book. It compresses very well with 7-Zip due to the large amount of duplicate content. I removed the pictures to make it smaller, but without removing any text content so that it would be a fair test of raw text strings search speed.

Try it for yourself and see if your results are any different on different hardware.
Attachments
Honour Thy Fathers.7z
(670.89 KiB) Downloaded 99 times
Windows 10 64-bit • AMD A10-6800K, 8 Gbyte RAM
Review: http://www.softerviews.org/PDF-XChange.html

User avatar
David.P
User
Posts: 869
Joined: Thu Feb 28, 2008 8:16 pm
Location: Germany

Re: Search Speed in PDF-XChange Editor

Post by David.P » Tue Mar 29, 2016 4:31 pm

Peter 123 wrote:Concerning the search speed the new version is not only a huge disappointment. I would call it a fiasko.
Same here, unfortunately. Search speed with v.6.0 Build 317.0 (for every first search of a document) takes roughly 10 times as long as with the previous version 5 of the Editor.

:(

Regards
David.P
David.P
PDF-XChange Pro

User avatar
Will - Tracker Supp
Site Admin
Posts: 6819
Joined: Mon Oct 15, 2012 9:21 pm
Location: London, UK
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Will - Tracker Supp » Tue Mar 29, 2016 4:46 pm

Hi guys,

It seems that there are some general issues with the advanced search, as I cannot reproduce the issue with the search speed, due to the fact that the Editor cannot find any entries for any words in the document, at all.

I've created a ticket for this issue and have requested that it be a given a critical priority, given how essential a search feature is.

Sincere apologies for the inconvenience!
If posting files to this forum, you must archive the files to a ZIP, RAR or 7z file or they will not be uploaded.
Thank you.

Best regards

Will Travaglini
Tracker Support (Europe)
Tracker Software Products Ltd.
http://www.tracker-software.com

User avatar
David.P
User
Posts: 869
Joined: Thu Feb 28, 2008 8:16 pm
Location: Germany

Re: Search Speed in PDF-XChange Editor

Post by David.P » Tue Mar 29, 2016 5:09 pm

:o
:D
David.P
PDF-XChange Pro

User avatar
Will - Tracker Supp
Site Admin
Posts: 6819
Joined: Mon Oct 15, 2012 9:21 pm
Location: London, UK
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Will - Tracker Supp » Tue Mar 29, 2016 5:30 pm

:)
If posting files to this forum, you must archive the files to a ZIP, RAR or 7z file or they will not be uploaded.
Thank you.

Best regards

Will Travaglini
Tracker Support (Europe)
Tracker Software Products Ltd.
http://www.tracker-software.com

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Tue Mar 29, 2016 8:05 pm

Many thanks, Will, for your efforts and the straightforward way you inform us about the problem. :D

User avatar
Will - Tracker Supp
Site Admin
Posts: 6819
Joined: Mon Oct 15, 2012 9:21 pm
Location: London, UK
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Will - Tracker Supp » Tue Mar 29, 2016 8:21 pm

No worries Peter, glad to help :)
If posting files to this forum, you must archive the files to a ZIP, RAR or 7z file or they will not be uploaded.
Thank you.

Best regards

Will Travaglini
Tracker Support (Europe)
Tracker Software Products Ltd.
http://www.tracker-software.com

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Thu Apr 21, 2016 1:34 am

Good news:

I just made a short try with version 6.0 build 317.1 of the PDF-XChange Editor.

The search speed is excellent again now, that means it is the the same or even better as it was with version 5.5.316.1. 8)

Many thanks to the team of Tracker for this important fix. :D

John - Tracker Supp
Site Admin
Posts: 8203
Joined: Tue Jun 29, 2004 10:34 am
Location: Vancouver Island - Canada
Contact:

Re: Search Speed in PDF-XChange Editor

Post by John - Tracker Supp » Thu Apr 21, 2016 6:21 am

Thanks Peter - kind words always appreciated ! :)
If posting files to this forum - you must archive the files to a ZIP, RAR or 7z file or they will not be uploaded - thank you.

Best regards
Tracker Support
http://www.tracker-software.com

User avatar
David.P
User
Posts: 869
Joined: Thu Feb 28, 2008 8:16 pm
Location: Germany

Re: Search Speed in PDF-XChange Editor

Post by David.P » Thu Apr 21, 2016 7:44 am

Many thanks also from here. I have instantly switched to v.6 now, for production work :)
Best regards
David.P
David.P
PDF-XChange Pro

John - Tracker Supp
Site Admin
Posts: 8203
Joined: Tue Jun 29, 2004 10:34 am
Location: Vancouver Island - Canada
Contact:

Re: Search Speed in PDF-XChange Editor

Post by John - Tracker Supp » Thu Apr 21, 2016 8:15 am

Cheers David.
If posting files to this forum - you must archive the files to a ZIP, RAR or 7z file or they will not be uploaded - thank you.

Best regards
Tracker Support
http://www.tracker-software.com

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Fri Jun 09, 2017 10:34 pm

Hi, I just wanted to ask if there is some progress concerning the following matter:
Tracker Supp-Stefan wrote: Yes - after a period of inactivity the Editor will release some memory that is no longer actively needed - this is a "good practice" among windows applications. Of course there will always be the power user like yourselves that want to push the program to it's limits :)
I will check with our devs if there are any plans on adding extra options/features to the current behaviour.
Lzcat - Tracker Supp wrote: Current "period of inactivity" is 10 minutes, so I don't think that 15 minutes will be much better. We will think about some extra options to tune such editor behavior.
At the moment the period of inactivity is still 10 minutes. This means unfortunately quite a long search time when the PDF file is rather big and the 10 minutes are over.

Thank you.

User avatar
Will - Tracker Supp
Site Admin
Posts: 6819
Joined: Mon Oct 15, 2012 9:21 pm
Location: London, UK
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Will - Tracker Supp » Sat Jun 10, 2017 4:42 pm

Hi Peter,

Thanks for the post - Not insofar as I'm aware, but I'll see if I can give the topic a little nudge.

Cheers,
If posting files to this forum, you must archive the files to a ZIP, RAR or 7z file or they will not be uploaded.
Thank you.

Best regards

Will Travaglini
Tracker Support (Europe)
Tracker Software Products Ltd.
http://www.tracker-software.com

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Sat Dec 16, 2017 2:37 am

A short time ago I updated to version 7.0.323.1 and I am very glad to see that the shortcoming with the search speed discussed in this thread is resolved now. The enormous search speed within a file can be retained now as long as the user likes, even if the inactivity period is longer than 10 minutes (this was the limit in version 6 - see the posting above: https://www.tracker-software.com/forum3 ... 508#p97508).

You can adjust now the time under Edit --> Preferences --> Performance.

There you will find a drop-down menu called "Maximum lifespan of the cached data that is not being used". From this menu you can choose the time you prefer - from 10 minutes to 7 days and to "Maximum" (which obviously means: as long as you have opened the file in the Editor):

Image

In order to achieve the desired effect you will have to determine there the time period you want because the default is "Auto", which seems to be a quite short period according with my initial test (even less than 10 minutes I assume).

That's really a very nice and useful feature which improves enormously the search procedure within the Editor.

Many thanks to the developers and to those members of the staff who decided to implement this feature. :D

User avatar
Will - Tracker Supp
Site Admin
Posts: 6819
Joined: Mon Oct 15, 2012 9:21 pm
Location: London, UK
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Will - Tracker Supp » Sat Dec 16, 2017 8:30 pm

No worries Peter, we're always glad to offer improvements where possible and glad that you like them! Thanks for the instructions too, very helpful :)
If posting files to this forum, you must archive the files to a ZIP, RAR or 7z file or they will not be uploaded.
Thank you.

Best regards

Will Travaglini
Tracker Support (Europe)
Tracker Software Products Ltd.
http://www.tracker-software.com

Timur Born
User
Posts: 617
Joined: Tue Jun 26, 2012 1:50 pm

Re: Search Speed in PDF-XChange Editor

Post by Timur Born » Mon Dec 25, 2017 10:29 am

First of all, thanks for the improvements, they are very well appreciated!

The cache clearing process is rather aggressive, though. I see cache drops of several GB once the cache is filled. 10% to 20% should suffice, should it not? Unfortunately, once you close the session everything is gone, because the disk cache doesn't hold any of this data.
Cache_clear_V7.jpg
Maximum cache size can be exceeded in some circumstance, which in the above graph happened with the first peak (11.5 GB maximum set, over 13 GB reached). The second peak kept memory usage limited to the set maximum (see how the curve does very small up and down before it plummeted down to 50%).

Lzcat - Tracker Supp
Site Admin
Posts: 714
Joined: Thu Jun 28, 2007 8:42 am

Re: Search Speed in PDF-XChange Editor

Post by Lzcat - Tracker Supp » Tue Dec 26, 2017 8:30 am

Hi Timur.
The cache clearing process is rather aggressive, though. I see cache drops of several GB once the cache is filled. 10% to 20% should suffice, should it not?
Looking per your graph - looks like you close some documents, and we release all cached data for it. Other possibility - you simply forgot to change cached items lifetime (by default it is very small). Cleaning mechanism is not so aggressive as you suppose, for illustration please see plateau in the middle of your graph, this is normal behavior. Of course in some scenarios we may release more memory that is needed to fit into cache limit, but making cleaner precise is too expensive in cost of CPU usage (or in cost of memory usage and code complexity).
Unfortunately, once you close the session everything is gone, because the disk cache doesn't hold any of this data.
Of cource everything is gone, this is by design and I don't belive that this will be changed in nearest future. What you can do - do not close Editor and user Sleep/Hibernate instead shutdown.
HTH.
Victor
Tracker Software
Project manager

Please archive any files posted to a ZIP, 7z or RAR file or they will be removed and not posted.

Timur Born
User
Posts: 617
Joined: Tue Jun 26, 2012 1:50 pm

Re: Search Speed in PDF-XChange Editor

Post by Timur Born » Tue Dec 26, 2017 9:40 am

Thanks for the reply and: Happy holidays! ;)
Lzcat - Tracker Supp wrote:Hi Timur.
Looking per your graph - looks like you close some documents, and we release all cached data for it.
No documents were closed during the whole session. I just opened documents and scrolled through them to have them cached and thus hit (over) the limit of allowed cache memory.
Other possibility - you simply forgot to change cached items lifetime (by default it is very small).
Lifetime was set to "Maximum".
Cleaning mechanism is not so aggressive as you suppose, for illustration please see plateau in the middle of your graph, this is normal behavior. Of course in some scenarios we may release more memory that is needed to fit into cache limit, but making cleaner precise is too expensive in cost of CPU usage (or in cost of memory usage and code complexity).
As you can see the plateau was an exception, though, while the massive drops were the norm. And right after the plateau cached memory dropped from 11.5 GB to around 6 GB, which is close to a 50% drop. Then I rebuild the cache to its maximum, just to have it dropped down to half again. When this happens Editor isn't usable for several seconds while memory is cleared, even on my 16 logical cores CPU.
Unfortunately, once you close the session everything is gone, because the disk cache doesn't hold any of this data.
Of cource everything is gone, this is by design and I don't belive that this will be changed in nearest future. What you can do - do not close Editor and user Sleep/Hibernate instead shutdown.
HTH.
I know that this is by design, which doesn't make it less unfortunate. Especially when the reasoning behind that design is that you don't want to waste users' disk space for caching, but at the same time you are asking me to keep 12 GB of RAM permanently occupied just to retain the cache data.

Currently the disk cache feature is practically useless, would it be fully implemented then text searches (and scrolling through complex documents) could be just as fast every time you open the same document after the cache was loaded from disk. I don't see why memory cache content for unchanged documents cannot be written to disk for later use, it would make working with repeatedly opened documents so much more convenient.

Disk space = no problem, disk speed = slight problem, memory space = problem, single-core processing power = big problem.

Timur Born
User
Posts: 617
Joined: Tue Jun 26, 2012 1:50 pm

Re: Search Speed in PDF-XChange Editor

Post by Timur Born » Tue Dec 26, 2017 10:47 am

One of my regularly used documents makes Editor use about 100 mb when it's freshly opened. Searching for a single word in this document takes 9 seconds when performed for the first time, memory usage then increases to 500 mb. Subsequent searches return instant results and don't increase memory usage any further.

If those extra 400 mb were cached to disk and loaded in the background after the document was opened then there would be no delay for searches once the pre-cached data is back in memory.

Worth mentioning: Creating a SHA-256 hash of said document takes less than 1 second on my system, more like instantly.

Lzcat - Tracker Supp
Site Admin
Posts: 714
Joined: Thu Jun 28, 2007 8:42 am

Re: Search Speed in PDF-XChange Editor

Post by Lzcat - Tracker Supp » Tue Dec 26, 2017 2:43 pm

Hi Timur.
Looks you are making stress testing, but not real work. Caching and cleaning mechanism works not as you expect, and it will not work as you expect until you write it yourself (and even this will not guarantee that it will work as you need). I may explain what and how is cached, but I'm not sure that it will be worth it and it will take a significant amount of time. If you have real workflow scenarios where caching is not working well and this has serious impact on performance - we will be happy to check and see what can be done.
Regarding disk cache - your points are noted, and I already explained why we do not want to use disk cache for such scenarios.
As I have said before the Editor is not developed for a specific use case only, and we try to realize something that will satisfy the needs of many people and usage scenarios. In many cases this means that we must find compromises between productivity, memory usage and user requirements. And often user requirements are opposite/conflicting. Some people want to minimize CPU/memory usage and rid off disk usage, others what to use their brand new hardware with 300% efficiency. We provide some settings to customize the Editor behaviour, but we can not implement a custom version of the Editor for each user, sorry.
HTH.
Victor
Tracker Software
Project manager

Please archive any files posted to a ZIP, 7z or RAR file or they will be removed and not posted.

Timur Born
User
Posts: 617
Joined: Tue Jun 26, 2012 1:50 pm

Re: Search Speed in PDF-XChange Editor

Post by Timur Born » Tue Dec 26, 2017 9:42 pm

If you say that opening the same (big) unchanged document over and over again is a special usage case, then I see your point. No disk caching for search results then.

Timur Born
User
Posts: 617
Joined: Tue Jun 26, 2012 1:50 pm

Re: Search Speed in PDF-XChange Editor

Post by Timur Born » Tue Dec 26, 2017 9:46 pm

Usage case: Several documents open, doing some searches here and there, flipping pages, 25% memory cache size, maximum lifespan set. Then all of a sudden this happened while flipping some more pages.
Cache_clear_V7_2.jpg

Lzcat - Tracker Supp
Site Admin
Posts: 714
Joined: Thu Jun 28, 2007 8:42 am

Re: Search Speed in PDF-XChange Editor

Post by Lzcat - Tracker Supp » Wed Dec 27, 2017 11:11 am

Hi Timur.
If you say that opening the same (big) unchanged document over and over again is a special usage case, then I see your point. No disk caching for search results then.
Yes, this is special usage case. I don't know which document you are using and for what purposes. As for me search working fast enough even first time (3-4 seconds for more than 1000 pages is acceptable), and even all caches will be dropped every 5-10 minutes - it is not big deal for me (in most cases I run search 3-5 times for different terms, and next time I may search again in 20 minutes or later, so in worst case I'll lose maximum 96 second per working day - much less than writing this message).
Usage case: Several documents open, doing some searches here and there, flipping pages, 25% memory cache size, maximum lifespan set. Then all of a sudden this happened while flipping some more pages.
As I can see from graph you have used a lot of memory for caching in a short period of time, than was long idle period, and after all you again started work with non-cached data. If my assumptions are correct this is expected behavior - caching is time based, and after long idle period cleaner may release even all old cached data at one pass - depend when they was used last time. This is known weak point of caching mechanism.
HTH.
Victor
Tracker Software
Project manager

Please archive any files posted to a ZIP, 7z or RAR file or they will be removed and not posted.

Timur Born
User
Posts: 617
Joined: Tue Jun 26, 2012 1:50 pm

Re: Search Speed in PDF-XChange Editor

Post by Timur Born » Wed Dec 27, 2017 11:37 am

Lzcat - Tracker Supp wrote:Yes, this is special usage case. I don't know which document you are using and for what purposes. As for me search working fast enough even first time (3-4 seconds for more than 1000 pages is acceptable),
On an overclocked Ryzen 1800X it takes 9 seconds to search through 578 pages. These are RPG manuals that include lots of images and usually two columns on each page. Since the memory usage of Editor increases by 400 mb just for text searching these documents seem quite demanding.
and even all caches will be dropped every 5-10 minutes - it is not big deal for me (in most cases I run search 3-5 times for different terms, and next time I may search again in 20 minutes or later, so in worst case I'll lose maximum 96 second per working day - much less than writing this message).
When five people are waiting for you to come up with an answer for a rules question while you are trying to keep a game-play flow then these seconds begin to stack.
As I can see from graph you have used a lot of memory for caching in a short period of time, than was long idle period, and after all you again started work with non-cached data. If my assumptions are correct this is expected behavior - caching is time based, and after long idle period cleaner may release even all old cached data at one pass - depend when they was used last time. This is known weak point of caching mechanism.
So "lifespan" is only used until the cache reaches its limit, at which point any "old" (as in minutes) data can and will be flushed aggressively?

To give you a better idea on memory demands of these documents: Opening the single 578 pages document in Editor uses 100 MB RAM, flipping through only ten (10!) pages adds at least another 100 MB on top of that, searching for a single word adds another 400 MB. At this point I barely just started using the document. Flip through another 20 pages and memory usage for this single document already reaches 800 MB. And then I open several other documents of similar graphical complexity, RAM usage hits 2 GB pretty quickly. This is not a stress test, but just working with these documents.

Every time the RAM cache is flushed I not only have to wait for searches to get quick again, but also have to wait for pages being flipped (single-core bottlenecks with images). Last but not least Editor flushing GB of data takes several seconds on its own.

PS: Is there a way to get one search result window per tab or have its content switch with tabs, or do I have to put every document in its own window for that?

John - Tracker Supp
Site Admin
Posts: 8203
Joined: Tue Jun 29, 2004 10:34 am
Location: Vancouver Island - Canada
Contact:

Re: Search Speed in PDF-XChange Editor

Post by John - Tracker Supp » Wed Dec 27, 2017 8:56 pm

Unless we receive other reports of memory issues in 'real working' situations and examples then we do not intend to investigate this further - but for now we are satisfied that there is nothing extraordinary going on here that is of an unreasonable nature and satisfied that in realistic working situations users will not experience unrealistic delays or unreasonable demands on their systems.
If posting files to this forum - you must archive the files to a ZIP, RAR or 7z file or they will not be uploaded - thank you.

Best regards
Tracker Support
http://www.tracker-software.com

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Thu Dec 28, 2017 2:03 am

Just a short question thrown in, concerning the following:
Lzcat - Tracker Supp wrote:[...] caching is time based, and after long idle period cleaner may release even all old cached data at one pass - depend when they was used last time. This is known weak point of caching mechanism.
Does this apply also when choosing the option "Maximum" for "Maximum lifespan of the cached data that is not being used"?

Or my question in other words:
When this is the scenario:
- Option "Maximum" is chosen for maximum lifespan
- Editor and document remain opened (after having performed at least one search within the document)
- there is a longer idle period (e.g. some hours or a day) while this document (or the Editor as a whole) was not used.

Can it happen that in this case cached data is lost and therefore the first (new) search (after the idle period) within this document has to start "from scratch" again (= slower search speed)?

I ask in order to understand the lifespan mechanism but also for a practical reason: I have the impression that what I mentioned just now possibly has happened to me two or three times.*

*) I write "possibly" as I am not sure that it was really my scenario. Because I cannot exclude the possibility that I simply forgot to make a "first search" immediately after opening a document (as I usually do) and later I wonder about the slow search speed for what I regard my second (or third etc.) search while in reality it is the first one. :wink: - In this case of course the delay would be natural and would have been "caused" by myself.

Lzcat - Tracker Supp
Site Admin
Posts: 714
Joined: Thu Jun 28, 2007 8:42 am

Re: Search Speed in PDF-XChange Editor

Post by Lzcat - Tracker Supp » Thu Dec 28, 2017 9:46 am

Hi Peter.
Does this apply also when choosing the option "Maximum" for "Maximum lifespan of the cached data that is not being used"?
This apply no matter which lifetime is set. Looks like I need explain more.
We release cached data when one of two conditions occur:
1. It was last time used more then "Maximum lifespan..." ago (yes, lifetime is calculated since last moment of usage, not creation). In this case it will be released regardless memory limits. This was the problem when maximum lifespan was hardcoded to 10 minutes for many usage scenarios, and now this can be changed.
2. When memory limit for process is reached. In this case we determine oldest cached data, calculate its lifetime and delete all cached data which lifetime is greater than some percent of lifetime of oldest object. This may release even almost all cached data.
For example you started work with Editor, open some documents and use about 99% of memory, avail to Editor. At this moment no data will be release since limit is not exceeded. Assume that you did this during 10 minutes, and then do not touch Editor for example 2 hours (120 minutes). And if you will try to open some document after this idle period in most cases you will exceed memory limit. In this case oldest data lifetime is 130 minutes, youngest - about 0 (several seconds maximum). So if Editor will try release cached data older than 90% of maximum lifetime it will release all data, allocated during first 10 minutes (actually 13 minutes), and you will se that almost all cached data is dropped. If you are intensively working with PDF files this is not a problem because there will be no such long idle period (it must be several times longer that working time to see such effect).
Hope this will explain why Editor sometimes drop greater amount of cached data that needed to fit memory limits. And again, I don't see a big problem if some data is dropped from cache once or twice per day, in most cases you will not notice whether data was taken out from the cache or not.
HTH.
Victor
Tracker Software
Project manager

Please archive any files posted to a ZIP, 7z or RAR file or they will be removed and not posted.

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Thu Dec 28, 2017 3:31 pm

Hi Victor,

many thanks for your detailed explanations. I understand better now how it works and that because of the second condition it is not such a simple mechanism as I originally imagined.

May I add some questions in order to assure that I understood some things correctly?
Lzcat - Tracker Supp wrote: We release cached data when one of two conditions occur:
1. It was last time used more then "Maximum lifespan..." ago (yes, lifetime is calculated since last moment of usage, not creation).[...] This was the problem when maximum lifespan was hardcoded to 10 minutes for many usage scenarios, and now this can be changed.
a) Does "last moment of usage" refer to the specific document or to the Editor as a whole?
For example: I have opened 3 documents (A, B, C). In order to secure the "quick search" for document A (= no new search "from scratch") must I have used this document A within the determined maximum lifespan, or is it enough to have used any of these documents (not necessarily document A)?

b) Does "usage" mean to run a search? Or is it enough to scroll within the document or to jump to another position within it etc.?


Concerning the second condition:
Lzcat - Tracker Supp wrote: We release cached data when one of two conditions occur:
[...]
2. When memory limit for process is reached.
c) Is this the limit I can increase (or decrease) by changing (in the "Performance Options") the values for "Use system memory (in percentage points) / (in megabytes)"?

Image

Lzcat - Tracker Supp wrote: For example you started work with Editor, open some documents and use about 99% of memory, avail to Editor. At this moment no data will be release since limit is not exceeded. Assume that you did this during 10 minutes, and then do not touch Editor for example 2 hours (120 minutes). And if you will try to open some document after this idle period in most cases you will exceed memory limit.
d) This applies (only) when I will indeed open a new document after the idle period? Or may the memory limit be exceeded also when I continue to work with the documents which were already opened during the idle period? (E.g.: 2 hours idle period. Then I run a new search within a document which was opened all the time.)

Lzcat - Tracker Supp
Site Admin
Posts: 714
Joined: Thu Jun 28, 2007 8:42 am

Re: Search Speed in PDF-XChange Editor

Post by Lzcat - Tracker Supp » Thu Dec 28, 2017 5:52 pm

Hi Peter.
a) Does "last moment of usage" refer to the specific document or to the Editor as a whole?
A bit weird question ... Each cached object must have its own time mark, otherwise editor will be not able to handle large documents. So actual cache is a lot of different objects, each with its own time mark. And when editor must free some memory it free oldest objects (objects that was not used more time will be released first), regardless the document. Same with maximum lifetime - it is object based, so to extend object lifetime span you should use it.
b) Does "usage" mean to run a search? Or is it enough to scroll within the document or to jump to another position within it etc.?
Editor use different data levels for caching, and search and page display use different structures.
For example to make search on page you must analyze page content (1), extract text (2) and only then proceed with search (3). In most cases most heaviest parts are first two, so if you have cached result after stage 2 search is fast enough (almost instant). Editor cache results of both stage 1 and stage 2, but search itself use only results of stage 2, so if you run search several times it will extend lifetime only for results of stage 2.
To display page on screen you must run three other stages: analyze page content (1), rasterize it (2) and only then flip rasterized content to screen (3). And again first two stages are heavy, so Editor cache their results. So if you navigate document forward, and then return to previously rendered page it renders almost instantly (while cached data is alive). But if you change page zoom you may notice that rendering will again take some time (but less that first time, since we must redo only stages 2 and 3 with different parameters).
So answering you question - to keep cached data alive you must use them. If you want to keep rendered pages cache you should redraw them, if you want to keep search speed - you should use search.
c) Is this the limit I can increase (or decrease) by changing (in the "Performance Options") the values for "Use system memory (in percentage points) / (in megabytes)"?
Yes, it is.
d) This applies (only) when I will indeed open a new document after the idle period? Or may the memory limit be exceeded also when I continue to work with the documents which were already opened during the idle period? (E.g.: 2 hours idle period. Then I run a new search within a document which was opened all the time.)
Se my explain to first quote. Each caching data element are independent and equals in priority, so adding anything new to cache may cause that old data will be released. In your case subsequent search will not add anything new to cache (all was cached before), it will just reset time marks for affected cache elements. But if you navigate to some non-visited page this will add page rasterized content to cache and this may cause that oldest data will be released. For example you have did search for document, then scroll thru document and fill cache with rasterized page data. In this case data used to search is older than rasterized pages and will be released before them when memory limit is reached. But if you search for one term, navigate to several pages, then search and navigate again, and so on data used to search will be used once and once again and should not be released from cache. Anyway it almost impossible to predict what will be released earlier because it depends on set memory limits, document content and user actions. There is only one strict rule: longer cache data is not used - sooner it will be released.
HTH.
PS. I still do not understand why caching DETAILS so bother you and Timur? It is works? Fine. Not always works as you expect - well, Editor is complicated product, and have caching working most time is much better that do not have it at all, isn't it? Some things may not work correctly (for example too short maximum lifetime span without ability to change it) and they should be fixed - they affect too much usage scenarios. Other things are not so obvious, sometimes they are tradeoff of other limitations and it is even hard to explain why they working that way. Improve things is much harder, so we prefer do not make serious changes without serious reasons, and saving several seconds per working day for one person instead of improving other things may not worse it.
Victor
Tracker Software
Project manager

Please archive any files posted to a ZIP, 7z or RAR file or they will be removed and not posted.

Peter 123
User
Posts: 112
Joined: Mon Jun 25, 2012 11:47 pm

Re: Search Speed in PDF-XChange Editor

Post by Peter 123 » Thu Dec 28, 2017 9:19 pm

Thanks again, Victor, for your patience and all your explanations.
Lzcat - Tracker Supp wrote: PS. I still do not understand why caching DETAILS so bother you and Timur?
As for my person: My aim is to retain the fast search speed as long as possible, especially in a quite large document where I take a look once in a while but not permanently - that means that there are always idle periods between. It is a dictionary with some thousand pages. The first search in this document takes about 30 seconds. Still a good time for such a big file, but nevertheless it is extremely comfortable when the following searches take place within one second or even less. 8)

Now thanks to your explanations I understand that this cannot always be achieved (not even with "Maximum" or "7 days" as chosen "maximum lifespan") but that on the other hand there are also some measures which can help to extend the time for which "fast search" is available:
Lzcat - Tracker Supp wrote:to keep cached data alive you must use them. If you want to keep rendered pages cache you should redraw them, if you want to keep search speed - you should use search.
Lzcat - Tracker Supp wrote:There is only one strict rule: longer cache data is not used - sooner it will be released.
Lzcat - Tracker Supp wrote:
c) Is this the limit I can increase (or decrease) by changing (in the "Performance Options") the values for "Use system memory (in percentage points) / (in megabytes)"?
Yes, it is.
And it is also good to hear from the expert that it is
Lzcat - Tracker Supp wrote: almost impossible to predict what will be released earlier because it depends on set memory limits, document content and user actions.
So I know that all is o.k., even when a new search takes the "full time".

In any case I agree with you when you write:
Lzcat - Tracker Supp wrote:caching working most time is much better that do not have it at all, isn't it?
Of course. No doubt. :wink:

As I have already written in an earlier posting: The possibility to adjust the maximum lifespan is a great feature and an enormous help for searching, especially in bigger documents.

Timur Born
User
Posts: 617
Joined: Tue Jun 26, 2012 1:50 pm

Re: Search Speed in PDF-XChange Editor

Post by Timur Born » Fri Dec 29, 2017 9:05 am

Lzcat - Tracker Supp wrote:PS. I still do not understand why caching DETAILS so bother you and Timur? It is works? Fine. Not always works as you expect - well, Editor is complicated product, and have caching working most time is much better that do not have it at all, isn't it?
Knowing the details allows us to workaround the restrictions. A single page being flipped can lead to gigabytes worth of cache data being flushed once the maximum size it reached, that's what I called "aggressive" cache management before.

Now we know to increase the cache size (and buy more RAM) to circumvent this situation and we know to hit the search button once in a while to make sure text search data remains cached. Maybe text search data should be higher prioritized compared to rendered pages when it comes to caching?! Maybe offer it as an option.

Average CPU power consumption during a new search can be over 60 Watts on my system. In comparison a cached search only draws a very short peak of power.

All that being said: XChange Editor still is among the fastest searching PDF viewers out there and my main grief is rather with search results presentation, not speed.

User avatar
Patrick-Tracker Supp
Site Admin
Posts: 1668
Joined: Thu Mar 27, 2014 6:14 pm
Location: Vancouver Island
Contact:

Re: Search Speed in PDF-XChange Editor

Post by Patrick-Tracker Supp » Fri Dec 29, 2017 8:07 pm

Hello,
I have spoken to our dev team and they are considering ways to improve this without sacrificing rendering performance.
Thank you for your suggestions.

Happy New Years!
If posting files to this forum, you must archive the files to a ZIP, RAR or 7z file or they will not be uploaded.
Thank you.

Cheers,

Patrick Charest
Tracker Support North America

Post Reply