Problem: CHS consistently very slow to display clips. Need workaround or fix.

[IainB]

For a few years now, and on different laptops with different Windows OSes, I have noticed that, when scrolling down the rows in the Grid area, CHS is very slow - frustratingly so - to display the content of clips in the Memo area.
Having recently migrated to a higher spec laptop with an Intel i7 processor with Win10-64 Pro, 16GB of RAM and with a slightly "pruned" system overhead, this characteristic slowness of CHS has become (relatively) painfully slow - it seems slow-as-molasses by comparison with everything else that is going on. I think I have referred to this CHS sluggishness, on and off, over time, in DC Forum posts relating to CHS, or in comparisons with other Clipboard Managers - e.g., NoteFrog.

Is there a workaround or fix for this CHS characteristic?
I have so far been unable to figure out what is, or might be causing it.

My CHS Clip Database details:

• The clip database is not especially large, having 566 rows with the majority marked as "Favorite".
• 92 of the clips are images, with Clip Format=1), the rest are text, most of which are Clip Format=0, but a few are merged Text clips with Clip Format=NULL and displayed in the colour blue in the Grid.
• There are some older (2010, 2012) text and merged text clips that have Clip Format=NULL, but are displayed in black text in the Grid.
• (I'm not sure, but I thought that some of the text colour might be due to some Virtual Folder sorting in SQL or dragging and dropping into folders.)
• Some of the image clips have the image (as usual) in the Clip Image tab, and also have text notes in addition to the text of the clip's image .PNG Path/Filename.
• The text clips typically range in size between a few bytes to around 48KB.
• The largest clips are the image clips, which range in size between a few bytes to 1.29MB

To keep things as trim as possible:

• I have deleted all the clips I don't want to keep, but that seemed to have no discernible effect on making things any quicker.
• I have used the Options | Backup Maintenance | Verify, Repair, Optimize Database, then I have closed CHS and restarted it, but that seemed to have no discernible effect on making things any quicker.
• I regularly empty the CHS Recycle Bin, but that seems to have no discernible effect on making things any quicker.

CHS is v2.36.0 (Beta) - Configuration is as "Portable":

• CHS is installed in: C:\UTIL\Windows utilities\FindAndRunRobot\Plugins\Clipboard Help+Spell
• The ConfigDir.ini file is installed in that directory also, with the single line PORTABLE=TRUE enabled.
• There is no other ConfigDir.ini file on the disk, for CHS.
• The Database folder and Backup folder are currently in the same path as above. I have tried putting them elsewhere, and have checked for any restrictive file security properties in the path, but have been unable to find anything that could account for the slowness. There have been no read/write protection or lock errors or anything like that. Apart from the slowness mentioned, CHS seems to be working just fine, otherwise.

[mouser]

First a quick comment, and then ill come back and we can try to figure out what is going wrong: There is no reason that a CHS database as small as 566 rows should be anything but blazing fast.
So i'm hopefull we can get to the bottom of your problem and fix it.

[IainB]

@mouser: Thanks. I also reckoned that it should be blazingly fast, and I am aware that a 560-row clip database is trivial/small (no significant load) for the technology that is manipulating it.
I felt sure that it must be attributable to something peculiar to my configuration, but i can't for the life of me figure out what, and it seems to have persisted across laptops and OSes and with different sets of clip data in the database. I bet it turns out to be simple.
The persistent and erratic lag in clip display is almost like something is looping, waiting for something else - some other process, maybe - to complete. Maybe waiting in a queue for a permission or for the assignment of an interrupt priority.
I dunno.

[mouser]

Your post has a lot of clear info in it -- but one thing that wasn't clear to me is exactly what part is slow..
Am i correct that what you are saying is slow is when you select a clip, it is slow to display it's details in the memo panel?

And could you be more specific about how slow is slow?

And here is another thing that would provide a clue -- if you find a clip -- let's say one without an image -- that is slow to load, is it always slow to load even if you keep alternating between selecting it and another clip?  or is it only slow the first time you select it but not the repeat times soon after?

[IainB]

Am i correct that what you are saying is slow is when you select a clip, it is slow to display it's details in the memo panel?
___________________________

Yes.

And could you be more specific about how slow is slow?
___________________________

Let's say I navigate the Grid pane using the Up/Down cursor keys. (This is how I usually do it.)
If the cursor is (say) being moved Up, it will frequently halt/pause - even though the cursor Up key has been pressed again once or even twice or thrice - for about 1 or 2 sec. Then the cursor will rapidly "catch up" the number of Up key depressions it has missed, missing out displaying (or maybe only momentarily displaying) any of the intervening clips in the Memo pane, until it is resting on and displaying in the Memo pane the latest clip the cursor was moved to.

Similarly if the cursor is (say) being moved Down.

It's like a buffer is waiting to be initialised/loaded with the data, in some circumstances. It's not consistent for every clip or clip type, and I can't see that the clip contents necessarily makes any difference to this behaviour.

This behaviour seems to be consistent and repeatable. One can repeat the movement of the cursor over the same clips in the Grid pane, and pretty much the same behaviour is repeated - as near as I can see, at any rate. So it's not as though a buffer array has been refreshed and is faster the second time around.

Whilst the cursor is in a "paused" state, the Memo pane display does not change (is not updated)

So that also answers the last part of your Q:

And here is another thing that would provide a clue -- if you find a clip -- let's say one without an image -- that is slow to load, is it always slow to load even if you keep alternating between selecting it and another clip?  or is it only slow the first time you select it but not the repeat times soon after?

That is, it's equally slow the second time - So it's not as though a buffer array has been refreshed and is faster the second time.

And here's another odd thing:
If I start off at the bottom of the Grid, at the "latest" clip and that row highlighted, and move the Up cursor several rows up the Grid, then that clip/row is highlighted in the Grid and will eventually be displayed (after the delay) in the Memo pane.

If I then press Ctrl+End, the cursor does not - as one would expect - go to the last clip in the Grid, but remains where it was.
However, if I then press the Up key, the highlight immediately appears in the penultimate clip row, so it must have been logically pointing at the last clip position before the Up key was pressed, but was not being displayed as such.
This is such odd behaviour that I have wondered whether some other process - e.g., (say) my use of SQL in Virtual Folders - isn't somehow interfering in the cursor movement process. It looks as though some computation or a wait might be going on, causing the delays.

[mouser]

Let me make you a debug version that reports how long it took to "load" each clip, then we can see what kind of pattern we are looking at.
Does it help the speed if you turn off spell checking? (click the small button in the memo panel toolbar with ABC checkmark).

[IainB]

Let me make you a debug version that reports how long it took to "load" each clip, then we can see what kind of pattern we are looking at.
Does it help the speed if you turn off spell checking? (click the small button in the memo panel toolbar with ABC checkmark).

Thanks. A timer log could be useful.

I usually have spell checking turned ON. I switched it off, and the stuttering (pausing) display seemed a bit quicker. So I turned spell checking back on, and it was definitely a bit slower, so that's a partial workaround - switch spell checking OFF. (I rarely use/rely on it anyway.)

I shall fiddle about with the settings a bit more to see if I can make the problem better or worse.

[IainB]

Not relevant, but interesting; I just discovered that:

1. EverDesk
2. FARR
3. InfoBase
4. PageFour
5. SynWrite
6. Tomahawk
7. WinOrganiser

- all use dictionary files with the .adu filename extension.

Presumably, they could all be using the same/similar spelling engine and autocorrect.adu file contents. That would be sensible. Would mean some duplication.
Hmm. File sizes are a close match.
EverDesk And InfoBase have an identical checksum for their 54KB autocorrect.adu files.

[IainB]

An outstanding bug. Probably not relevant to my prob, but may be relevant to timing on copy (?):
Grab browser URL as Notes is ON, but only seems to work in Firefox and IE

[IainB]

Copy of CHS Statistics, as at 2016-11-24 0007hrs:
Information:
  Database File: C:\UTIL\Windows utilities\FindAndRunRobot\Plugins\Clipboard Help+Spell\Database\ClipboardHelpAndSpell
  Temp Directory: C:\UTIL\Windows utilities\FindAndRunRobot\Plugins\Clipboard Help+Spell\Database\Temp (0)

Totals:
  Text Clips Added To Database: 16437  (4.84mb)
  Image Clips Added to Database: 916  (224.55mb)
  Items Pasted: 2878
This Run
  Text Clips Added To Database: 61  (38.60kb)
  Image Clips Added to Database: 0  (0b)
  Items Pasted: 8
This Run, Total Detected Clipboard Events: 813
  DB Retryable Errors: 0 (0)
  ClipEvents Triggered: 836
  ClipEvents Processed: 805
  ClipEvents TooSoon: 661
  ClipEvents Identical: 18
  ClipEvents Duplicate: 3
  ClipEvents Ignore: 8
  ClipEvents TooBig: 0
  ClipEvents BadFocus: 66
  ClipEvents BadMutex: 0
  ClipEvents IgnoredEvents: 0
  ClipEvents TooFast: 0
  ClipEvents GetUrl: 35
  ClipEvents GetUrlException: 0
  ClipEvents Exception: 0
___________________________

[IainB]

Yesterday, I set CHS.exe to "Run this program as an Administrator" for all Users, but it seemed to make no difference then or later.
I have left it set as that.

[IainB]

Out of interest, I set the Temp setting to force the Temp directory to the System temp directory: R:\Temp\ (2) <--{this is copied from the Statistics pane; I am not sure what the number (2) signifies.}
R:\ is a RAM disk with dynamically assigned size (created by ImDisk ToolKit). Is very stable.
I then restarted CHS.

The result seemed to be a much faster movement up/down the Grid, but still stuttering. with some pauses being about 3 or 4 seconds, and in those cases the difference between the clips seemed to be insignificant.

I have retained this setting to R:\Temp\ since it seems to be an improvement.
However, the problem remains.

[IainB]

I used Process Hacker to restart RuntimeBroker.exe, as the latter has proven to be a constipating factor for the performance of several other processes, however the restart of RuntimeBroker.exe seemed to make no apparent difference to the performance of CHS.

I'm running out of ideas for performance tweaks for CHS now.

[IainB]

Sorry. This post made in error and retracted.
Had not got all my facts right.
Working on it.

[IainB]

@mouser: I have drawn a blank.
Despite all the fiddling about, the problem persists and is inconsistent.
I was able to eliminate the problem by reducing the database to 14 clips!!!
(So the problem may be related to DB size.)

In a DB of (now) 600+ rows, CHS can sometimes be scrolling very fast, and then - Whammo! - it's back to the stuttering 1 to 3 second delays in screen display - and that's scrolling over new stuff it hasn't scrolled over yet and re-scrolling over stuff it's already scrolled over. Very frustrating.

Are you any nearer to making a debug version - as you mentioned above - that reports how long it took to "load" each clip, so we can see what kind of pattern we are looking at?
(Thanks.)

[mouser]

I'll have you a debug version before tomorrow.  Email me at mouser@donationcoder.com, we can talk more quickly over email.

[mouser]

Can you try this version:

• http://www.donationcoder.com/Software/Mouser/clipboardhelpandspell/downloads/alpha/ClipboardHelpAndSpellSetup.exe
• or portable version: http://www.donationcoder.com/Software/Mouser/clipboardhelpandspell/downloads/alpha/ClipboardHelpAndSpellPortable.zip

This version adds an option for updating the last-view-date of each clip, setting this to false by default (you'll find it in the Misc Options 1 tab).  In prior versions this was always done.
It may be that this is slowing down your browsing of clips.

If this doesn't help I have a debug version ready for you to try and send me feedback on.

[mouser]

Looks like Iain and I may have made some progress beta testing some changes.. Stay tuned.

[IainB]

This can definitely be flagged as "FIXED" now!
I have downloaded CHS v2.38 BETA Portable.zip and installed it, from
CLIPBOARD HELP+SPELL LATEST VERSION INFO THREAD - v2.38.0 BETA - Dec 16, 2016

Thanks a lot for doing this speed-up. Nice work. You raised the bar of CHS' performance. 
CHS v2.38 BETA is blazingly fast - just like it should be. Out of interest, I was just comparing it side-by-side with CHS v2.36 BETA on another laptop. No contest!
And thanks also for improving the functional ergonomics of the Home/End keys. It's only a small ergonomic improvement, but, when one uses those keys as frequently as I tend do on a daily basis, then it can all add up so that even a small ergonomic improvement like that can make for a vast improvement over time, in terms of time saved.

By contrast, the speed-up is also an ergonomic improvement, but it's a big deal (massive) improvement for this user, and it changes one's whole perception of the ease-of-use of CHS. After this fix, one can now better use CHS for the sorts of things it should and could have been useful for, by design, but which one had avoided using it for, because it was effectively crippled by such a proverbial PITB. (Because of my critical view and high expectations, I am usually extremely impatient with computer software and can't abide "laggy" functionality in a GUI that I need to use.)

A much happier user now!   

[mouser]

Awesome 

Thanks for all your patience and perseverance helping me solve it.