topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Sunday December 1, 2024, 2:02 pm
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Last post Author Topic: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread  (Read 88836 times)

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #25 on: February 14, 2020, 12:38 AM »
Updated with somewhat improved capture options:
Screenshot - 2_14_2020 , 12_37_42 AM.png

cranioscopical

  • Friend of the Site
  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 4,776
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #26 on: February 14, 2020, 08:17 AM »
I'd be curious if folks would prefer to have image data stored in the normal clip database as well?

For me, keeping the database robust trumps duplication.

djMot

  • Supporting Member
  • Joined in 2018
  • **
  • Posts: 40
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #27 on: February 15, 2020, 08:25 AM »
I'd be curious if folks would prefer to have image data stored in the normal clip database as well?

For me, keeping the database robust trumps duplication.
-cranioscopical (February 14, 2020, 08:17 AM)

I agree.  I think it should all go into the database, but given that you (mouser) are worried about database bloat, a good backup and purge feature might help in that regard.  Perhaps a dropdown listing 1, 3, 6, 9, 12 month or older, and a button to backup and purge clips beyond the selected cleanup period.  On click, open a file dialog to the db directory and provide a default purge database name based on the selected cleanup period and the current date.  Maybe even give the option to just purge without backup (not the default) because there will probably be many who would be fine with a straight delete without save.

But now that I think of it, aren't there already automated backup, and purge settings?

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #28 on: February 17, 2020, 07:25 AM »
New version uploaded.
I had to make some more elaborate changes to the multi-format "blob" data used by chs, such that previous blob data in clips from prior alpha versions won't be readable by this version.

I've still got quite a bit of work to do bringing everything in line with the new code, but CHS should be pretty good at saving and restoring the complete set of formats associated with clips, and hopefully handling duplicate detection, etc.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #29 on: February 17, 2020, 04:45 PM »
So what I've done so far:

  • I've migrated the core clipboard capturing and pasting functions from using a single format (plaintext or image), to using a multi-format "blob" data structure.
  • So when capturing it captures all formats in the clipboard and saves the collection (blob) of formats in the database.
  • It has some basic options for what formats you want it to ignore, and tries to identify redundant formats that can be ignored, for example unicode/oemtext/dib bitmaps that are equivalent to other stored formats.
  • It also handles plaintext and images specially; while preserving their order in the blob collection (if multiple formats found), it pulls out images to external png files with comments, and pulls out plaintext to sit in the database as a separate searchable field.  In this way, I'm hoping to get the best of both worlds (multi-format support but more convenient access to plaintext and images and minimizing database size).
  • Order of the multiple formats in a blob is preserved even in the case of images and plaintext, which may occasionally be important to some scenarios.
  • A single checkbox in the option will toggle the program from capturing multiple formats vs the old pure-plaintext behavior of CHS, so if you don't want the new formats and prefer a leaner pure plaintext database you can still use CHS that way.
  • Duplicate checking should be somewhat improved, and work for multiple-format blobs and images.

What's left to be done:
  • Lots more testing -- I'm hoping to get some help with that from the forum.
  • There are a bunch of ancillary things regarding copying+pasting that I might not have fully implemented in the new code (flashing clipboard icon on capture, dealing with stubborn situations where the clipboard is being used by another app, updating clip modification dates on pasting, etc.).
  • There are a few functions that are specifically plaintext related, like when performing format presets, and pasting text using sendkeys, that I have not fully moved over, or that use some old code.  I still have to go through and update all of those functions.  They will will only work with plaintext, but I think the existence of multi-format blob data may confuse some of these functions.
  • Similar behavior might be expected after manually editing a clip -- you will have changed the PLAINTEXT content of the clip, but not the unicode/richtext version, So I think what has to happen in such cases is a clearing of the unicode/richtext data in such cases.
  • I want to add more features to help you see how much space is being used by the multi-format data, and to do some maintenance operations on it to keep your database lean (like functions for removing multi-format data from old clips, etc.).
  • I would like to add more flexible rules for deciding which formats you want saved.. For example being able to specify a custom list of formats to keep or ignore, and maybe setting arbitrary limits on size, or based on the application running.  I'm not sure the best way to do this that would satisfy experts and novices alike..
  • Verify that the new image capture/pasting DIB format works in all cases.
« Last Edit: February 17, 2020, 04:57 PM by mouser »

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #30 on: February 18, 2020, 04:05 AM »
Ok new version uploaded.. I'd like to get some feedback before I do too much more.
You might want to click the small button to the far left on the grid header row to show some of the columns to see more Blob info, etc.

djMot

  • Supporting Member
  • Joined in 2018
  • **
  • Posts: 40
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #31 on: February 18, 2020, 08:21 AM »
I'm still not seeing images in a the HTML Preview tab.  Is there something I'm missing, or are you decisive about just not going there?

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #32 on: February 18, 2020, 05:07 PM »
I'm still not seeing images in a the HTML Preview tab. 



Try re-downloading to get the latest version and then enable this option:
Screenshot - 2_18_2020 , 5_07_02 PM.png

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #33 on: February 18, 2020, 09:25 PM »
Here's a little trick, create a virtual folder(group) showing you only Large clips in your database (those bigger than 100k):
Screenshot - 2_18_2020 , 9_24_57 PM.png


There are actually now 2 fields related to a clip's size.  DataSize is total size of clip INCLUDING any external image size.  BlobSize does not include size of external image.

You can click on the little button to the far left on the column header to see and show more clip field names.
« Last Edit: February 18, 2020, 11:40 PM by mouser »

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #34 on: February 18, 2020, 11:09 PM »
Updated with some improvement to the organization of the options dialog.

djMot

  • Supporting Member
  • Joined in 2018
  • **
  • Posts: 40
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #35 on: February 19, 2020, 09:21 AM »
I'm still not seeing images in a the HTML Preview tab. 



Try re-downloading to get the latest version and then enable this option:
[ Invalid Attachment ]

Yup!  Beautiful!  THANK YOU!   8) :D :Thmbsup:

djMot

  • Supporting Member
  • Joined in 2018
  • **
  • Posts: 40
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #36 on: February 19, 2020, 09:34 AM »
Here's a little trick, create a virtual folder(group) showing you only Large clips in your database (those bigger than 100k):
[ Invalid Attachment ]


There are actually now 2 fields related to a clip's size.  DataSize is total size of clip INCLUDING any external image size.  BlobSize does not include size of external image.

You can click on the little button to the far left on the column header to see and show more clip field names.

Interesting.  So the SQL text box contains a WHERE clause, 'eh?  Is there a database schema somewhere so we can understand what's available to construct a WHERE clause from?

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #37 on: February 19, 2020, 05:48 PM »
Is there a database schema somewhere so we can understand what's available to construct a WHERE clause from?

I will try to make one.  The help file needs some real updating.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #38 on: February 19, 2020, 06:07 PM »
Any more thoughts about what features are most important that still need to be added to make this multi-format stuff useful?

tomos

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 11,964
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #39 on: February 19, 2020, 07:20 PM »
A quick look at the most recent version:

1. vector clips are being duplicated.

2. I havent used CHS for images of late. I changed default image size to 10,000 and copied a few images under 1MB in size (from dopus). The clips are listed but no images are showing in quick-paste nor in main window. If I copy an image again, I get the 'I'm already in the database' sound.  Am I doing something wrong?
Blob info
2.01kb (19 formats)

"DataObject" [0000C009]: 8b (query 0ms)
"Shell IDList Array" [0000C0FB]: 367b (query 0ms)
"dopus_cf_hdrop" [0000C2F1]: 90b (query 0ms)
"CF_HDROP" [0000000F]: 90b (query 0ms)
"FileName" [0000C006]: 34b (query 0ms)
"FileNameW" [0000C007]: 68b (query 0ms)
"dopus_cf_typetable" [0000C369]: 5b (query 0ms)
"dopus_cf_sizetable" [0000C36A]: 40b (query 0ms)
"DOpusSourcePath" [0000C365]: 40b (query 0ms)
"dopus_cf_rotation" [0000C367]: 2b (query 0ms)
"InShellDragLoop" [0000C10A]: 0b (query 0ms)
"Preferred DropEffect" [0000C0E3]: 4b (query 0ms)
"Performed DropEffect" [0000C106]: 4b (query 0ms)
"Paste Succeeded" [0000C108]: 4b (query 0ms)
"TargetCLSID" [0000C10C]: 16b (query 0ms)
"dopus_cf_sourcethread" [0000C36B]: 4b (query 0ms)
"dopus_cf_fromopus" [0000C36E]: 4b (query 0ms)
"dopus_cf_namespace" [0000C366]: 0b (query 0ms)
"Ole Private Data" [0000C013]: 936b (query 0ms)
-----

Extended (non-redundant) data save size: 2.01kb
Hash: 4D5AE5F1


3. HTML: it's very nice that the url is saved when copying text from a webpage
Tom
« Last Edit: February 19, 2020, 07:27 PM by tomos, Reason: blob info added »

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #40 on: February 19, 2020, 07:25 PM »
Can you give me some screenshots to show me what you are talking about regarding #1 and #2?

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #41 on: February 19, 2020, 07:26 PM »
it's very nice that the url is saved when copying text from a webpage
Yeah i think i will add code to grab those source urls and put them into the normal clip info, so they are available in the grid and even if you don't store html.

tomos

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 11,964
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #42 on: February 19, 2020, 08:05 PM »
Can you give me some screenshots to show me what you are talking about regarding #1 and #2?
I only have the ancient Freehand9 on this machine (but presume that that is not actually relevant). The vector clips from Freehand shown here are identical:

Screenshot - 2020-02-20 , 02_54_50.jpgClipboard Help+Spell Extended Clipboard Format Data Discussion Thread

1,2,4-9 and A are all image files copied from dopus, shown below (21=1, etc)

Screenshot - 2020-02-20 , 03_04_14.jpgClipboard Help+Spell Extended Clipboard Format Data Discussion Thread
Tom

tomos

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 11,964
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #43 on: February 19, 2020, 08:09 PM »
The vector clips from Freehand shown here are identical:
FWIW the blob info has different hash numbers
Tom

IainB

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 7,544
  • @Slartibartfarst
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #44 on: February 19, 2020, 08:46 PM »
@mouser:
Any more thoughts about what features are most important that still need to be added to make this multi-format stuff useful?
:Thmbsup:  :Thmbsup:  :Thmbsup:

Yes, I have a couple:
  • Enable wiki-like linking within Clip contents, to other CHS Clip IDs in the same database.
  • Enable viewing and editing of RTF and HTML clip content in the Memo pane (similar to last ß version NoteFrog).

The changes you are making would seem to be in line with some already defined requirements, as detailed in User Requirements for CHS
I gather some other users have added their requirements to that document.
The more you could meet of those requirements, the better!   :Thmbsup:


mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #45 on: February 19, 2020, 08:54 PM »
The vector clips from Freehand shown here are identical:

Ok, so what you are saying is you copied something multiple times, and CHS is not figuring out that the clips are identical and should be regarded as duplicates?
I noticed this happening in firefox too, but as you point out, the hashes were different, so I'm guessing that their is something in the blob data that is NOT identical, like perhaps a timestamp.

The solution to this is:
  • 1. Live with it and don't worry about having duplicate clips in such a case since it won't happen too often.
  • 2. Wait for rules that would let you ignore blob items irrelevant to the content, which could solve your problem if the timestamp (or whatever) is not the main vector content.
  • 3. I may add display of hash values for the individual format chunks in a blob so you could ID which one is changing..

I'll also try to add a better caption in the quick paste menu for blob data that has no text.
It might be nice to have a hex viewer dialog you could bring up for blob contents so you could actually see the raw contents of the blob data for this kind of debugging.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #46 on: February 20, 2020, 12:46 AM »
Updated with some fixes.  No more blank captions on quick-paste popup if a clip has no text or image.
Grabbing urls when copying from web pages is now done by looking at the html source content (and saved in Notes field of clips); in addition to being more robust it captures much faster and solves the slow firefox capture in previous versions.

I'd appreciate testing on more web browsers to see if there are any where CHS can't capture the urls. (i have tried firefox, ie, and chrome).
« Last Edit: February 20, 2020, 12:52 AM by mouser »

tomos

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 11,964
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #47 on: February 20, 2020, 04:52 AM »
Ok, so what you are saying is you copied something multiple times, and CHS is not figuring out that the clips are identical and should be regarded as duplicates?
I noticed this happening in firefox too, but as you point out, the hashes were different, so I'm guessing that their is something in the blob data that is NOT identical, like perhaps a timestamp.

The solution to this is:

    1. Live with it and don't worry about having duplicate clips in such a case since it won't happen too often.
    2. Wait for rules that would let you ignore blob items irrelevant to the content, which could solve your problem if the timestamp (or whatever) is not the main vector content.
    3. I may add display of hash values for the individual format chunks in a blob so you could ID which one is changing..
not currently a problem for me (and probably wont be in the future), but for reference, here I compare two blobs -- more different than I thought...

Screenshot - 2020-02-20 , 11_49_46.jpgClipboard Help+Spell Extended Clipboard Format Data Discussion Thread
Tom

tomos

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 11,964
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #48 on: February 20, 2020, 04:57 AM »
1,2,4-9 and A are all image files copied from dopus, shown below (21=1, etc)
can anyone comment about images: am I misunderstanding how that works? Can I not simply copy an image file?
@mouser, still nothing shown in most recent alpha for images as per screenshots in quoted post
Tom

tomos

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 11,964
    • View Profile
    • Donate to Member
Re: Clipboard Help+Spell Extended Clipboard Format Data Discussion Thread
« Reply #49 on: February 20, 2020, 05:05 AM »
[Request / rambling thoughts]

nice would be the ability to name image (and vector) clips. Could e.g. then be saved in favourites with clarity as to what exactly it is (often not clearly visible in thumbnail).

Can we actually *stop* image thumbnails showing in the quick-paste menu /sub-menus? Just curious -- I believe I would mostly reuse vector clips from the favourites and would be happy enough to not be able to see thumbnails if they have relevant titles.
Tom