SQLNotes...what is it exactly?

[Armando]

Just an update... I'm finishing up the FireFox extension, which will ease web content clipping
1- Grab whole page, plus URL
2- Grab current selection, plus URL

Anything else worth grabbing? i.e. Title, page date, grab date, etc.

What are your thoughts on UI. Should the clip go straight to the database, or should you be presented the New Item UI, to better control the import? I can hear the answer... you want both 

-PPLandry (November 06, 2008, 09:56 PM)

I think : title, page date and grab date are important -- of course, the item's creation date tells the grab date already (and it's read only, so...).

And, yes, I'd like to be able to choose if I want my item directly to the database, or presented with the "add item" UI. For the reasons you explained + be able to accumulate clips inside the same item, etc.... Great, great ideas.

Not sure that I understand you Pierre.  What is a "folder" in IQ? Actually I don’t have any! And this is absolutely the first time I have heard anyone mention folders in IQ.

Did I miss something? You have "field" in parenthises; is every field a folder? (Oh no, now I'm confusing me too!)

-J-Mac (November 06, 2008, 11:00 PM)

I don't think you missed anything. 

Pierre used the folder metaphor to describe fields -- in the wiki, fields are sometimes described as folders. Personally (IMVHO...) I think we should really avoid doing that... it IS confusing. Even if it sorta makes sense (fields contain the info of different items, so they can be compared to folders... but... but...)

How do you mean "control the item text"? I'm talking about capturing a web page or part of a web page. Do you mean edit the text on the page? If so, I explained that above.

And as for the "item keeps accumulating your clips", how would clipping directly to the database affect that?

Jim

-J-Mac (November 06, 2008, 11:00 PM)

If you clip directly to the database, each clip would be recorded as an independent item, most probably (although, that could be set as an option within the context menu attached to the "clip as IQ item" firefox button/option.... But I digress...). So Pierre only suggested (among other things) that some of us might want the option of accumulating several clips inside the same item -- e.g. : you're clipping several parts of one web page, and you want all these parts in the same item for increased coherence and ease of data manipulation...

[J-Mac]

Gotcha.

Thanks Armando!

Jim

[tomos]

clipping several parts of one page in one item sounds great - I guess it would be unfeasible to clip parts of various pages to one item?

This is probably your intention anyway:-
How about if the new item window comes up with Title of webpage in "Item Text" and if one simply presses return, the new item is saved (i.e. pretty much same as now except all areas automatically filled)

[freewareking101]

ohh! its good to have this. I didn't thought there is something like this.
very helpful
Thanks for sharing!

[superboyac]

I'm trying to export to Excel...I highlight all the rows of items in my grid (all of them), and then I click the export to excel button.  I open the file, and only the very top level item is there.  How do I get all the other ones to appear in the excel file?

[J-Mac]

I'm trying to export to Excel...I highlight all the rows of items in my grid (all of them), and then I click the export to excel button.  I open the file, and only the very top level item is there.  How do I get all the other ones to appear in the excel file?

-superboyac (November 10, 2008, 05:25 PM)

Are you sure that they're not all there in that one item, but just not visible because of the row height?

Jim

[PPLandry]

I'm trying to export to Excel...I highlight all the rows of items in my grid (all of them), and then I click the export to excel button.  I open the file, and only the very top level item is there.  How do I get all the other ones to appear in the excel file?

-superboyac (November 10, 2008, 05:25 PM)

Export to excel needs improvements. Currently, it exports all items that belong to the grid, not all the items displayed.

To do what you want,
1- use copy/paste and use tab-delimited format, or
2- you can tag all desired items (right-click > tag), open a grid and set the source to ItemTagged and do an Excel export

[superboyac]

OK, thanks, I'll try it a little later...

But, yet another reason why I feel if an item is created in a grid, that grid should by default be the source for that item.  I just can't see much use for having a bunch of sub-items not having any source.

[PPLandry]

OK, thanks, I'll try it a little later...

But, yet another reason why I feel if an item is created in a grid, that grid should by default be the source for that item.  I just can't see much use for having a bunch of sub-items not having any source.

-superboyac (November 10, 2008, 06:14 PM)

if the source is a yes/no field (i.e. the concept of a folder), and when the item is created there (as opposed to created elsewhere and later on added as a child to a parent, perhaps even a 2nd, 3rd parent), I'd agree with you (and you can set it in the field property), but for a grid with another type of field as its source (date, text, number), it does not make much sense.

[PPLandry]

I just can't see much use for having a bunch of sub-items not having any source.

-superboyac (November 10, 2008, 06:14 PM)

Keep in mind that the basic (and simple) concept behind IQ is that items are like individuals. They exist and they have properties (i.e. field values).

Items exist independant of their parents, their children, and field-values. If you die (delete an item), the children, parents remain, etc. This is absolutely critical in understanding IQ. You have properties (field) or different types (text, number, date, etc) (age=38, hair=brown, Birthday=1965-06-21, etc)

If your grid source is a yes/no field called WorksForCompanyABC, and you enter all the people working for CompanyABC there, then all main items should have that field checked. I agree.

But if you start adding sub-items, representing say, the children of the employee, then you don't want them to have that field checked (the child does not work there).

Another example, is that at any time, you may want to move an item under a parent (changing parent or adding a second parent) (e.g. a file may be used under 2 projects). You want to associate the file with the project (by being a child), but you don't want to affect the field values of that file).

[Armando]

I'll think about it more.... But, quickly, I tend to find that in the context of IQ, the "Ecco way" makes filtering operations more complex -- ie : I'd prefer to have all items in a grid appear there because they meet the source.


1- I constantly need to filter items out if they "shouldn't" appear un the grid... (It seems like it would be easier to filter out items not meeting the source, than filtering out items not really meeting the source, but appearing in the grid nevertheless (because their parents meet the source...).

2- "Flat view" (hierarchy off) only show top level items,

3- Search for certain fields/tags related items only return top level items and misses all the others,

etc.

Just some quick thoughts before going out for a bite... 

[Armando]

edited previous post

[J-Mac]

Pierre, Armando, superboyac:

These are the kinds of details that make IQ so difficult for me (and I suspect, others) to learn!

I also have "lost" items that I was sure had been in a given grid, and "found" items where I didn't expect them. Is the above concept documented anywhere at all? So many times when I ask how to do or find certain things I am told by someone or another that it isn't yet in the wiki or anywhere else in particular, but that if I keep trying this and that and read forum posts I'll eventually learn about these features.

Is that really how everyone else has learned how to use IQ? Don’t get me wrong; I know that any worthwhile software application, and particularly a database-type application, requires the user to do some degree of trial and error in order to become proficient in the various features. I definitely don’t expect to just open IQ and know everything. But the learning curve really has been painfully slow with IQ. Mostly because I want to start using it for real, often-used data; stuff that I need to access and edit/update regularly. But up to now I am only putting "test data" into it. Real enough data that I can play around with, but nothing that I truly need. I'm still not confident enough in how I can make IQ work to put my important data into it.

Someone who knows IQ - really knows it well - needs to compose some sort of documentation that covers all the bases. If not in detail at least an overview, but an overview that covers all the topics, all the features. And then it must be fleshed out as quickly as reasonably possible. I don’t think that a lot of people will be willing to buy into a program that is as complex as IQ without any kind of "instruction manual" - that is, other than former Ecco users.

Thank you.

Jim

[Armando]

I also have "lost" items that I was sure had been in a given grid, and "found" items where I didn't expect them.

-J-Mac (November 10, 2008, 09:31 PM)

It can be easy to "loose" items** :

1- when one doesn't really understand why items show up in a grid or not. (I'm not saying that in a dismissive way...)

2- because some parts of the UI need to be rearranged to allow the user to have a clearer view of what filter is at work.

Concerning point 1 : really, the only thing one needs to understand is that SQLNotes/IQ  grids show items that meet the filters. They are NOT folders. That's all. There's "nothing" more to it (at a basic level). Fundamentally, SQLNotes works like a desktop search software : take X1 (very similar to SQLNotes in many ways) : the different grids show what meets the filters/sources...

Now, what usually happens is that a lot of users don't realize that some filters are on, and others are off, etc. and that "hierarchy mode" is off, and that "show context  parent" is on, etc.
This could be because the UI needs some improvement (a Basic/Advanced UI will eventually be available). It could also be that some of the ways SQLNotes works should be slightly modified. And it could also be that a lot of users just don't ready the wiki at all... (I'm not saying you haven't, J-mac... But it's clear to me that most don't. Which is okay, I guess.)

** Note that if it's easy to "loose" items, it's equally easy to find them : ctrl-f (show-details can be ticked if you want to have more control over your search): type a few words... Select the items you want to modify, press "enter".

In the newly opened/focused "search grid" : modify the fields that need to be modified for these lost items to meet the grids filters/source you want. OR, even easier,  drag/drop the items to the different opened grids, if they are opened (normally, the appropriate fields will be ticked/modified when you drop the items on a grid, so that they meet the grid's source -- unless you've modified some "per grid" settings in the manage grids window...)

[...] the learning curve really has been painfully slow with IQ. Mostly because I want to start using it for real, often-used data; stuff that I need to access and edit/update regularly. But up to now I am only putting "test data" into it.

-J-Mac (November 10, 2008, 09:31 PM)

The learning curve can be quite steep at the moment... Yes. I hope it'll get better, with clearer documentation, maybe, and also (maybe again), with some UI simplifications.

One thing that really helps though, is to REALLY go through the already available documentation. Because, it contains enough info to get someone started.

Someone who knows IQ - really knows it well - needs to compose some sort of documentation that covers all the bases. If not in detail at least an overview, but an overview that covers all the topics, all the features. And then it must be fleshed out as quickly as reasonably possible.

I don’t think that a lot of people will be willing to buy into a program that is as complex as IQ without any kind of "instruction manual" - that is, other than former Ecco users.

-J-Mac (November 10, 2008, 09:31 PM)

I'm sure that once SQLNotes gets out of beta stage, all the necessary documentation will be there... Features and the UI has not completely stabilized... so....

Some have started to enrich the wiki with their own experiences, so there's progress.
I will try to contribute too.

Thanks for your comments J-Mac. 

[superboyac]

Egh...ok, Pierre, I definitely didn't think about all those scenarios you described and I can see how the behavior is useful.  On the other hand, I probably agree with Armando that perhaps the "ecco" way may not be the best way for default behavior, or not the most intuitive.  But I really haven't thought it through enough to really have an opinion, especially once you bring up all those scenarios.

Jim, don't be afraid of it, though, whether you understand the program or not, it's still usable.  Actually, don't even bother learning more than you need to.

So, Pierre, regarding my problem, the issue for me is not the source/grids/checkbox thing.  To be more accurate, my real problem is that I have a grid open that shows a bunch of items.  I highlight all the items and export it to Excel.  I expect to see all those items in Excel, naturally.  So, like you said before, the exporting is the one that needs to be fixed.  How you do the source/parent thing is fine with me, either way.

So, you see, Jim?  I don't need to understand how the filtering works if I don't need to, when the real problem is that the Excel export doesn't work intuitively.

(Nice post above Armando.  Yes, good documentation is going to be a challenge for Pierre, there's so much to cover!)

[PPLandry]

Egh...ok, Pierre, I definitely didn't think about all those scenarios you described and I can see how the behavior is useful.  On the other hand, I probably agree with Armando that perhaps the "ecco" way may not be the best way for default behavior, or not the most intuitive.  But I really haven't thought it through enough to really have an opinion, especially once you bring up all those scenarios.

Jim, don't be afraid of it, though, whether you understand the program or not, it's still usable.  Actually, don't even bother learning more than you need to.

So, Pierre, regarding my problem, the issue for me is not the source/grids/checkbox thing.  To be more accurate, my real problem is that I have a grid open that shows a bunch of items.  I highlight all the items and export it to Excel.  I expect to see all those items in Excel, naturally.  So, like you said before, the exporting is the one that needs to be fixed.  How you do the source/parent thing is fine with me, either way.

So, you see, Jim?  I don't need to understand how the filtering works if I don't need to, when the real problem is that the Excel export doesn't work intuitively.

(Nice post above Armando.  Yes, good documentation is going to be a challenge for Pierre, there's so much to cover!)

-superboyac (November 10, 2008, 10:37 PM)

In the next version, I'll display an msgbox when exporting to Excel:
1- Export all grid items
2- Export selected items
3- Export items that meet the grid criteria
4- Cancel

Expect his version tomorrow, with the brand new FF Extension, which will ease web clippings

[J-Mac]

1- when one doesn't really understand why items show up in a grid or not. (I'm not saying that in a dismissive way...)

-Armando (November 10, 2008, 10:26 PM)

I agree. But that's exactly the problem: Why DO items show up sometimes - and not show up at others? If it is due to filters, then I'm baffled, as I have not touched any of the filters.

2- because some parts of the UI need to be rearranged to allow the user to have a clearer view of what filter is at work.

-Armando (November 10, 2008, 10:26 PM)

Amen!

Concerning point 1 : really, the only thing one needs to understand is that SQLNotes/IQ  grids show items that meet the filters. They are NOT folders. That's all. There's "nothing" more to it (at a basic level). Fundamentally, SQLNotes works like a desktop search software : take X1 (very similar to SQLNotes in many ways) : the different grids show what meets the filters/sources...

Now, what usually happens is that a lot of users don't realize that some filters are on, and others are off, etc. and that "hierarchy mode" is off, and that "show context  parent" is on, etc.
This could be because the UI needs some improvement (a Basic/Advanced UI will eventually be available). It could also be that some of the ways SQLNotes works should be slightly modified. And it could also be that a lot of users just don't ready the wiki at all... (I'm not saying you haven't, J-mac... But it's clear to me that most don't. Which is okay, I guess.)

-Armando (November 10, 2008, 10:26 PM)

True enough about the filters. But... I don’t go off looking at all the various filters (and there are a few of them, aren't there?) because I never touched them. Are some filters set automatically? Or are some filters already set by default when you first start using the program? If I didn't put anything in a filter, I don’t expect it to be there.

And yes - I have read the wiki, front to back. And I go back and check often to see what, if anything, has been added. (BTW, the "New" tag sint working too well in the IQ wiki. I still see "New" tags on articles that haven't changed since the beginning.) Actually I have put every page from the wiki into a single PDF file so that I can read it when I'm not sitting at the computer. The wiki has only just started getting some much-needed attention. Up until now it read more like a brochure than an instruction document.

One thing that really helps though, is to REALLY go through the already available documentation. Because, it contains enough info to get someone started.

-Armando (November 10, 2008, 10:26 PM)

I have read everything written about IQ, including most of the forum over there - which is not easy as that forum is not working well at all; hasn’t since I first saw it. And that's about all I'll say about the state of the documentation.

Thank you.

Jim

[tomos]

I agree. But that's exactly the problem: Why DO items show up sometimes - and not show up at others? If it is due to filters, then I'm baffled, as I have not touched any of the filters.

-J-Mac (November 10, 2008, 11:42 PM)

In this scenario, I think you'ld have to explain in detail what happened - the problem is, as you say: you dont know what happened.
This happend to me at the beginning & I've no idea why but now I'm reasonably familiar with the programme it hasnt happened since so I'm presuming I did something wrong somewhere.

If you figure out how to get those items back, I think that will also help with your understanding of the programme and maybe help you find out why they stopped showing - search will find individual items but probably better to use the date filter if they all created around the same time ( [item created] on such n such dates).  When you find the items have a close look in the properties pane at what fields are filled. Possibly what happened (if they were sub-items in a grid) is that for some reason the parent link got broken. If they were top level items the source field was deselected but I cant imagine that happening.

Filters include the Date filter and the Alpha-numeric filters as well as the filter in the sourcebar

Dont know if any of that help (I'm also possibly repeating things you know...)

~~~~~~~~~~~~~~~~~~~~~~~~~~


RE all items meeting source or not -

I was happy with current situation (only top-level items meeting source) because it meant I could display hierarchy in an expected way
As far as I know if I use hierarchy display mode and some sub-items also meet source they are displayed twice (once as sub-item, once at top-level) which disturbed my sense of "order"
Correct me if I'm wrong there - I havent checked this in a long time so situation may have changed - will test it later

If I could get the display to ensure that an item only shows once (presuming it's only in grid once) I would be happy to have some sub-items meeting grid-source -as Pierre says sometimes it would be appropriate, sometimes not.

[Armando]

Tom's advice is good.

J-Mac :

Like I said, there's stuff missing in the documentation, and yes, it might not be enough to understand why items appear and disappear. Obviously... if you don't know why some of your items disappeared.  This will get fixed in time. I will contribute, etc. And the UI will improve.


In the mean time :

Here are the areas/options that have a direct effect on what appears on a given grid.

1- the source box (main filter)

2a- the filter box (secondary filter)
  2b- the date filter (This is just a convenient way of creating quick date filters without having to no any SQL syntax)
  2c- the alpha numeric filter (This is just a convenient way of creating quick text/number filters without having to no any SQL syntax)
  2d- filter applies to sub-items (will apply the filters -- not the source though -- to subitems)

3- context parents (will show the item's main parent in blue, if not meeting the grid's source/filter)

4- The various column filters (but these, right now, disappear after a refresh ; they're temporary)

Then there's also the Sort filter (+ sorting applies to sub-items) and the hierarchy mode, but these really, can't "remove" items from a grid. There really only for display.

Now, if any of your items don't show up in the grid, it might not be because you touched the filters, but it could also be because you've modified the items so that they don't match the filters anymore. This is very easy to do.

Otherwise, I don't see what could've happen. Like I said, it functions a bit like X1 or any searching software with filters : remove key words from a document, and it won't show up in the result area (the grid). Change the filters : displayed documents will change in the result area. That,s the basic idea.

------------

What I suggest is follow the steps I described earlier to find back your items (or you can also try to deactivate all previous options for a given grid : filter, alphanumeric, filter applies to sub-items, etc.) and then tick the field corresponding to the source of a given grid. And see what happens.

Then if you still don't see the item, deactivate all filters (there are buttons for that on the source bar...  but the filter applies to sub-items is on an separate toolbar -- yes, this is a UI problem that will get fixed... it should just be an extension/"sub-button" of the other button).

The source bar was described (by Keith I think... Thanks Keith!) on the getting started page :
http://sqlnotes.wiki....com/Getting+started

RE all items meeting source or not -

I was happy with current situation (only top-level items meeting source) because it meant I could display hierarchy in an expected way
As far as I know if I use hierarchy display mode and some sub-items also meet source they are displayed twice (once as sub-item, once at top-level) which disturbed my sense of "order"
Correct me if I'm wrong there - I havent checked this in a long time so situation may have changed - will test it later

If I could get the display to ensure that an item only shows once (presuming it's only in grid once) I would be happy to have some sub-items meeting grid-source -as Pierre says sometimes it would be appropriate, sometimes not.

-tomos (November 11, 2008, 03:00 AM)

The problem you describe (with sub-items showing at top-level) is still occurring, but it will get fixed... I've had a long conversation with Pierre yesterday, and a slightly new way of managing sources to have all sub-items meeting the source might also be created. But I'll let Pierre comment on that himself... 

[PPLandry]

I'll add to armando excellent post, that you can use the Journal Grid. You can use the DateFilter toolbar to show all items created on a specified day, week, month. It is useful to find items.

Once you've found your items, you can examine the filled fields and determine why they are not shown in the problematic grid

[superboyac]

If it is due to filters, then I'm baffled, as I have not touched any of the filters.

-J-Mac (November 10, 2008, 11:42 PM)

This is a point I like to emphasize also.  This behavior should be minimized as much as possible.

Expect his version tomorrow, with the brand new FF Extension, which will ease web clippings

-PPLandry (November 10, 2008, 11:20 PM)

Pierre, no one can accuse you of not working hard!  We really appreciate you continuously adding features and dealing with our issues.

[Armando]

If it is due to filters, then I'm baffled, as I have not touched any of the filters.

-J-Mac (November 10, 2008, 11:42 PM)

This is a point I like to emphasize also.  This behavior should be minimized as much as possible.

-superboyac (November 12, 2008, 12:01 PM)

Scoop :  "All items meet the source in a grid"   +    a slightly redesigned GUI (where everything filter related is centralized +  applied filters appear in the status bar + button to activate/deactivate all filters +  the birth of the famous "named filters" ) should help to fix this.

[J-Mac]

Armando,

Thanks for the detailed reply! I'm still going through it in detail while IQ is open on my desktop - trying to learn more here. The documentation is getting better, BTW. Much appreciated.

Pierre,

I must agree that you are a tireless worker; no argument there! I'll get better w/IQ, I'm sure. Please do not take my posts as criticism - if I felt critical all the time, I'd just stop trying to use it altogether. I do want to use IQ for tasks that I am currently accomplishing with other applications, but I need to feel more comfortable first.

Workin' it, Boss!

Jim

[J-Mac]

BTW, one thing I tried to do a couple of days ago was to export a grid from one database and then import it into another. I'm trying to get stuff that I had put into separate databases into one. However it isn't working that well. I posted about this in the IQ forum but haven't received any replies.

I followed the instructions in the wiki:  I copy the contents of the grid into an XML file, paste that into a text editor and save it as an XML file, copy that, and then try to paste it into a newly created grid in the other database. No indication that anything happened. However if I do Ctrl+F and look at all items I see the data, but the fields, columns, etc. apparently don’t come over that way.

So I just ditched the database and started over.

Is there any working way to move data from one database to another?

Thanks!

Jim

[PPLandry]

You did the correct thing (copying to a file was not required, as you can solely work from the clipboard)

However, fields are not created when pasting XML (they will eventually), so you need to create your fields and grids in the destination database first, and then paste