"Good enough" and NANY proto-components!?

[TaoPhoenix]

I like one of the trends of this year's nany towards "back to the basics" text processing. Merging some "innovative directions in nany" I thrashed out with Mouser a year or two ago, after thinking "gee, look at all the neat text apps under pledge", I suddenly thought to poke around my oDesk account.

Turns out I have some proto-components still in Alpha stages that aren't good enough yet to be legit nany by themselves. But maybe one of y'all could adapt them and/or the ideas into features of your own programs. They were parts of projects I was working on a couple of years ago until I ran out of development money!

The next couple of messages will rough out stuff as I poke around my archives and see what I have on tap. Sometimes I had a couple of different coders work on the same theme just to see different approaches, so sometimes if y'all don't care for a particular implementation, I might have a second version, and then you'd get the overall idea and can put it into your own versions.

These are all fragments that I custom commissioned and have not been released anywhere else.

[TaoPhoenix]

Spawn TextFile

Given a large-ish inbound source text file, this app prototype looks for a delimiter and then rips apart the inbound source and resaves several smaller files.





Tip #1: Some initial support was put in to deal with "rhetorical" annotation styles. So for example if the initial file was "separated" by a lot of dashes to show segments, the app is supposed to treat them all as one split point and not pound out 15 blank files.

Tip #2: Watch that there aren't accidental cases of the delimiter. So in my example text (borrowed from some Slashdot comments of mine) I used Asterisks * as an emphasis device! But this app will split at those points too! So before you run the splitter you may want to use a really crafty search&replace to make sure the delimiter between sections is something you never nomally use. For example I never use ^ so done right, the "emphasis" * marks would stay put and only the splitter ^ would kick in. On the same lines, if you for ex wanted to split at Dashes ------, then watch for Hyphenated words!

Included are what I believe is the (mostly) complete source code, a test run showing two split-sets from an example source file, thus showing two levels of analysis on the initial source for "different bosses", and the app.

Known Bug #1: There is a glitch in the file naming output that has different numbers of digits on the exported filenames.

(Pyrohacker, you were a major inspiration for me to remember I had this stuff! You said "This app will make it easier to analyze a chunk of text and see useful information. Its focus is analysis rather than manipulation." However, specifically with a large input file in mind, I found it useful to be able to yank out the parts of text you need!)

[TaoPhoenix]

This version has an option to name the files by date of processing.

The source and the app are in the same batch this time.

P.S. I also have a proof of concept awk script version 3, but I won't burn a whole other post unless someone wants it. These were the two main versions I commissioned.

I don't recall what bugs were left in this version but I signed off at the time as "good enough". : )

[TaoPhoenix]

This is the grand overall fragment that led me to make this thread.

Before even this year's NANY pledges, but especially in light of them, I dreamed up the idea ... what if you merged a word processor but with custom power-user features?

Warning: This is Pre-Alpha mockup-concept level only! But since my programmer already whacked at it, at least it's half built into a shell that one of you might pick up and run with!

The idea in action:

1. I commissioned "modules" from third tier outsource programmers on simple stuff "that didn't matter if it had an obscure bug".

2. I gave the modules to my project design lead programmer to merge into this processor shell as a feature module. He had the leeway to clean up the initial module codes etc.

We tried to design a shell that you could just for ex drop "coding snacks and NANY's into all day long". So for example, the spawn utililities above "are what they are" ... but what if that was built right into the text reader app? So our proof of concept has a couple of different versions of that algorithm. (Footnote - I didn't burn time above posting the third version - those were enough for you to get the idea. It was Bassem Fawzy's that's in here but not posted standalone above.)

"So if you'll forgive the crudity of the model" (Doc Brown!) here we go!

This project was def at the "Coding Lunch" level, and then I ran out of funds to keep going, plus one feature no longer matters because it's obsolete now etc. (TreeDB into CssZenGarden - A lost interest and B I now use MyInfo and their dev might be able to fix the MyInfo code on his end to make things easier.) But the grand vision was something like merging *all* of your text and file NANY's and Snacks into "one shell to rule them all"! This would be the text Turbo Processor that did EVERYTHING!

1. The shell is based off the Scintilla project. We put a little work into finding "open licensed" source shells to drop into.

2. The first few menus are your typical word processor ones. For you programmers, it has Scintilla's pre-built highlighting support per language.

3. But the real power is in the menus like Data Control, Tools, and Options. Unfortunately the only module we had time to put in was the file spawner. But for ex if you didn't like how the spawn files came out, the app can nuke them too (or any other junk files you don't need after doing some test. )

So I hope this framework inspires someone!

"New Components for the New Year!"

[TaoPhoenix]

PDF Generator

Here is another example of a StandAlone module that could theoretically be part of the Turbo Processor.

Suppose you are doing web research.

For background, MilesAhead wrote me a "Beta" version of his BBSS that creates paired lists of titles and URLs.

Here is Mouser's Newsletter:
https://www.donation....msg343854#msg343854

We have decided we want to look at the NANY section.


1. Open them all up in tabs in Firefox. (Note - there seems to be a bug and this doesn't work right in Pale Moon.)

2. MilesAhead's BBSS reads the tabs and creates a paired list of your research tabs as titles and URLs in ... wait for it ... text files! : )

3. My module reads the URL and then creates matching PDFs for each of those URLs.



Known bugs:

1. As I just found out in this run, there are some unclear problems with the filenames of the resulting PDFs.
2. We put a bit of thought into how Duplicate URLs in a list are handled. I think the file number in the first character is supposed to keep increasing because it was supposed to be line count equal to the number of original URLs. But this is why this is a component - this part needs work.
3. Misc "Look and Feel" bugs - for example if you're working with a "logged in system" such as email or Monster (my original use case) or other, you might get badly formatted PDFs because the app wouldn't have correct login permissions etc. There are others. That's why I have to sign off with this "only as a component".

4. The package is rather large - it uses QtWebKit I think, and that's more than the post limit here. If anyone wants this piece I'll find somewhere to upload it, maybe my private server.

Some info:
The BBSS thread from Miles is here:
https://www.donation...ex.php?topic=30913.0

Attached is the sample title and URL list and a couple of sample output PDFs.

[kyrathaba]

Very nice, Tao! 

[TaoPhoenix]

Bumping this because Asudem likes some of the stuff and I have a bug request into Mouser.

Memo to self
(To find the path look at the C drive read in Super Resources, then type "turbo")

[TaoPhoenix]

Because it's far from clear to anyone, I'm posting notice here that there was a long standing bug for the "Turbo Processor" (Word Processor + custom functions prototype) that was never meant to be a tease.

In original Nany a couple years ago I ran into a bug (that was still there) but I was too tired to chase it down then, so just had to finish my Nany posts and live parts of life.

This week mouser did agree it's a funky bug, so he's still poking at it with a stick, but in the meantime, he smashed it up there. So here's a quick recap a couple years later:

The compressed file is TextInfra.rar
Because of the bug, it's in two places - the original post where it belonged all along and Mouser mashed it into, and the new post on the bottom while we were testing things. They are the same version.

I commissioned all this stuff because I don't program, but my spirit is Nany/CodingSnacks. After checking with Mouser, he decided that if it's posted here in DC spirit it's fine even if it's not technically hard coded by me; I design and commission my stuff and leave the things like language choice to devs, mostly from oDesk so far.

So this was my prototype of the concept you saw years ago. There's tons of problems with it, but the idea was in for example the Data Control menu are functions you would never see in any standard word processor but resemble a lot of stuff you see here as individual modules. Unfortunately they're pretty broken! : (

But at least it's up now and it's free/available for people to whack at.

[TaoPhoenix]

By my lazy blind guestimate, we have *over a hundred* text widget-snacks-apps here on DC. So not counting coding language clashes, the idea was you just make each one of them called as a module from a menu and then you have a super-processor that just does everything! : )