Recent Posts

426

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 21, 2018, 12:26 PM »

Any idea why the below does not work?

-kalos (August 21, 2018, 09:42 AM)

As usual you are asking half questions without *any* documentation. And you still haven't answered all previous questions, as requested, (and even have asked new questions in the half-baked 'answer') so in my book, you're not yet ready to ask new questions.

427

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 20, 2018, 08:59 AM »

How can I do that?

-kalos (August 20, 2018, 08:37 AM)

That's why we asked more specific questions, but you never answered them.
So then I gave you the assignment of answering all our unanswered questions, but you haven't done that up until now, so basically, we are waiting (but not holding our breath) for your answers, before accepting new questions.

428

Living Room / Re: Good links in english to the mathematics a programmer needs

« Last post by Ath on August 20, 2018, 01:10 AM »

O.M.G.

429

General Software Discussion / Re: Splinter

« Last post by Ath on August 20, 2018, 01:06 AM »

"hey, if your app is dope and you want to publish the source into a specific software category forum, we will look it over and if it meets whatever standards, you have a spot for a thread dedicated to such n such"

-dmd7978 (August 19, 2018, 11:25 AM)

That would only happen if we lived in an ideal world..., IMHO


Or on DonationCoder, but as there is no way other than sharing a .zip file or your own (dcmembers-) site for sharing source, we usually point to github for that

430

General Software Discussion / Re: difference in windows explorer and window search with file path

« Last post by Ath on August 15, 2018, 01:31 AM »

It is the result of how windows account separation works.
Search is done by a background/service process, and that process is running under another (system) account, not your current-user account. Drive mappings are specific to user accounts, so the service process can't know how your mappings are configured. To start the search, Explorer hands down the unc path of the folder to the search process, so search doesn't have a clue you are even using a mapped drive. Translating that back to a user-local mapped drive is a job for the application receiving the path to the file, as Explorer doesn't do that for you. (Retrieve all mapped drives and substitute the mapping-unc path part with the mapped drive in the found filepath).

431

Skwire Empire / Re: Release: sWeather (tray-based weather app)

« Last post by Ath on August 14, 2018, 06:23 AM »

First I had the sWeather directory in C:\Program Files (x86) which is privileged

-theinfinitypoint (August 14, 2018, 04:51 AM)

This is portable software, and thus shouldn't be installed in Program Files or other protected directories.
Settings are stored in an .ini file in the directory where the .exe is found.

432

Skwire Empire / Re: Release: sWeather (tray-based weather app)

« Last post by Ath on August 14, 2018, 04:39 AM »

Have you tried switching to https:// mode in the settings? AFAICS, sWeather only uses port 80 (http) or 443 (https) for communicating with the Yahoo API, and that would be really funny if blocked by your router.

Do you have an add-blocking dns service in use, maybe a Pi-hole or something similar, or a provider-applied filter? It can be set to block any url that's Yahoo-like...

433

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 13, 2018, 02:05 PM »

OK, so the input is:
<html:products>
    <html:prod id="prod1">
      <html:referenceData>
        <html:product>
          <html:classificationType>PRD</html:classificationType>
          <html:productType>PRD_XE</html:productType>
          <html:productId>10004</html:productId>
          <html:assignedDate>2018-07-23</html:assignedDate>
        </html:product>
        <html:book>
          <html:name>REPAIRS</html:name>
          <html:Entity>REP_XE</html:legalEntity>
          <html:location>ED</html:location>
        </html:book>
      </html:referenceData>
   </html:prod>

The above continues to prod2 etc.

The output of the data would be:
prod1; PRD; PRD_XE; 10004; 2018-07-23; REPAIRS; REP_XE; ED
Then a new line would start with:
prod2; etc

-kalos (August 13, 2018, 05:03 AM)

That finally makes some sense. Here is an example solution for putting that into a .csv formatted file.
You didn't give the specification for that html: namespace though. (But as it's the only namespace used, for data-extraction it can be filtered out)

  But, earlier in this thread you wrote this:

<CATALOG>
 <PLANT>
 <COMMON>Bloodroot</COMMON>
 <BOTANICAL>Sanguinaria canadensis</BOTANICAL>
 <ZONE>4</ZONE>
 <LIGHT>Mostly Shady</LIGHT>
 <PRICE>$2.44</PRICE>
 <AVAILABILITY>031599</AVAILABILITY>
 </PLANT>

-kalos (August 06, 2018, 03:26 AM)

Well guys, the data is what I posted in my last post (Plants),

-kalos (August 06, 2018, 07:01 AM)

It doesn't even look a teensy bit like this new data you've given just now, are you playing us?

However, I want to convert the input data in a string, because, I may need to match longer substrings than eg "<html:classificationType>(.+?)</html:classificationType>"

-kalos (August 13, 2018, 05:03 AM)

You are talking b.s. here.

Also, I think there may be duplicates for each prod, e.g. more than one assignedDate node with different values, so MatchAll would be best.

-kalos (August 13, 2018, 05:03 AM)

This doesn't make sense without an example, and MatchAll is inappropriate here.

extract the appropriate regex

-kalos (August 13, 2018, 10:47 AM)

PLEASE STOP TELLING US HOW TO SOLVE YOUR CHALLENGE!
(This could have been bigger and in red, but I'm trying to stay nice, so I didn't)
If you want to learn regex, go get a book or on-line course, there are plenty here and here, and stop feeding us xml.

When handling XML, no regexes are usually involved, unless the data elements contain 'complex', somewhat structured, data that needs to be broken down.

I have this assignment for you:
- read the entire thread from OP to the end and formulate an answer to all unanswered questions we asked you. (Just quote the question and type the answer below the quote)
After all the answers are given you can ask 1 new question. As 4wd already stated, and you said yourself but in other words, you aren't good in answering questions, but it is required for other people to help you solve your challenge/quest.

434

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 10, 2018, 01:37 PM »

Also, I want to run sequential several regex matches with their own references, one by one and append each result to the output file.

-kalos (August 10, 2018, 09:43 AM)

You have to make clear whether the results from the separate queries have any positional relation to each other, or can the queries be run one after the other and the output of the second, third, etc., runs appended to the first regex run?

435

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 08, 2018, 01:39 PM »

Name, Matches, Groups, Value, something standard in Powershell or they are random variables names?

-kalos (August 08, 2018, 08:59 AM)

4wd did a fine job of explaining all the special chars / shortcuts he used in the post just above yours.
Name is the standard attribute holding, wait for it..., the name of the object.
Matches, Groups and Value are attributes of the Regex resulttype Match (.NET, the base of Powershell, as I explained earlier), containing an entire match, all groups found in the regex provided, with Groups[0] holding the entire result and Group[1] etc. reflecting the standard regex \1 result etc, and value holding the found value, usually the same as Groups[0].

436

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 07, 2018, 09:29 AM »

most of my questions are for my own understanding and may not directly relate to the specific problem.

-kalos (August 07, 2018, 09:01 AM)

You obviously don't understand what we wrote earlier. I already gave 2 (two) possible ways how to handle that. And there are other solutions too.

-Ath (August 07, 2018, 08:50 AM)

To spell it out, again:

• Use a programming or script language
• Use sed
• And a third solution to add to the confusion: Use awk^w

437

Coding Snacks / Re: Custom WinPE build

« Last post by Ath on August 07, 2018, 09:20 AM »

It seems to me that he's running games.  In a console cabinet.  Which means he's probably running one of those book sized machines.

-wraith808 (August 07, 2018, 09:08 AM)

Yes, on a system with 16GB of memory (as it seems) and a quad-core, if that won't perform properly, what else will?... As said, I think he's hunting for ghosts. Overclocking, or selecting a higher frequency CPU, will add more bang than running Win PE with all it's limitations.

438

Coding Snacks / Re: Custom WinPE build

« Last post by Ath on August 07, 2018, 09:06 AM »

You seem to run it on a quad-core CPU with HT. What horrible amount of media do you need to run simultaneously that can't be handled by a normal Windows? Or is it a whoofully inefficient/old system?
And there are ample recipes for stopping any unneeded Windows services, might that be a bottleneck.
IMHO, you are hunting for ghosts.

Btw, do you have a solution for the (free and forced, but usually undesired) reboot after running Win PE for 72 hours?

439

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 07, 2018, 08:50 AM »

You obviously don't understand what we wrote earlier. I already gave 2 (two) possible ways how to handle that. And there are other solutions too.

Like 4wd, I'll pick up again after you start giving complete examples of real pieces of a data file with exact descriptions of what you want to achieve, not of descriptions of how you think it should be handled/solved.

440

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 06, 2018, 02:45 PM »

Can anyone tell me please how to find a regex in a file and append it to a file? Also, how to loop that? Last, how to find the next regex match in the file?

-kalos (August 06, 2018, 02:27 PM)

Well, the trouble is you'll have to do it in some script or programming language, as regex is actually a selection mechanism using pattern matching ('regular expressions').
For such a task I'd advise to use sed, the Stream EDitor, originally from unix, but also available for Windows, that is built for jobs like this.
I've made a Sed-Tester tool for NANY a couple years back, find it from the link below this post and try it out, it includes sed.exe, and has a link to sed documentation in the gui.
You can also continue on the PS-script 4wd gave here earlier, but that doesn't go through the data sequentially in the way sed does.

441

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 06, 2018, 02:14 PM »

If I could convert the XML file in a flat structured file

-kalos (August 06, 2018, 08:13 AM)

Converting your .xml to .csv is a quite easy one-liner in Powershell, assuming a single xml file, into a single .csv file:

Code: PowerShell [Select]

1. $([xml]$(Get-Content .\kalos-data1.xml)).SelectNodes("/CATALOG/PLANT")|Export-Csv .\kalos-data1.csv

The reason it wasn't working for you was probably that you didn't account for xml to be case-sensitive.

Parsing an .xml file containing CDATA tags with a regex is close to impossible to get right, as nearly any content is possible inside such CDATA tag, including valid xml...

Guys, the more I am looking on it, the more I am convinced that Regex would be the best solution.

-kalos (August 06, 2018, 11:36 AM)

Please listen to people with more (programming) experience than you have, you are really trying to hammer round screws into square holes here, don't do that, you'll hurt yourself.

442

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 06, 2018, 05:31 AM »

it will limit my learning a lot

-kalos (August 06, 2018, 03:26 AM)

Well, please first try to learn how to describe your challenge well, a tutorial was linked earlier by 4wd, then we will try to teach you how to best solve your challenge. It may not need regex at all.

A common saying about regexes goes like this: You try to solve a problem with a regex. Now you've got 2 problems...

443

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 06, 2018, 05:23 AM »

,["t--ddbPTeIsNI","iGTzEhwhMx4U","r-iGTzEhwhMx4U",[["debug",null,null,null,null,[null,null,null,null,0]
]
,["ui_mode",1234125123l,[null,null,"inline"]
]
,["num_cols",14351435,[null,null,null,2.0]
]
,["max_timing",235123512,[null,null,null,2500.0]
]
,["check_parent_card",143512122,[null,null,null,null,1]
]
,["counterfactual_logging",213513212412,[null,null,null,null,0]
]
]
]

-kalos (August 05, 2018, 01:55 PM)

<PLANT>
 <COMMON>Bloodroot</COMMON>
 <BOTANICAL>Sanguinaria canadensis</BOTANICAL>
 <ZONE>4</ZONE>
 <LIGHT>Mostly Shady</LIGHT>
 <PRICE>$2.44</PRICE>
 <AVAILABILITY>031599</AVAILABILITY>
 </PLANT>

-kalos (August 06, 2018, 03:26 AM)

A couple of questions:

• In what way do these totally different types/forms of data relate to each other?
• Are they from different files?
• Is the first file some sort of definition file and the second the actual data?
• Is the first part embedded in a CDATA tag like this: "<![CDATA[ your non-xml-formed data like in the first quote goes here ]]>" ?
• Please provide the exact filenames.

444

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 05, 2018, 03:51 PM »

Now we are getting somewhere, sort of.

You only didn't tell what other parts of the data you need extracted from each record, besides the ui_mode field, and what the identifying field is that should go in the first column of the csv output you suggested earlier.

-Ath (August 05, 2018, 03:24 PM)

Is your second line a question?

-kalos (August 05, 2018, 03:48 PM)

Yes, 2 actually.

445

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 05, 2018, 03:24 PM »

Now we are getting somewhere, sort of.

You only didn't tell what other parts of the data you need extracted from each record, besides the ui_mode field, and what the identifying field is that should go in the first column of the csv output you suggested earlier.

446

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 05, 2018, 11:58 AM »

This example is totally useless.

Please extract 2 or 3 of those (complete) product records from your actual data file. Optionally replace confidential stuff (data, prices)with aaaaa, bbbbb, 1.23, etc., but leave the structure exactly as it is!
Then post that here.

447

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 05, 2018, 06:20 AM »

If you are searching for a regex within a regex, 'You Are Doing It Wrong' (T).

You initial requirement was to find and extract content using a regex, but now you need parts of that regex to be split out? That can be done using a single regex, grouping the stuff you need to split out.
And for this whole exersize to make any sense, where is the variable part of the data to find? When searching for explicit text(s), a count would suffice...
Please provide a complete example, with actual data (not an entire file!), clearly marking the stuff you need to extract, of what you want to achieve, not how you think it could/should be solved.

448

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 05, 2018, 03:10 AM »

In the meantime I will read https://www.itprotod...course-you-can-learn

-kalos (August 04, 2018, 04:59 PM)

I expect you comprehend what's written there, it seems quite suited for powershell n00bs.

Thanks but I struggle to follow. I find AHK much more straight forward. But how can I make it work with a 25GB?

-kalos (August 04, 2018, 03:46 PM)

Please stop asking for an AHK solution, those that participated here sofar aren't going to provide it, as a perfect solution is already provided.

If you had tried the script at actual data, you wouldn't have asked again about the 'measily' 25 GB files; yes, ofcourse it will take some time to process, but so does an 18.8 minute powershell crash course.
Powershell is built on the foundation of .NET, so it knows how to handle files efficiently.

Why did you leave these rather important 'details' out in your original question?

-Ath (August 04, 2018, 07:03 AM)

Um .... because that's what he normally does ... it always takes at least a week, (sometimes longer or never), before all pertinent information is obtained ...

-4wd (August 04, 2018, 07:37 AM)

I know, I know, I'm just trying to educate someone (again, but it doesn't seem to be picked up much), see my quote below...

449

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 04, 2018, 07:03 AM »

Also, I need to append to the output file several regex matches/returns, how do I do that?
Also, if I specify a regex match, how do I specify what I want to be returned from this match?

-kalos (August 03, 2018, 05:15 PM)

Why did you leave these rather important 'details' out in your original question?

450

General Software Discussion / Re: Extract REGEX matches from multiple text files

« Last post by Ath on August 03, 2018, 12:16 PM »

The scriptlanguage used is Microsoft's PowerShell, the aimed successor of cmd with its relatively poor language batch (.bat/.cmd) scripts, that comes standard installed with Win10, Win8.1 and Win8, and can easily be installed on older Windows versions.

Copy the script to a file with .ps1 extension, adjust the 1st line to your desired resultsfile, adjust in the 3rd line *.txt to the extension of your data files, press the Start button and start typing powershell to find that, then run the script from the directory where your data files are.
Largish files are no issue for PowerShell.