Big Data tools

[kalos]

Hello!
Is there a 'free for commercial use' software that opens and processes with regex large text files of 20GB?
Thanks!

[Ath]

Is there a 'free for commercial use' software that opens and processes with regex large text files of 20GB?

TL;DR; Yes.

Suggestions:

• Any Linux distro with a non-commercial license (most will fit)
• Linux-tools for Windows (if it's supposed to run on Windows, you didn't say it should)

In any of these environments use tools like grep, sed, awk, perl, python etc. for text processing.

• 'Plain' Windows (7 and newer)[/i]

Use powershell, like shown & explained in your other thread.

[Shades]

RJ TextEd

Is freeware, multi-lingual and the website says it is allowed to be used in commercial environments too. Donations are accepted. This text editor has support for regex and I tested (previous versions of) the software with 2GByte log files and search/replace actions in those files were fast on a 7 year old PC (16GB RAM, i5 CPU (3th generation, 3.5GHz)).

Caveats:
- Regex I haven't played with, so how well/functional the support is I can't tell.
- Haven't tested with log files bigger than 2GByte, so if 20GByte log files can be processed, I can't tell.

[Ath]

Haven't tested with log files bigger than 2GByte, so if 20GByte log files can be processed, I can't tell.

Well, I did and it wasn't nice,
Created a large file using this method
Installed RJ TextEd and tried to open it, with this result:

[kalos]

Is there a 'free for commercial use' software that opens and processes with regex large text files of 20GB?

TL;DR; Yes.

Suggestions:

• Any Linux distro with a non-commercial license (most will fit)
• Linux-tools for Windows (if it's supposed to run on Windows, you didn't say it should)

In any of these environments use tools like grep, sed, awk, perl, python etc. for text processing.

• 'Plain' Windows (7 and newer)[/i]

Use powershell, like shown & explained in your other thread.

Thanks but I want to also be able to view the content, like EmEditor

[Ath]

I want to also be able to view the content,

Well, you could just buy EmEditor, it's not that expensive, or EditPad Pro, that does also handle really big files with ease.

For a freebee, check out the really old, and a bit harsh to download, MadEdit, it opened a > 20GB file without a hitch, and supports regex search and replace.


Like I suggested above, and me and others before, it would really get you more helpful answers if you specify what you want, need and desire complete instead of letting us pull those details with pliers, in the dark

[wraith808]

https://stackoverflow.com/questions/159521/text-editor-to-open-big-giant-huge-large-text-files

You can try those and see if any satisfy your need.

From those, my personal recommendation is EmEditor, 010, or LogExpert.

https://github.com/zarunbal/LogExpert

I don't know what processes means, however in that context.  And on this Ath is correct.  You want better recommendations, give better requirements.