|
f0dder
|
 |
« on: August 27, 2009, 01:07:07 AM » |
|
I've seen a few inquiries here and there for people who need to deal with huge text files - we're talking the multi-gigabyte range here. Usually it's for log files and the people really just need viewing (and grepping) and not editing, but edit facilities might be necessary every now and then. Does any of you guys know a text viewer (or, better, editor) that handles such files without hiccup? We're obviously talking something that doesn't use 32bit variables for file size or line count, and doesn't try to load the entire file at once... I'm pretty sure I bumped into such an editor years ago, but haven't had the need for huge files since. Freeware would of course be preferred, but any suggestions are welcome. Some people seem to think that you'd need a 64bit editor to handle such big files, but imho it's perfectly possibly to handle with a (smartly programmed) 32bit editor... if you need to handle really big files, it's folly to try and do full-memory loading anyway  |
|
|
|
|
Logged
|
 - carpe noctem |
|
|
|
|
|
f0dder
|
|
« Reply #2 on: August 27, 2009, 02:57:59 AM » |
|
Doesn't VIM load the entire file into memory? EDIT: I know UltraEdit can handle files > 4gig, and doesn't try to load the entire file... but I'd like to see other alternatives |
|
|
|
« Last Edit: August 27, 2009, 03:34:20 AM by f0dder »
|
Logged
|
- carpe noctem |
|
|
|
|
jgpaiva
|
|
« Reply #3 on: August 27, 2009, 03:48:43 AM » |
|
From what I know, vim does work with large files. It does take a while to load the file, however. I've just found this plugin which will probably solve that problem |
|
|
|
|
Logged
|
|
|
|
|
f0dder
|
|
« Reply #4 on: August 27, 2009, 04:03:43 AM » |
|
From what I know, vim does work with large files. It does take a while to load the file, however. I've just found this plugin which will probably solve that problem housetier already posted that script - taking a while to load kinda indicates to me that it does full loading? I can't see why disabling an undo system would make initial file opening faster, for instance. |
|
|
|
|
Logged
|
- carpe noctem |
|
|
|
qwibbles
|
|
« Reply #5 on: August 27, 2009, 04:13:40 AM » |
|
http://en.wikipedia.org/w...omparison_of_text_editorsLook for Large file supportEditPad I think was ok for this and LTFViewer - Had to open and search very large website logs once.. LTFViewer was very fast to open as it only open's a section of the file. A few seconds for 10GB file TextPad and UltraEdit are also generally ok with this size of file. You would have to test them for your application of course and see which one suits you.
|
|
|
|
|
Logged
|
|
|
|
|
f0dder
|
|
« Reply #6 on: August 27, 2009, 04:35:28 AM » |
|
Well, I looked at that, but I'm not sure the feature matrix is so useful... what do they mean by "large file support"? Is it 2GB, 4GB, or "available memory"? It says nothing about memory vs. diskbased either EditPad I think was ok
Their site says: " Edit huge files without breaking a sweat. EditPad Pro will instantly open files up to 2 GB in size, even if your PC has less than 2 GB of RAM." - what is with people using 32bit signed integers for file sizes? As if 32bit itself wasn't bad enough, wtf use SIGNED integers? Ever seen a negative-size file? >_< TextPad and UltraEdit are also generally ok with this size of file.
TextPad doesn't seem to do the trick, quoting their features: " It can handle file sizes up to the largest contiguous chunk of 32-bit virtual memory." LTFViewer sounds like something that could be useful PS: thanks for your effort, even if this post sounds a bit critical overall |
|
|
|
« Last Edit: August 27, 2009, 04:49:05 AM by f0dder »
|
Logged
|
- carpe noctem |
|
|
|
jgpaiva
|
|
« Reply #7 on: August 27, 2009, 04:43:15 AM » |
|
housetier already posted that script Oops.. Sorry. I just tested it with a large ISO, but even with the plugin, looks like it'll take a few minutes to load (I've waited for about 1min). |
|
|
|
|
Logged
|
|
|
|
|
qwibbles
|
|
« Reply #8 on: August 27, 2009, 05:08:03 AM » |
|
|
|
|
|
|
Logged
|
|
|
|
|
f0dder
|
|
« Reply #9 on: August 27, 2009, 11:05:46 AM » |
|
...and I even posted in that thread  It was mainly a thread about viewing though, this thread also hints at editing  |
|
|
|
|
Logged
|
- carpe noctem |
|
|
|
wraith808
|
|
« Reply #10 on: August 27, 2009, 12:41:18 PM » |
|
Well, without the editing there's V. That's what I use to view.
|
|
|
|
|
Logged
|
|
|
|
|
urlwolf
|
|
« Reply #11 on: August 27, 2009, 03:43:29 PM » |
|
there's an editor that trumps all others for large files: emeditor. No contest
|
|
|
|
|
Logged
|
|
|
|
|
qwibbles
|
|
« Reply #12 on: August 27, 2009, 04:05:12 PM » |
|
Or you could just split the file with a file splitter so it's smaller  I number of Hex Editors Support very large files HxD is a free one that works on 5GB files on my machine. Frhed and 010editor may also work for you. That just about exhausts me ..... I need to go lie down....  Edit: off to play with emEditor  |
|
|
|
« Last Edit: August 27, 2009, 04:07:16 PM by qwibbles »
|
Logged
|
|
|
|
|
lothark
|
|
« Reply #13 on: September 28, 2009, 12:54:36 AM » |
|
You could also try command line less which will instantly open and read files of any size without loading them into memory. You can get it here. |
|
|
|
|
Logged
|
|
|
|
|
