DonationCoder.com Software > Finished Programs

DONE: Delete double lines (all but the first) in a text file

<< < (2/6) > >>

TWmailrec:
Re: IDEA: Delete double Lines (all but the first) in a Text-File

The solution from jgpaiva (RepeatedEntries.ahk) solves a problem I had, but can it be modified to ignore blank lines ( CR only to aid intelligability)??

jgpaiva:
Here is a new version, that checks for blank lines.
Note: a line that only has SPACEs or TABS, is considered a blank line. I hope this was what you were asking for.

.exe version
.ahk version

TWmailrec:
Many thanks to jgpaiva for the new program mod.
The repeated strings msgbox now works well, but the output file
did not copy blank lines.
Is there any way to replicate the blank lines in the output file?
Im new to Autohotkey program language & cant cope with loops.

TWmailrec

jgpaiva:
EhEh TW..
You hade some work adapting the script.
There are a few "return"s missing, though.
I didn't get what you meant, you mean the problem was onkly in the messagebox?
You only wanted the msgbox fixed, but still having the blank lines in the file?

Gerome:
Yo !
Here's the modified version i mentioned.
It's algorithm is quite good, but ahk is a script language, so, it takes more time than C, for sure.
It took 1 minute 45 seconds to find repeated entries on a 9000 lines file, on my laptop centrino 2.0.
Still, it does solve your problem.
Doesn't alter the initial file, but the file created doesn't have the repeated entries.
It has a small bug: the progress bar doesn't correspond to the truth. In the end of the file, it's way faster than in the beggining. Just leaving the heads-up, in case you start thinking about giving up at the beggining.
It is supposed to be able to hadle 64mb of plain text, by the ahk references.

Hope it solves your problem.
(btw: the .ahk file needs autohotkey to run, and the exe file only accepts a file called "textfile.txt" as input, and only outputs to a file called "out.txt". Both are in the attached compressed file)
-jgpaiva (March 06, 2006, 05:57 PM)
--- End quote ---

I've taken your script sources copied 2520 times onto themselves : it gave a 3,2 MB text file...
Tested your script under Win2k Sp4 256 Mb Ram without any other programm running and after 1 hour it has only found 50% of the duplicates...
There were only 168 840 lines... and took 35 Mb of RAM trying to aggregate...
Make your own conclusions man...

Navigation

[0] Message Index

[#] Next page

[*] Previous page