Messages

1

Post New Requests Here / Re: Idea - Return *new* lines from a text file (w/ example)

« on: August 09, 2010, 12:31 PM »

Is this supposed to be a general-purpose utility, or "pretty specific"? Do you want the program to output new lines and exit, or would continually running & monitoring work? This all affects strategies

If the program can run continuously, it can monitor changes and only read the file on change - and it can rate-limit this to whatever/second in case of often modified files, plus it can cache the "where'd I toddle off to last" in memory.

If you want run-dump-exit, there "where'd I toddle off to last" will have to be stored somewhere. If the file(s) will always be on NTFS partitions, ADS could be a good idea.

-f0dder (August 09, 2010, 12:25 PM)

General purpose was what I was thinking.  Definitely run and exit though. 

CmputrAce's idea of the utility returning the file size to stdout on the first call and then on the second call if the filesize is supplied it would just return the new lines.  This would be easy enough to handle because you could just use whatever program\script is calling the utility (AHK in my case) to call and store the size and then just supply that size on the second call.

2

Post New Requests Here / Re: Idea - Return *new* lines from a text file (w/ example)

« on: August 09, 2010, 09:07 AM »

From what I recall from some #ahk IRC conversations a while back, the original poster is calling this function four times per second.  Also, the file to be retrieved is over a network and not a local file.  strictlyfocused02, please correct me if I'm wrong.

-skwire (August 09, 2010, 08:54 AM)

Im impressed at your memeory, skwire .  You are correct that it is being called four times a second, but its a local file.  The file can get quite large after a short while (100,000+ lines).

3

Post New Requests Here / Re: Idea - Return *new* lines from a text file (w/ example)

« on: August 07, 2010, 08:56 AM »

1. The calling script is written in what language?
2. Do you want a command-line app that just returns the new test to stdout (CON:)?

I have a few thoughts... if I can write a function that you can embed in your script, then keeping the file size between calls is much easier. If I write a quick DOS program to do this, then I'd build it this way:
if you call the program with only the file name, I will return the file size to you. If you call the program with the file name AND the file size, I will return the new file size, a newline, then the text beginning at the character+1 position you submitted as the second parameter to the call.

This should be pretty easy, IMO.

-CmputrAce (July 16, 2010, 08:23 AM)

1. The calling script is written in AHK
2. Command line going to to stdout would be *perfect*

Regarding your other thoughts on it, the DOS route would be ideal following the procedure you mention.

P.S. Sorry for the delayed response, crazy busy at work with a new promotion =D

4

Post New Requests Here / Re: Idea - Return *new* lines from a text file (w/ example)

« on: July 08, 2010, 05:25 PM »

Could you:

1) use a "tail" utility to save the last n lines
2) when you want to check what's changed, use "tail" again to save the new tail
3) use a "diff" utility to see the difference?

The UnxUtils project on SourceForge contains command-line versions of both "tail" and "diff."

-rjbull (July 08, 2010, 03:39 PM)

I suppose this could make this work, but it will definitely end up convoluted. I would probably have to tail the last 500 lines twice and then diff them which would really suck if only 2 lines have been added.  Even worse if more than 500 lines have been written a lot of info could get overlooked.

Even if someone more familiar with AutoHotkey were able to pinpoint where the function I posted is flawed that would be fine.

5

Post New Requests Here / Re: Idea - Return *new* lines from a text file (w/ example)

« on: July 08, 2010, 02:55 PM »

You just need the common tail utility.

I frequently use a Windows GUI-based version called BareTail, which is free and recommended.

-CWuestefeld (July 08, 2010, 12:05 PM)

You just need the common tail utility.

-CWuestefeld (July 08, 2010, 12:05 PM)

I thought of that, but I didn't think it was smart enough to only display the newest lines and not just the last N lines.

-daddydave (July 08, 2010, 12:10 PM)

daddydave is correct, tail performs a similar function to what Im looking for, but lacks in one area. The log could have 3 lines written to it or it could have 300 lines written to it, I would like to be able to retrieve and store solely those new lines.

Hopefully to save some time, I have attempted to use various line counting FileRead techniques in AutoHotkey but with my log files being potentially enormous this takes forever.

6

Post New Requests Here / Re: Idea - Return *new* lines from a text file (w/ example)

« on: July 07, 2010, 11:54 AM »

Not sure if you want to do it this way, but it looks like it can be done from a batch file:

if not exist file.txt exit
fc oldfile.txt file.txt | find /v "*****"
copy file.txt oldfile.txt
pause

-daddydave (July 07, 2010, 11:46 AM)

I cant copy the files. The log files being read can sometimes be well over 10mb so a copy based solutions isnt pratical.

7

Post New Requests Here / Re: Idea - Return *new* lines from a text file (w/ example)

« on: July 07, 2010, 09:55 AM »

Point of clarification: Are the new lines guaranteed to be at the end of the file?

-daddydave (July 07, 2010, 09:54 AM)

Yes

8

Post New Requests Here / Idea - Return *new* lines from a text file (w/ example)

« on: July 07, 2010, 09:28 AM »

I would love for a proper version of this function I found which returns *solely* the new lines from a text/log file. The first call gets the file size and the second call will return only the text that has been added since the first call. Im not sure if there is any other information someone would need from me to make this happen but I would be more than happy to answer any questions to the best of my ability.  

I have a semi-working function of this that was written in AHK. The reason I say semi-working is because in my script this function is called very often and after a couple hundred calls it causes the local machine to hard-lock.

P.S. - The original author of this function (denick) has since removed the code from his post because of the issue I mentioned above.

[Select]

;Returns new lines from a text file.  First call gets FileSize, 2nd+ call gets new lines.  Converted German to English.
FileTail(File) ;;by denick (http://de.autohotkey.com/forum/topic4619.html)
{
   Static OPEN_EXISTING := 3
   Static GENERIC_READ := 0x80000000
   Static FILE_SHARE_READ := 1
   Static FILE_SHARE_WRITE := 2
   Static FILE_SHARE_DELETE := 4
   Static FILE_BEGIN := 0
   Static INVALID_HANDLE_VALUE := -1
   Static CF := ""   ; Aktuelle Datei (Current File)
   Static FP := 0    ; Aktuelle Position in der Datei (File Pointer)
   FH := 0           ; Dateihandle (File Handle)
   FS := 0           ; Dateigröße (File Size)
   FC := ""          ; Inhalt der gelesenen Zeilen (File Content)
   CL := 0           ; Länge des gelesenen Zeilen (Content Length)
   If (File != CF) {
      CF := File, FP := 0
   }
   FH := DllCall("CreateFile"
               , "Str",  File
               , "UInt", GENERIC_READ
               , "UInt", FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE
               , "UInt", 0
               , "UInt", OPEN_EXISTING
               , "UInt", 0
               , "UInt", 0)
   If (FH = INVALID_HANDLE_VALUE) {
      CF := "", FP := 0
      MsgBox, 262160, FileTail, The file *%File%* could not be opened!
   } Else {
      DllCall("GetFileSizeEx"
            , "UInt",   FH
            , "Int64P", FS)
      If (FP = 0 Or FS <= FP) {
         FP := FS
      } Else {
         DllCall("SetFilePointerEx"
               , "UInt",  FH
               , "Int64", FP
               , "UInt",  0
               , "UInt",  FILE_BEGIN)
         CL := VarSetCapacity(FC, FS - FP, 0)
         DllCall("ReadFile"
               , "UInt",  FH
               , "UInt",  &FC
               , "UInt",  CL
               , "UIntP", CL
               , "UInt",  0)
         VarSetCapacity(FC, -1)
         FP := FS
      }
      DllCall("CloseHandle", "UInt", FH)
   }
   Return FC
}

9

Post New Requests Here / Re: Flush system cache using AHK (NtSetSystemInformation API)

« on: June 14, 2010, 01:30 PM »

Mouser in the IRC channel was kind enough to help me out to get this far, but our collective knowledge wasnt qutie enough to finish the function.  Heres where Im at:

[Select]

p := ClearCache()

ClearCache()
{
   VarSetCapacity(Var, 36)
   NumPut(-1, Var, 12, "ULong")
   NumPut(-1, Var, 16, "ULong")
   DllCall("ntdll.dll\NtSetSystemInformation", "UInt", 0x15, "UInt", &Var, "UInt", 36)
   Return
}

Anyone able to point me in the right direction?

Edit - BTW, mouser found and shared THIS with me showing the structure for what were working with here

10

Post New Requests Here / Flush system cache using AHK (NtSetSystemInformation API)

« on: June 07, 2010, 02:20 PM »

Im trying to write a function that when called will perform the same actions as Microsoft's CacheSet tool to flush the system cache.

Here is my non-working attempt:

[Select]

p := ClearCache()

If not p
   MsgBox, Success
If p
   MsgBox, Fail`n%p%

ClearCache()
{
   k := DllCall("ntdll.dll\NtSetSystemInformation", "Str", SYSTEM_CACHE_INFORMATION, "UInt", -1, "UInt", -1)
   If not k
      k := DllCall("Kernel32.dll\GetLastError")
   Return k
}

P.S. - I dont know how much help it will be but just in case, HERE is a topic I started on the AHK forums that never amounted to much ...
P.P.S - It was recommended to me to also post the CacheSet source ... Here ... Line 257 is the start of the function Im interested in.