topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Tuesday March 19, 2024, 3:43 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Any webserver-desktop-logfile-analysis-Software recommendations?  (Read 10439 times)

crono

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 179
    • View Profile
    • Donate to Member
Hello everyone,

Im searching a programm for analyse my Apache-Access Logfiles. On the server Im using http://www.awstats.net/ but I want something which is not web-based. Any recommendations?

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Any webserver-desktop-logfile-analysis-Software recommendations?
« Reply #1 on: September 07, 2006, 08:15 AM »
ive used a ton and will try to post some thoughts later - am also interested in other people's views.

Gothi[c]

  • DC Server Admin
  • Charter Honorary Member
  • Joined in 2006
  • ***
  • Posts: 873
    • View Profile
    • linkerror
    • Donate to Member
Re: Any webserver-desktop-logfile-analysis-Software recommendations?
« Reply #2 on: September 07, 2006, 08:43 AM »
please do post some,.. the only one i tried was a php thing and it was crap :D

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Any webserver-desktop-logfile-analysis-Software recommendations?
« Reply #3 on: September 07, 2006, 10:55 PM »
Analog - Web Storming - ClickTrax - Urchin - roll your own...

The best solution is to actually roll your own. That way you get to include real business logic.

Web log analysis is at best sketchy and doesn't provide you with "true" numbers. They are only a rough guess. (Another reason to roll your own.)

The best solution is to run things on the client for detection and store it on the server in a database. You also get the added advantage of only storing what you need.

All large scale web analysis must be done this way because logs get too large to process. Even on a relatively small site (or medium depending on what you consider large) you can have log files that are GBs for a day. Very small sites have log files in the 10's of MB per day.

Most software can't keep up with that kind of volume. That's where a dataserver can solve the problem. They're optimized to deal with data processing whereas your average log analysis software just can't compare. (A lot of resources go into optimizing RDBMSs like MS SQL Server, DB2, Informix, Oracle, etc.)

But for small sites this can often be overkill. A quick over view with Analog is good for most people with hobby sites or small web sites.

I'm not a fan of AwStats. I've found it just miserable to work with.

Another thing you can do is to write a program to dump your logs into a DB then just use pure SQL to ask questions. That's a cheap and easy way, though it is a bit time consuming as you'll still end up waiting for queries. But the quality of answers you can get is VERY high.
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

crono

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 179
    • View Profile
    • Donate to Member
Re: Any webserver-desktop-logfile-analysis-Software recommendations?
« Reply #4 on: September 11, 2006, 06:49 PM »
Thanks you very much Renegade. What you write sounds really experienced. I think Im going to roll my own - which shouldnt be to hard... Parsing into a DB and running some SQL should be straight forward :)

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Any webserver-desktop-logfile-analysis-Software recommendations?
« Reply #5 on: September 11, 2006, 08:25 PM »
i guess my view would be the opposite to renegade's.  the idea of rolling my own when sooo much work has been done on this seems a bit crazy.

i also tend to like local apps, and there are some good ones for the pc (Mach 5 Analyzer is pretty nice and fast).

though its true that if you start getting 1gb log files, these programs start to choke and you need to find alternative strategies.

Gothi[c]

  • DC Server Admin
  • Charter Honorary Member
  • Joined in 2006
  • ***
  • Posts: 873
    • View Profile
    • linkerror
    • Donate to Member
Re: Any webserver-desktop-logfile-analysis-Software recommendations?
« Reply #6 on: September 12, 2006, 04:32 AM »
I didn't like analog too much.
I'd much rather see something in between rolling your own and not, that is:
  a commandline utility that runs in a terminal that will sort through the logs as they come in, with colored output and statistics, as opposed to web-based solutions. grep'ing through the logs and using tail -f works too, but i'd like to see it better formatted, with a pretty ncurses interface or something. just like htop looks nicer than top :) And it'd be handy to have counters of pageviews (even though they don't mean anything), failed requests, intrusions, etc,..
Maybe with pluggable progressbar-like-meters as in htop. :)

housetier

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 1,321
    • View Profile
    • Donate to Member
Re: Any webserver-desktop-logfile-analysis-Software recommendations?
« Reply #7 on: September 12, 2006, 01:55 PM »
Today I found Logfilter: An Apache Log Filter.

Logfilter is a tool for performing ad-hoc filtering of apache website log files.

This seems to be tightly coupled to the Apache Webserver.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Any webserver-desktop-logfile-analysis-Software recommendations?
« Reply #8 on: September 12, 2006, 02:13 PM »
house i've been looking for something like that for a while, nice find. :up:

Terry

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 25
    • View Profile
    • Falcon21 Home PC Security
    • Donate to Member
Re: Any webserver-desktop-logfile-analysis-Software recommendations?
« Reply #9 on: September 16, 2006, 03:18 PM »
AlterWind Log Analyzer - http://www.alterwind.com

kartal

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 1,529
    • View Profile
    • Donate to Member
Re: Any webserver-desktop-logfile-analysis-Software recommendations?
« Reply #10 on: August 18, 2008, 07:22 PM »
I like Alterwind, it is neat. I also am looking one for that also shows the cities in different countries rather than only countries.

AlterWind Log Analyzer - http://www.alterwind.com