topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Friday March 29, 2024, 2:10 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Needed ... MD5 checksum utility  (Read 42976 times)

Carol Haynes

  • Waffles for England (patent pending)
  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 8,066
    • View Profile
    • Donate to Member
Needed ... MD5 checksum utility
« on: November 19, 2005, 10:07 AM »
Ideally what I need is a utility that will take two folders and recursively calculate MD5 checksums (or similar) on both folders and compare them to check that two folders and their contents are identical.

Whn I upgraded my hard discs I copied all the partitions from my old discs using Partition Magic in DOS mode.

This seems to have worked fine but before I delete the old partitions I want to ensure that they are bit perfect.

What I need is:

Input HD/File/Folder1 HD/File/Folder 2
Compare on a file by file basis

Output:
files in one location and not in the other
compare files where they both appear and display warnings for files with different MD5 values

I have run CHKDSK on all my drives and there don't seem to be disc issues - but I am not sure if Partition Magic takes the manufacturers bad sector file into account when copying partition contents, and there doesn't seem to be any way in Windows to compare binary files easily (on a rang of files).

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #1 on: November 19, 2005, 10:34 AM »
more than what you are asking for but
the folder comparison tools will do this stuff and more of course.
beyond compare is my choice.
winmerge is free and should do it too.

Carol Haynes

  • Waffles for England (patent pending)
  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 8,066
    • View Profile
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #2 on: November 19, 2005, 11:28 AM »
Thanks, I'm having a look at Beyond compare. It has a 30 day trial which is useful.

I also found http://www.fastsum.com/ which has a useful utility for this sort of thing. It will go through whole trees and produce MD5 values for individual files / folders - storing them in the folder. Then you can just run the utility and it will calculate and compare the MD5 for the current file and last MD5 calculation.

This is idea for future copying (since the MD5 values will be copied too) but not that easy at the mo to use.

Anyone got any idea how to copy all *.md5 files recursively from one tree to an identical tree on another disc ??

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #3 on: November 19, 2005, 12:11 PM »
note beyond compare is in our discount this month, and we are giving away some copies in the drawing.
also see our review of such tools: https://www.donation...pareTools/index.html

it's one of my favorite programs.


Carol Haynes

  • Waffles for England (patent pending)
  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 8,066
    • View Profile
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #4 on: November 19, 2005, 01:51 PM »
Yes, I have spotted the discount ... thanks

Carol Haynes

  • Waffles for England (patent pending)
  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 8,066
    • View Profile
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #5 on: November 19, 2005, 02:00 PM »
Mouser - (anyone) any idea on this ...

Using BeyondCompare it says two files are not the same. Here is the screen shot - can anyone see a difference or explain?


mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #6 on: November 19, 2005, 02:09 PM »
hahah - they sure look the same.

but i'll bet its that they have different settings in:
archive bit, readonly bit, etc.

you can configure bc to igore or care about such things.

bc can be a bit confusing at first but once you get used to it its very powerful and convenient too (once you learn where things are).

Carol Haynes

  • Waffles for England (patent pending)
  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 8,066
    • View Profile
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #7 on: November 19, 2005, 02:12 PM »
Here is another - and I have displayed all colmns - so it isn't attributes ...

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #8 on: November 19, 2005, 02:16 PM »
could be one of the other 2 timestamps that are not shown?
modification date, read date, creation date?

Carol Haynes

  • Waffles for England (patent pending)
  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 8,066
    • View Profile
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #9 on: November 19, 2005, 02:24 PM »
I notice in the browser context menu is a 'Compare Contents ..." option. If I use that on all the files flagged so far it says they are identical ???

Strange.

Timestamps should be the smae as they were copied a a straight aprtition copy.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #10 on: November 19, 2005, 02:42 PM »
bc has 2 kinds of compares that it does, a quick compare and an indepth compare.
in the main window you can also select files and hit the "?=" button to have it do an indepth compare.

Carol Haynes

  • Waffles for England (patent pending)
  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 8,066
    • View Profile
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #11 on: November 19, 2005, 02:53 PM »
I think I discovered the prob - I had selected treat archives as directories and it was struggling to open non-standard CAB files.

Innuendo

  • Charter Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 2,266
    • View Profile
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #12 on: November 19, 2005, 03:24 PM »
Only thing I can think to add to this thread is that you are using the right tool for the job, Carol. Beyond Compare is widely considered the best program for this sort of work & it's the gold standard that all other programs of this ilk are compared to.

jpfx

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 155
    • View Profile
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #13 on: November 21, 2005, 08:36 AM »
I don't really see the point because if the old files were in any way damaged, all you have is a bit perfect copy of a damaged file?
The same would apply to a bit for bit copy of a partition wouldn't it?
       |\      _,,,---,,_         
ZZZzzz /,`.-'`'    -.  ;-;;, 
      |,4-  ) )-,_. ,\ (  `'-'    
     '---''(_/--'  `-'\_)

Carol Haynes

  • Waffles for England (patent pending)
  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 8,066
    • View Profile
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #14 on: November 21, 2005, 09:50 AM »
True - but I wanted to verify that the partition copy was valid. I did find a handful of files that seemed to have copy problems - at least I could correct them.

The other thing is I want to keep a load of stuff offline for security, but online for quick access - a quick utility to check integrity means any disk faults can be fixed easily.

I do find Windows FS (FAT and NTFS) handling totally crap at integrity. If you get a bad block on your hard disk the first thing you know about it is when you run CHKDSK - it doesn't even seem to flag errors when you try and load a damaged file!!

Anyway I have settled on two tools: BeyonDCompare 2 and FastSum.

Beyond compare is great for comparing folders of files, whereas FastSum stores hidden MD5 files (either one per file or one per folder) so that you can just hit "Check MD5 data" in the Explorer Context menu. It will flag any files/folders that don't match their MD5 value (so quick to track down problems) and automatically update the MD5 files for any additions since the last MD5 build. Neat little tool and cheap.

jpfx

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 155
    • View Profile
    • Donate to Member
Re: Needed ... MD5 checksum utility
« Reply #15 on: November 21, 2005, 10:57 AM »
yeah, windows file systems go bad with nary a peep out of the OS.
       |\      _,,,---,,_         
ZZZzzz /,`.-'`'    -.  ;-;;, 
      |,4-  ) )-,_. ,\ (  `'-'    
     '---''(_/--'  `-'\_)