How 4 Microsoft engineers proved that the “darknet” would defeat DRM

[Carol Haynes]

Best solution for a media server would be simple JBOD - just turns any old bunch of disks into one apparently enormous disk.

[40hz]

Best solution for a media server would be simple JBOD - just turns any old bunch of disks into one apparently enormous disk.

-Carol Haynes (December 03, 2012, 10:25 AM)

JBOD is one approach in that media serves really need to have a "scale-out" (ala GlusterFS) as opposed to "scale-up" capability since you never know how much space and resources you'll need until you need it. And being able to add it piecemeal and as needed is the most cost effective way to do it. Especially since there won't be anything on most media servers that can't be replaced/reloaded relatively easily.
 

[tslim]

If you are talking about RAID 0, again a synchronizing  program which synchronizes between a pair of 2TB HDD periodically seems to be a better solution then RAID 0, because you can copy file to the HDD faster.

-tslim (December 02, 2012, 08:37 AM)

Eh... wat? Copying a file to a single harddrive is no faster than copying it to a raid mirror. The same data chunk can be written to all mirrored drives in parallel, and there's practically no CPU overhead. There's a fair amount of filesystem traversal (which means both CPU and HDD overhead) if you do periodic syncing.

-f0dder (December 03, 2012, 09:36 AM)

"There's a fair amount of filesystem traversal (which means both CPU and HDD overhead) if you do periodic syncing."
Are you implying the act of transferring data to RAID O mirrors is not HDD overhead and do not required CPU?

1. If there is practically no CPU overhead, then who starts the copy of data to all the mirrors? Are we talking about hardware raid or software raid here? Have you practically experience running Windows on RAID 0 and compare that to running on a single HDD? Man... try it and I can assure you the difference.

2. Using synchronizing program means resource consumption is irrelevant, you just need to to schedule it at the right time.

[wraith808]

Just curious, why don't you rip your DVDs and keep them in ISO format (like I do)?
If you convert them into mp4 format, you tend to loss quality, further more what about the DVD startup menu? Are you doing your own authoring work?

May I also ask this, what is the advantage of keeping a movie in mp4 format contained in a mkv file than directly keep it as mp4 file? (Isn't mp4 file sort of container by itself?)

-tslim (December 03, 2012, 07:36 AM)

1. All players don't support mounting ISOs.  MKV is the format that I chose that works on everything I want to use it for.  
2. I'm not encoding it- I'm transcoding it (and that should be mpeg2, not mp4).  There is a small difference in size because MakeMKV strips out the extraneous information.  But it's not compressed.  
3. There's also a difference in size because I don't get audio and subtitle tracks I don't need.  I only speak English, so I don't need spanish, chinese, etc... this lets me do that.
4. I don't have the menus- in fact, I don't want them.  I just want to click a file and it start playing.

[40hz]

@Wraith - ever consider doing a how-to or thread on how to do all this?

There's plenty up on the web - but most of it seems directed at the media/home theater aficionado and not the ordinary viewer who is just looking for a simple approach like yours.
 

[wraith808]

I guess I could... the only problem with this (and the others I've seen on the web) is that a lot of this is player specific.  Even though I have my setup done... I have three other devices that can't work with the setup I have.  There's so much out there in the way of hardware and software that it just changes too fast.  I think the others sort of ignore that X factor, and speak as if their way is the correct way, when the correct way is actually what works for you.

Oh... and if you want an idea of how big mine actually is- this is mine as of 5 years ago.

http://invelos.com/D...ction.aspx/wraith808

I've slowed down since then, and just got tired of entering DVDs into DVD Profiler. 

[4wd]

Best solution for a media server would be simple JBOD - just turns any old bunch of disks into one apparently enormous disk.

-Carol Haynes (December 03, 2012, 10:25 AM)

Just been wondering what to do about all my media files spread over 3 HDDs, was thinking of drive pooling, (including JBOD), but there are no free ones that suited, (Liquesce can't share).

Ended up doing simple junction points on the server...now a 250GB HDD apparently holds 4.5TB of files.  Saves a bit of navigation from the media player.

[wraith808]

Ended up doing simple junction points on the server...now a 250GB HDD apparently holds 4.5TB of files.  Saves a bit of navigation from the media player.

-4wd (December 04, 2012, 12:03 AM)

That's actually a very cool idea!  I'd not thought about using junctions, even though they've been very useful to me in the past.  I've been bitten by having one volume before- things spread among disks, and the controller failing, and not being able to get another controller to work with the array, so losing everything on all the disks, because of not knowing where things are.  Not sure if it's better these days- but once bitten, twice shy.

[tslim]

Just curious, why don't you rip your DVDs and keep them in ISO format (like I do)?
If you convert them into mp4 format, you tend to loss quality, further more what about the DVD startup menu? Are you doing your own authoring work?

May I also ask this, what is the advantage of keeping a movie in mp4 format contained in a mkv file than directly keep it as mp4 file? (Isn't mp4 file sort of container by itself?)

-tslim (December 03, 2012, 07:36 AM)

1. All players don't support mounting ISOs.  MKV is the format that I chose that works on everything I want to use it for. 
2. I'm not encoding it- I'm transcoding it (and that should be mpeg2, not mp4).  There is a small difference in size because MakeMKV strips out the extraneous information.  But it's not compressed. 
3. There's also a difference in size because I don't get audio and subtitle tracks I don't need.  I only speak English, so I don't need spanish, chinese, etc... this lets me do that.
4. I don't have the menus- in fact, I don't want them.  I just want to click a file and it start playing.

-wraith808 (December 03, 2012, 01:28 PM)

I don't know about the "transcoding" of movie, my main concern is to store my DVD movies as original as possible. Particularly when I have a DTS movie, no matter how hard I try to convert it from VOB format (I have tried various formats), I always find loss of DTS quality and I finally decide to store the original image of the DVD.

[wraith808]

I don't know about the "transcoding" of movie, my main concern is to store my DVD movies as original as possible. Particularly when I have a DTS movie, no matter how hard I try to convert it from VOB format (I have tried various formats), I always find loss of DTS quality and I finally decide to store the original image of the DVD.

-tslim (December 04, 2012, 08:46 AM)

from Wikipedia^w

Transcoding is the direct digital-to-digital data conversion of one encoding to another,[1] such as for movie data files or audio files. This is usually done in cases where a target device (or workflow) does not support the format or has limited storage capacity that mandates a reduced file size,[1] or to convert incompatible or obsolete data to a better-supported or modern format. Transcoding can be performed just while files are being searched, as well as for presentation. For example, Cineon and DPX files have been widely used as a common format for digital cinema, but the data size of a two-hour movie is about 8 terabytes (TB).[1] That large size can increase the cost and difficulty of handling movie files. However, transcoding into a JPEG2000 lossless format has better compression performance than other lossless coding technologies, and in many cases, JPEG2000 can compress images to half-size.[1]
Transcoding is commonly a lossy process, introducing generation loss; however, transcoding can be lossless if the input is losslessly compressed and the output is either losslessly compressed or uncompressed.[1] The process of lossy-to-lossy transcoding introduces varying degrees of generation loss. In other cases, the transcoding of lossy to lossless or uncompressed is technically a lossless conversion because no information is lost, however the process is irreversible and is more suitably known as destructive.

As I'm using lossless transcoding, it is a direct digital-to-digital translation, moving the mpeg2 file from the DVD to the MKV container.

An ISO^w is much the same thing - it is a container that represents an image of a file system.  You seem to be getting hung up on the container, and equating it to lossy encoding.  I'm copying the files I need from one container to another, as the other container is better for my workflow.  I don't care about being able to recreate the DVD in it's entirety; if I decide to archive the films, it won't be on optical media in any case, and for the ones I really care about, I'm keeping the optical media intact.

[tslim]

Guess what, I have never in my life try to run MediaInfo (or other video format identifying program) against a VOB file. I always assume the VOB is a video format (rather than container) and those IFO, BUP files are storing non-video materials like menu and subtitles.

Man... after reading your explanation, I MediaInfo one of my VOB file and see all the various audio tracks, subtitles and of course the Mpeg version 2 video itself.
Thank you.

Btw, what program do you use to do your transcoding?

[f0dder]

Nitpicking: if you simply copy (untouched) streams from one container to another, are you really transcoding? One might be able to argue so from the wiki definition, but personally I'd expect transcoding to involve de- and re-encoding of the actual streams.

Are there any better terms that could be used when you're simply copying streams from one container format to another?

[wraith808]

Guess what, I have never in my life try to run MediaInfo (or other video format identifying program) against a VOB file. I always assume the VOB is a video format (rather than container) and those IFO, BUP files are storing non-video materials like menu and subtitles.

Man... after reading your explanation, I MediaInfo one of my VOB file and see all the various audio tracks, subtitles and of course the Mpeg version 2 video itself.
Thank you.

Btw, what program do you use to do your transcoding?

-tslim (December 04, 2012, 12:10 PM)

It's taken me a long and painful time coming to that, as there's so much conflicting and just plain wrong information out there.  It probably needs some refinement- Oh, look!  Here comes f0dder, I'm sure he'll clean it up...

Nitpicking: if you simply copy (untouched) streams from one container to another, are you really transcoding? One might be able to argue so from the wiki definition, but personally I'd expect transcoding to involve de- and re-encoding of the actual streams.

Are there any better terms that could be used when you're simply copying streams from one container format to another?

-f0dder (December 04, 2012, 01:01 PM)

I don't know.  I know that MakeMKV does do something (has to really) in order to break the encryption.  And I also cleans up the stream; looking at the mess that comes up when you look at a DVD shows some stupid stuff they do in the name of 'copy protection',  So maybe it's still transcoding because of that?

And I use MakeMKV.  I did use PowerISO to make ISOs, until I found that I had problems with less hardware with MKVs than ISOs.

[f0dder]

Oh, look!  Here comes f0dder, I'm sure he'll clean it up...

-wraith808 (December 04, 2012, 02:05 PM)

Hardly, since I don't actually know whether transcoding is the correct term or not - which is why I asked if there's any better terms :-)

IMHO it's useful to distinguish between re-compressing streams and "just" copying them (even if there's an outer layer of DRM crap stripped away) - my above comment wasn't meant as a "silly wraith808 uses wrong terms" but as "are there more appropriate terms?, because just like tslim I was surprised to learn that the term 'transcode' was used when no loss of quality appears". Hope that clears up any eventual misunderstanding :-)

[40hz]

f0dder's observation about "better terms" and wraith's point about conflicting and flat out wrong information is exactly what made me wash my hands of all this video voodoo ages ago.

Does anybody know where you can find a reliable and accurate introduction to PC video and video formats. I don't mind buying a book or two - or paying for an online course if that's what it takes. Ideally something that explains things such as containers, how video and audio data gets encoded, the different types of encoding along with their respective pros and cons...you get the idea: something for an intelligent but total (video tech) newbie who is not afraid of hearing some serious tech talk?

Anybody? 

[wraith808]

Well, there is an article on LifeHacker about using the MakeMKV/HandBrake combo.  That's pretty much the best and simplest approach I've found.  It doesn't help you understand all that much, but for a guide, it's a pretty good place to start.

http://lifehacker.co...r-blu+ray-collection

As in many cases, the Tom's Hardware of old has good information: http://www.tomshardw...uide-part-3,130.html

Of course, it's 13 years old.  I miss the old Tom's Hardware.

There's also an introduction by Mark Pilgrim.  Again, it's old.  But it's so good, that someone else archived it when the original site went down.

http://www.simonwhat...ng-container-formats

And another.  Again old.  But again, good, even though its aimed more at amateur video editors than those that want to archive their collections.

http://www.pcworld.c..._and_containers.html

Those are the most useful and non-biased ones I've stumbled across.  The others are either made by companies trying to sell their software, or just confusing and/or wrong.  At least the ones I've found.

[4wd]

Nitpicking: if you simply copy (untouched) streams from one container to another, are you really transcoding? One might be able to argue so from the wiki definition, but personally I'd expect transcoding to involve de- and re-encoding of the actual streams.

Are there any better terms that could be used when you're simply copying streams from one container format to another?

-f0dder (December 04, 2012, 01:01 PM)

I don't know.  I know that MakeMKV does do something (has to really) in order to break the encryption.  And I also cleans up the stream; looking at the mess that comes up when you look at a DVD shows some stupid stuff they do in the name of 'copy protection',  So maybe it's still transcoding because of that?

-wraith808 (December 04, 2012, 02:05 PM)

Technically:

• Change of container only: Remuxing - Demultiplexing + Multiplexing of the original media streams into a different container format, (eg. AVI->MKV, VOB->MKV).  You may need to decrypt contents to access them but the original encoding/bitrate remains unchanged.
• Change of stream encoding: Transcoding - you are changing the encoding of the original stream format, (eg. MPEG2->MPG4-AVC).  Doesn't necessarily mean a change of container, eg. MPEG4-ASP AVI -> MPEG4-AVC AVI.
• Change of bitrate: Transrating - you are reducing the bitrate of the streams while keeping both original container and encoding, this is what DVD Shrink, CloneDVD, etc do.

Ended up doing simple junction points on the server...now a 250GB HDD apparently holds 4.5TB of files.  Saves a bit of navigation from the media player.

-4wd (December 04, 2012, 12:03 AM)

That's actually a very cool idea!  I'd not thought about using junctions, even though they've been very useful to me in the past.

-wraith808 (December 04, 2012, 08:04 AM)

Not having used them before I have to say they've worked out better than I thought.  By giving two different account permissions, (read and full), the media players just get to read what's available but if I access via computer I get full access without having separate shares all over the place to do file management.

[40hz]

@Wraith- thx for those links in your previous post.  Good info and very helpful. 

[wraith808]

Technically:

• Change of container only: Remuxing - Demultiplexing + Multiplexing of the original media streams into a different container format, (eg. AVI->MKV, VOB->MKV).  You may need to decrypt contents to access them but the original encoding/bitrate remains unchanged.
• Change of stream encoding: Transcoding - you are changing the encoding of the original stream format, (eg. MPEG2->MPG4-AVC).  Doesn't necessarily mean a change of container, eg. MPEG4-ASP AVI -> MPEG4-AVC AVI.
• Change of bitrate: Transrating - you are reducing the bitrate of the streams while keeping both original container and encoding, this is what DVD Shrink, CloneDVD, etc do.

-4wd (December 04, 2012, 06:02 PM)

Thanks for that- I'd not heard the term remuxing before.  Good stuff!  So lossless transcoding still means that you are changing the encoding, is that correct?  You're just not losing any quality over generations?

[4wd]

Technically:

• Change of container only: Remuxing - Demultiplexing + Multiplexing of the original media streams into a different container format, (eg. AVI->MKV, VOB->MKV).  You may need to decrypt contents to access them but the original encoding/bitrate remains unchanged.
• Change of stream encoding: Transcoding - you are changing the encoding of the original stream format, (eg. MPEG2->MPG4-AVC).  Doesn't necessarily mean a change of container, eg. MPEG4-ASP AVI -> MPEG4-AVC AVI.
• Change of bitrate: Transrating - you are reducing the bitrate of the streams while keeping both original container and encoding, this is what DVD Shrink, CloneDVD, etc do.

-4wd (December 04, 2012, 06:02 PM)

Thanks for that- I'd not heard the term remuxing before.  Good stuff!  So lossless transcoding still means that you are changing the encoding, is that correct?  You're just not losing any quality over generations?

-wraith808 (December 04, 2012, 06:23 PM)

Yes - generally you do transcoding if the playback device can't understand a particular format encoding or another format offers better compression, (lossless or otherwise).

What MakeMKV is doing is remuxing, it might not be including some of the original streams from the DVD, (all the menu/warning rubbish), but neither is it changing the original encoded data.

Just for interests sake, I use MKVTools to remux into MKV containers, (for unenrypted formats), and AVIMux-GUI for remuxing to AVI, (it can do MKV also).

Also, it pays to make sure your MKV containers use uncompressed headers to ensure playback on the majority of devices, eg. the WDTV Live HD will not playback MKV files with compressed headers - remuxing to uncompressed headers takes less than a minute.

EDIT: Actually transrating may encompass change of container, (I'm not sure), the majority of its use is to make something big fit into something small, (eg. dual->single layer DVD/BD or bandwidth reduction) - the primary point is that there is just a reduction of bitrate while still in the original format encoding.

[wraith808]

Is MKVTools recommended over MakeMKV?  And My WDTV Live HD does recognize my MKVs... so I guess that means that MakeMKV rips with decompressed headers by default?

[4wd]

Is MKVTools recommended over MakeMKV?  And My WDTV Live HD does recognize my MKVs... so I guess that means that MakeMKV rips with decompressed headers by default?

-wraith808 (December 04, 2012, 07:47 PM)

In the case of you ripping your DVDs, (or BD), no - since it won't do that kind of work, ie. decrypt and concatenate the VOB files.

But in the case where you have other file container formats, (eg. AVI, MP4, MPG, etc), and you just want consistency, (all MKV), or where you have an AVI file with associated subtitle file, you can just drop them on the MKVToolnix frontend, (mmg.exe), and tell it to mux them into one MKV.

FWIW, I rip all my DVDs to MPG4-AVC + whatever audio suits using VidCoder - I'm not overly concerned about the slight loss in quality because I doubt whether I would detect it, (my eyes are getting older), I usually watch them in a window on my computer as I'm doing something else and I still have a boring old Sony Trinitron CRT television that works perfectly.