Advanced Bluray/HDDVD Workflow & Features Discussion

[nightstrm]

I've made a lot of progress in 24 hours, and now have encoded both a Bluray and HDDVD title and successfully inserted chapter markers and soft subtitles. To fix my problem with chapter markers getting progressively further off on the HDDVD I ripped, I used a freeware program named ChapterGrabber (Windows) to convert the subtitles from using 24fps to 23.976fps to calculate the timing. Once completed, the chapters occur in their correct spots throughout the movie.

Thanks for sharing your workflow, nightstrm. I've now tweaked mine so it's, er, exactly the same

Are you finding that most HDDVDs have subtitles coded for 24.000fps or is it the exception? Is there a way to tell apart from trial and error? And is it the same for Blu-ray?

Actually, I've had zero problems with subtitle timing, it was chapter timing (I accidently typed "subtitles" in the quote you used). Chapters from VC-1 sources on HD-DVDs always seem to be a tad off, converting them from 24fps to 23.976fps resolves this issue. I have not found this to be a problem with AVC HD-DVDs, or any Bluray disc as of yet. I believe I know the reason as to why this is happening, but I really need to get to bed so maybe I'll share it another day.

[peterjcat]

Sorry I meant chapters as well, but accidentally typed subtitles, it must be going around

Thanks for the tips.

[ArrMiHardies]

7.) Use mp4tags (Mac) to set hdvd, stik, and cnID on both HD/SD files (for iTunes grouping to behave).
8.) Use mp4chaps (Mac) to add chapter track. As of SVN revision 265 of the mp4v2 library, the output from eac3to/ChapterGrabber can be fed directly into mp4chaps, so no modifications are necessary any longer.
9.) Use MetaX (Mac) to set various remaining tags (could probably combine this with step #4 as mp4tags has had a lot of recent improvements, just need to work out the syntax).

Might I suggest another method.
Swap step 8 and 7, so that the chapter markers are inserted first.
Then replace the new steps 8 and 9 by feeding the file into iDentify http://www.macupdate.com/info.php/id/29847/identify
iDentify will look at your file name, and go to the imdb or tvdb and automatically retrieve -all- the necessary tags for the file as well as a whole slew of other tags like cast, description, rating, etc. It will even visit tagchimp and retrieve names for the chapters if chapter markers are present in the file. No manual entering of data. all done automatically by just dropping the file on the iDentify icon.

This results in more complete tags, with a whole lot less manual work, and the removal of an entire step.

I don't know if the chapter grabber is necessary to insert the time codes into the mp4 file; from the sounds of it, it is required, but if the chapter markers already exist, then Identify can name them. Dunno if chapter grabber does that. Im part way through ripping my first BD right now. Glad I found this thread.

[nightstrm]

7.) Use mp4tags (Mac) to set hdvd, stik, and cnID on both HD/SD files (for iTunes grouping to behave).
8.) Use mp4chaps (Mac) to add chapter track. As of SVN revision 265 of the mp4v2 library, the output from eac3to/ChapterGrabber can be fed directly into mp4chaps, so no modifications are necessary any longer.
9.) Use MetaX (Mac) to set various remaining tags (could probably combine this with step #4 as mp4tags has had a lot of recent improvements, just need to work out the syntax).

Might I suggest another method.
Swap step 8 and 7, so that the chapter markers are inserted first.
Then replace the new steps 8 and 9 by feeding the file into iDentify http://www.macupdate.com/info.php/id/29847/identify
iDentify will look at your file name, and go to the imdb or tvdb and automatically retrieve -all- the necessary tags for the file as well as a whole slew of other tags like cast, description, rating, etc. It will even visit tagchimp and retrieve names for the chapters if chapter markers are present in the file. No manual entering of data. all done automatically by just dropping the file on the iDentify icon.

This results in more complete tags, with a whole lot less manual work, and the removal of an entire step.

Thanks for the input; does iDentify handle the cnID and hdvd atoms? As far as I know, the mp4v2 library is the only one currently out there (that said, iDentify could be using this library). I'm too much of a control freak to have a program automatically download and tag my files; I like to ensure everything is exactly as I want it to appear in iTunes/AppleTV before tagging.

Based on the comments of the recent Subler beta release, I may be able to further combine some of my steps as the author has added more metadata tagging capabilities and the import of chapters from text files. I plan on looking into this more when I get home from work later this afternoon.

[ArrMiHardies]

Im not sure which ones the cnID and hDVD are. iDentify uses AtomicParsley.
I'd reccommend at least giving iDentify a look. After its done tagging, you can always throw it back into metaX and tweak what you need tweaked. But I have found that I have never needed to do that, and I've tagged hundreds of files with it.

[ArrMiHardies]

Could I get a bit more info on how you are using eac3to? I ripped the blu-ray Fifth Element using AnyDVD, fed the folder into eac3to and it extracted all the audio tracks as well as subtitle, chapters, and the video track. All the audio tracks have a .flac extension which tsmuxer doesn't seem to like (except one french one which is .ac3, which tsmuxer seems willing to take but is useless for me). Is there a command line option I am missing or forgetting? I looked at the command line help output by eac3to and none of the options sound like what is needed. is there a way to extract -only- the tracks I need? The command line help is unclear whether all tracks are still extracted if you specify destination names for only some of them.

[peterjcat]

You have to specify all of the tracks you want to extract, and the format in which to extract/convert them. Use "eac3to" to list the titles, "eac3to 1)" to list the tracks in the first title, and eg "eac3to 1) 1: movie.chapters.txt 2: movie.h264 3: movie.ac3" to extract the chapter list and main video and convert the main audio to AC3 (assuming those are the track numbers).

What I've been doing is muxing a TS version of the movie with main video + main audio + AC3 copy of the main audio, with external chapters and subtitles. That results in a file that will play at full AV resolution on a compatible PC or NMT, and can also be read by Handbrake for ATV/iPhone conversion. But if you don't want to keep a full-rez copy (apart from the original disc) you can just do main video+AC3+chapters+subs.

[ArrMiHardies]

So, simply specifying an .ac3 extension for the audio track converts it to ac3?

[nightstrm]

No, in your case I bet you were using the old Fifth Element Bluray that has the English 5.1 PCM soundtrack. That's why you ended up with FLAC files. I haven't dealt with one of these discs, so I can't really offer any help.

[ArrMiHardies]

Actually, I'm using the remastered version. I returned the old one to the store in disgust as the abysmal picture quality. My dvd version had better pq than the original fifth element bd release.

[ArrMiHardies]

I guess my main question is, if there is no AC3 version availble, or there is a combo track, like TrueHD+AC3, how do I go about getting an ac3 track to work with?

peters' post seems to imply that by specifying .ac3 as the extension for a audio track that eac3to will convert to ac3 while demuxing. If this is not the case, then I need to know how to make non-ac3 tracks usable by handbrake.

[nightstrm]

I guess my main question is, if there is no AC3 version availble, or there is a combo track, like TrueHD+AC3, how do I go about getting an ac3 track to work with?

peters' post seems to imply that by specifying .ac3 as the extension for a audio track that eac3to will convert to ac3 while demuxing. If this is not the case, then I need to know how to make non-ac3 tracks usable by handbrake.

Well, if this is a Bluray disc and it has a TrueHD soundtrack, eac3to should be able to extract the AC3 substream that you could use. This step may be unnecessary as I've had good luck simply letting Handbrake extract the AC3 substream.

If you do not have an AC3 source, you need to convert whatever is there to AC3. eac3to can do this as well (I've converted DTS-MA to AC3 @ 640kbps); the best thing I can recommend is to take a look at this Wiki that covers some of the basic eac3to CLI options (it is somewhat dated though).

http://en.wikibooks.org/wiki/Eac3to/Introduction

[ArrMiHardies]

Thanks. I'll take a look at the wiki

[ArrMiHardies]

Gah. stupid Fusion won't let me copy the m2ts file out of the virtual machine onto my mac. smaller files transfer fine, but that large one returns an access error after a period of time. I could copy it to my nas first and then to my mac, but that would take a good while for that much data. Anyone else encounter this issue? I suppose I could always use the windows version of handbrake.... But I'd rather not.

[iiro]

Nice workflow!

Have you ever tried mkvdts2ac3?
I use it on my Mac side to convert DTS to AC3 inside an MKV I can create with mkvtoolnix.
mkvdts2ac3 automates the process like this: DTS -> Individual WAVs per channels -> proper mapping -> AC3.

It's a bit tricky to compile on the mac because it wants aften installed (which in turns uses cmake), but
once that's out of the way it's a really good executable to have!

[ArrMiHardies]

My m2ts file opens and plays in mplayer, but I see no preview images in handbrake. is this normal?

[Northy]

No it isn't, I see preview images (although I am using the SVN so... ), which HB are you using?

[ArrMiHardies]

I'm using the 0.93 public release. The conversion finished last night, and its gorgeous, but It's letterboxed. and near 6 gb. I didn't think the appletv was capable of streaming files that large. last I heard, the cap was 4 gb.

EDIT: Building the svn now. I'll give it a go.

[ArrMiHardies]

can I ask what build you are using? my attempts to compile the svn result in errors when it gets to .contrib

EDIT: This is getting frustrating. all the svn builds I have tried fail to build for me.

[realityking]

Do I get 5 bucks if I guess whats your problem?

Try installing yasm.

(I assume you're on a mac, else disregard everything I said)

[Northy]

Yh you need Yasm, if you don't have it, it will fail.

[peterjcat]

I guess my main question is, if there is no AC3 version availble, or there is a combo track, like TrueHD+AC3, how do I go about getting an ac3 track to work with?

peters' post seems to imply that by specifying .ac3 as the extension for a audio track that eac3to will convert to ac3 while demuxing. If this is not the case, then I need to know how to make non-ac3 tracks usable by handbrake.

Yes, specifying an .ac3 extension for an audio track will tell eac3to to extract the ac3 track from a TrueHD+AC3 track (if for whatever reason Handbrake won't read it directly) OR convert just about anything else to ac3 (640k using libaften). I use it to convert DTS discs. I haven't tried it on LPCM but can't see why it wouldn't work; and all BDs with LPCM should have a lossy audio track (usually AC3) anyway.

[ArrMiHardies]

Yep, i needed YASM. seems very easy to miss that in the wiki. Thanks for the help on that.
Now that the HB svn is compiled, i finally got a good bd rip that will actually play on my ATV. Very nice picture quality. not as good as pure bd, but good enough when im feeling too lazy to get the disc. The HB stable release generates a video that won't play on the appletv, but will play in quicktime. probably due to the file size. the svn is also infinitely faster at loading those large m2ts files. fps is about double what i was getting on the stable release too.

question though,

7.) Use mp4tags (Mac) to set hdvd, stik, and cnID on both HD/SD files (for iTunes grouping to behave).

Im sure these can be written with AtomicParslesy if you use the advanced reverse dns feature. Unfortunately, I do not have any videos with these flags. can someone tell me what data is contained within the hdvd and cnID tags for an hd video? I don't keep both hd and sd versions, so I only need the hd info. Then I can write them manually using atomic parsley and perhaps implement it into iDentify.

[realityking]

You don't need the cnID for just the HD flag.

To my knowledge the hdvd atom can't be written with atomicparsley, it is not a reverse dns style atom. Since atomicparsley isn't actively developed by anyone I wouldn't hold my breath for it to appear anytime soon.

[mac.jedi]

If anyone's interested, I've added the cnID, chapters and hd flagging to an encode batch script. It'll log and select the next cnID from a text file and add them during the HD/SD encodes.

Code: Select all

#!/bin/sh
# HandbrakeCLI-HD.sh

# Finds moves and renames all the eac3to chapter text files from the BD folders to the encode folder.
find /Volumes/SpeedDisk/Batch\ Rip/ -type f -name "*.chapters.txt" | while read i ; do
echo "$i"
fileName=`basename "$i" \ BD.chapters.txt`
cp "$i" /Volumes/SpeedDisk/Batch\ Encode/"$fileName".chapters.txt
cp /Volumes/SpeedDisk/Batch\ Encode/"$fileName".chapters.txt /Volumes/SpeedDisk/Batch\ Encode/"$fileName"\ 1.chapters.txt
done

# Finds all the muxed *BD.m2ts files and processes each file.
find /Volumes/SpeedDisk/Batch\ Rip/ -type f -name "*BD.m2ts" | while read i ; do
echo "$i"
dvdName=`basename "$i" \ BD.m2ts`
skipDuplicates=1
	if [[ ! -e  /Volumes/SpeedDisk/Batch\ Encode/"$dvdName".m4v || skipDuplicates -eq 0 ]];
	then

# Encodes file for 720P AppleTV
	/Applications/HandBrakeCLI -i "$i" -o /Volumes/SpeedDisk/Batch\ Encode/"$dvdName".m4v -e x264 -q 0.56 -a 1,1 -E faac,ac3 -B 160,auto -R 48,Auto -6 dpl2,auto -f mp4 -4 -w 1280 -m -x level=30:cabac=0:ref=3:mixed-refs=1:bframes=6:weightb=1:direct=auto:no-fast-pskip=1:me=umh:subq=7:analyse=all 	& wait

# Adds chapters, cnID and HD flag
# Note: You need to create a text file named cnID.txt with the number you want to start with, ex. 000001000)
	chapsTool="/usr/local/bin/mp4chaps"
	tagsTool="/usr/local/bin/mp4tags"
	$chapsTool -i /Volumes/SpeedDisk/Batch\ Encode/"$dvdName".m4v
	lastcnID=$( tail -1 /Volumes/SpeedDisk/Batch\ Rip/cnID.txt )
	cnIDnum=$( printf "%09d" $( expr $lastcnID + 1 ) )
	echo $cnIDnum >> /Volumes/SpeedDisk/Batch\ Rip/cnID.txt
	$tagsTool -H 1 -I $cnIDnum -i 10 /Volumes/SpeedDisk/Batch\ Encode/"$dvdName".m4v

# Encodes file for Universal Preset
	/Applications/HandBrakeCLI -i "$i" -o /Volumes/SpeedDisk/Batch\ Encode/"$dvdName"\ 1.m4v -e x264 -q 0.56 -a 1 -E faac -B 128 -R 48 -6 dpl2 -f mp4 -X 720 -P -m -x level=30:cabac=0:ref=3:mixed-refs=1:analyse=all:me=umh:no-fast-pskip=1 	& wait
	$chapsTool -i /Volumes/SpeedDisk/Batch\ Encode/"$dvdName"\ 1.m4v
	$tagsTool -I $cnIDnum -i 10 /Volumes/SpeedDisk/Batch\ Encode/"$dvdName"\ 1.m4v

else
	echo "   Output file SKIPPED because it ALREADY EXISTS" & wait
	
fi
done