[Patch (v0.001)] Subtitle support for mp4 files

Archive of historical development discussions
Discussions / Development has moved to GitHub
Forum rules
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.

*******************************
entropic
Novice
Posts: 51
Joined: Wed Apr 11, 2007 8:38 am

[Patch (v0.001)] Subtitle support for mp4 files

Post by entropic »

If you followed my earlier posts in the Tiki Bar - there we go.

First of all, the stuff I implemented is in a -very- early stage. I was hoping that I have some time this weeked to get it ready - but in the end i barely had enough time to add an srt reader. Ahh.. If only I had no life... and no job... :D
Anyway... the options were either wait till i can finally squeeze out enough time to get this done - or just post what i got so far now.

So, where are we now...
libmp4v2 has been extened to fully support tx3g, ftab and nmhd atoms. A new method has been added MP4AddSubtitleTrack. This method adds the tx3g atom and the ftab with hardcoded values... (which is required on the iphone/ipod touch - but not in qt... and both seem to ignore its properties anyway).

I didn't get as far as i wanted this weekend with libhb... but there we go:
- I added the files srt.h/.c (not sure if they should really be in libhb) which contain a really simple srt reader. (Can't handle any malformed srts, style tages, etc...)
- Some new helper methods in muxmp4.c:
MP4SetTrackYTranslation: required for QT but not the iphone/ipod touch (since defTextBox is ignored(??) for sbtl streams)
MP4CreateSubtitleTrack: creates the track and sets all kinds of properties and sets the tracks timescale to ms.
MP4WriteSubtitleSample

Now the really ugly stuff...
In mp4init i call MP4CreateSubtitleTrack, call srt_readfile - with a hardcoded path ( :mrgreen: ) - and call MP4WriteSubtitleSample() for each entry.
:arrow: So... as long as you set the hardcoded file path - and start ripping at the first chapter we got mostly correct subtitles. (There are a couple of syncing issues it seems tho)

Asides from really integrating it in libhb and the GUI, there is still enough stuff missing... like support to add subtitles starting at a custom chapter or a more flexible srt reader, etc....

libhb-subtitles.patch: http://pastebin.ca/963559
mp4v2-subtitles.patch: http://pastebin.ca/963569

I'll continue working on it as soon as possible - but feel free to help...
I really want to get this feature finished asap - but my time is -very- limited right now.... :(

Ben

ps
Awesome... just found a typo in read_time_from_string() in srt.cpp... an houre is 36000ms on my planet it seems. Sorry about that.. Might explain some of my syncing issues....

libhb patch backup

Code: Select all

Index: libhb/muxmp4.c
===================================================================
--- libhb/muxmp4.c	(revision 1364)
+++ libhb/muxmp4.c	(working copy)
@@ -9,6 +9,8 @@
 
 #include "hb.h"
 
+#include "srt.h"
+
 void AddIPodUUID(MP4FileHandle, MP4TrackId);
 
 struct hb_mux_object_s
@@ -27,6 +29,9 @@
     MP4TrackId chapter_track;
     int current_chapter;
     uint64_t chapter_duration;
+	
+	/* Subtitle stuff for muxing */
+    MP4TrackId subtitle_track;
 };
 
 struct hb_mux_data_s
@@ -119,7 +124,114 @@
 }
 
 
+void MP4SetTrackYTranslation(const MP4FileHandle file,
+							const MP4TrackId track,
+							const u_int16_t translation)
+{
+	uint8_t* val;
+	uint8_t nval[38];
+	uint32_t *ptr32 = (uint32_t*) (nval + 2);
+	uint32_t size;
+	MP4GetTrackBytesProperty(file, track, "tkhd.reserved3", &val, &size);
+	
+	memcpy(nval, val, size);
+	
+	const uint32_t ytranslation = translation * 0x10000;
+	
+#ifdef WORDS_BIGENDIAN
+	ptr32[7] = ytranslation;
+#else
+	/* we need to switch the endianness, as the file format expects big endian */
+	ptr32[7] = ((ytranslation & 0x000000FF) << 24) + ((ytranslation & 0x0000FF00) << 8) + ((ytranslation & 0x00FF0000) >> 8) + ((ytranslation & 0xFF000000) >> 24);
+#endif
+	
+	MP4SetTrackBytesProperty(file, track, "tkhd.reserved3", nval, size);	
+}
+
 /**********************************************************************
+ * MP4CreateSubtitleTrack
+ **********************************************************************
+ * Creates a new subtitle track
+ *********************************************************************/
+MP4TrackId MP4CreateSubtitleTrack(const MP4FileHandle file,
+								  const MP4TrackId refTrack,
+								  const uint16_t height,
+								  const uint16_t width,
+								  const uint16_t translation,
+								  const uint16_t language_code)
+{
+	const MP4TrackId subtitleTrack = MP4AddSubtitleTrack(file, refTrack);
+	
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.mdhd.language", language_code);
+	MP4SetTrackFloatProperty(file, subtitleTrack, "tkhd.width", width);
+	MP4SetTrackFloatProperty(file, subtitleTrack, "tkhd.height", height);
+	
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "tkhd.alternate_group", 2);
+	
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.defTextBoxTop", 0);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.defTextBoxLeft", 0);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.defTextBoxBottom", 50);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.defTextBoxRight", 0);
+	
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontID", 1);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontSize", 12);
+	
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.horizontalJustification", 1);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.verticalJustification", 255);
+	
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontColorRed", 255);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontColorGreen", 255);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontColorBlue", 255);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontColorAlpha", 255);
+	
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.bgColorRed", 0);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.bgColorGreen", 0);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.bgColorBlue", 0);
+	MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.bgColorAlpha", 255);
+	
+	MP4SetTrackYTranslation(file, subtitleTrack, translation);
+	
+	MP4SetTrackTimeScale(file, subtitleTrack, 1000); // Set to ms
+
+	return subtitleTrack;
+}
+
+void MP4WriteSubtitleSample(const MP4FileHandle file,
+							const MP4TrackId track,
+							const char* string,
+							const long duration,
+							const long offset)
+{
+	if (duration <= 0)
+		return;
+	
+	if (offset) {
+		// Write the offset...
+		u_int8_t empty[2] = {0,0};
+		MP4WriteSample(file,
+					   track,
+					   empty,
+					   2,
+					   duration,
+					   0, true);
+	}
+
+	// .. and write the actual sample
+	const size_t stringLength = strlen(string);
+	u_int8_t buffer[1024];
+	memcpy(buffer+2, string, strlen(string)); // strlen > 1024 -> booom
+	buffer[0] = (stringLength >> 8) & 0xff;
+	buffer[1] = stringLength & 0xff;
+	
+	MP4WriteSample(file,
+				   track,
+				   buffer,
+				   stringLength + 2,
+				   duration,
+				   0, true);	
+}
+
+/**********************************************************************
  * MP4Init
  **********************************************************************
  * Allocates hb_mux_data_t structures, create file and write headers
@@ -335,7 +447,26 @@
         m->chapter_duration = 0;
         m->current_chapter = job->chapter_start;
 	}
-
+	
+	const uint16_t subtitleHeight = 60;
+	m->subtitle_track = MP4CreateSubtitleTrack(m->file,
+											   mux_data->track,
+											   subtitleHeight,
+											   m->job->width,
+											   m->job->height - subtitleHeight,
+											   language_code);
+	
+	
+	char filename[] = "/Users/ben/Developer/SrtParser/test.srt";
+	struct srt_entry_s* subtitles = srt_readfile(filename);
+	while (subtitles) {
+		MP4WriteSubtitleSample(m->file, m->subtitle_track, subtitles->text, subtitles->offset, subtitles->duration);			
+		subtitles = subtitles->next;
+	}
+	
+	srt_free_entries(subtitles);
+	
+	
     /* Add encoded-by metadata listing version and build date */
     char *tool_string;
     tool_string = (char *)malloc(80);
@@ -387,7 +518,8 @@
                 hb_error("Failed to write to output file, disk full?");
                 *job->die = 1;
             }
-            free(sample);
+			
+			free(sample);
             m->current_chapter++;
             m->chapter_duration += duration;
         }
@@ -465,9 +597,10 @@
             hb_error("Failed to write to output file, disk full?");
             *job->die = 1;
         }
-        free(sample);
+		
+		free(sample);
     }
-
+	
     if (job->areBframes)
     {
            // Insert track edit to get A/V back in sync.  The edit amount is
@@ -507,4 +640,3 @@
     m->job       = job;
     return m;
 }
-
Index: libhb/srt.c
===================================================================
--- libhb/srt.c	(revision 0)
+++ libhb/srt.c	(revision 0)
@@ -0,0 +1,134 @@
+#include "srt.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+
+struct start_and_end {
+	unsigned long start, end;
+};
+
+static struct start_and_end read_time_from_string(const char* timeString) {
+	// for ex. 00:00:15,248 --> 00:00:16,545
+	
+	long houres1, minutes1, seconds1, milliseconds1,
+	houres2, minutes2, seconds2, milliseconds2;
+	
+	sscanf(timeString, "%d:%d:%d,%d --> %d:%d:%d,%d\n",	&houres1, &minutes1, &seconds1, &milliseconds1,
+		   &houres2, &minutes2, &seconds2, &milliseconds2);
+	
+	struct start_and_end result = {milliseconds1 + seconds1*1000 + minutes1*60000 + houres1*36000,
+	milliseconds2 + seconds2*1000 + minutes2*60000 + houres2*36000};
+	return result;
+}
+
+enum
+{
+	k_state_inEntry,
+	k_state_potential_new_entry,
+	k_state_timecode,
+};
+
+struct srt_entry_s* srt_readfile(const char* filename)
+{
+	FILE *file = fopen ( filename, "r" );
+	
+	if (file == NULL)
+		return NULL;
+	
+	char line_buffer[1024];
+	unsigned long	current_time = 0,
+	number_of_entries = 0,
+	current_state = k_state_potential_new_entry;
+	
+	struct srt_entry_s	*first_entry = malloc(sizeof(struct srt_entry_s)),
+	*current_entry = first_entry;
+	memset(first_entry, 0, sizeof(struct srt_entry_s));
+	
+	while (fgets(line_buffer,sizeof(line_buffer),file)) {
+		switch (current_state)
+		{
+			case k_state_timecode:
+			{
+				struct start_and_end timing = read_time_from_string(line_buffer);			
+				
+				current_entry->duration = timing.end - timing.start;
+				current_entry->offset = timing.start - current_time;
+				
+				current_time = timing.end;
+				
+				current_state = k_state_inEntry;
+				continue;				
+			}
+				
+			case k_state_inEntry:
+			{
+				// If the current line is empty, we assume this is the
+				//	seperation betwene two entries. In case we are wrong,
+				//	the mistake is corrected in the next state.
+				if (strcmp(line_buffer, "\n") == 0 || strcmp(line_buffer, "\r\n") == 0) {					
+					current_state = k_state_potential_new_entry;
+					continue;
+				}
+				
+				// Append the current line to the entry's text buffer;
+				strncat(current_entry->text, line_buffer, 1024);
+				
+				break;				
+			}
+				
+			case k_state_potential_new_entry:
+			{
+				char endpoint[] = "\0";
+				const unsigned long potential_entry_number = strtol(line_buffer, (char**)&endpoint, 10);
+				
+				// Is this really new next entry begin?
+				if (potential_entry_number == number_of_entries + 1) {
+					// We actuall found the next entry - or a really rare error condition
+					
+					// The first entry was preallocated - all others are alloced here
+					//	plus we overwrite zero-duration entries
+					if (number_of_entries != 0) {
+						current_entry->next = malloc(sizeof(struct srt_entry_s));						
+						current_entry = current_entry->next;
+					}
+					
+					memset(current_entry, 0, sizeof(struct srt_entry_s));
+					
+					++number_of_entries;
+					current_state = k_state_timecode;
+					continue;
+				} else {
+					// Well.. looks like we are in the wrong mode.. lets add the
+					//	newline we misinterpreted...
+					strncat(current_entry->text, "\n", 1024);
+					current_state = k_state_inEntry;
+				}
+				
+				break;
+			}
+		}
+	}
+	
+	fclose ( file );
+	
+	// Since the number of entries is incremented when we find the next entry,
+	//	we missed the lastone
+	if (strlen(current_entry->text)) {
+		++number_of_entries;
+	}
+	
+	// No entries found - free the preallocaed firstone
+	if (number_of_entries == 0) {
+		free(first_entry);
+		return NULL;
+	}
+	
+	return first_entry;
+}
+
+void srt_free_entries(struct srt_entry_s* entries)
+{
+	if (entries && entries->next)
+		srt_free_entries(entries->next);
+	free(entries);
+}
\ No newline at end of file
Index: libhb/srt.h
===================================================================
--- libhb/srt.h	(revision 0)
+++ libhb/srt.h	(revision 0)
@@ -0,0 +1,11 @@
+#pragma once
+
+struct srt_entry_s {
+	long offset, duration;
+	char text[1024];
+	
+	struct srt_entry_s* next;
+};
+
+struct srt_entry_s* srt_readfile(const char* filename);
+void srt_free_entries(struct srt_entry_s* entries);
\ No newline at end of file
mp4v2 backup

Code: Select all

Index: mp4file.h
===================================================================
--- mp4file.h	(revision 4)
+++ mp4file.h	(working copy)
@@ -303,6 +303,7 @@
 	MP4TrackId AddHintTrack(MP4TrackId refTrackId);
 	MP4TrackId AddTextTrack(MP4TrackId refTrackId);
 	MP4TrackId AddChapterTextTrack(MP4TrackId refTrackId);
+	MP4TrackId AddSubtitleTrack(MP4TrackId refTrackId);
 
 	MP4TrackId AddPixelAspectRatio(MP4TrackId trackId, u_int32_t hSpacing, u_int32_t vSpacing);
 
Index: atom_nmhd.cpp
===================================================================
--- atom_nmhd.cpp	(revision 0)
+++ atom_nmhd.cpp	(revision 0)
@@ -0,0 +1,25 @@
+/*
+ * The contents of this file are subject to the Mozilla Public
+ * License Version 1.1 (the "License"); you may not use this file
+ * except in compliance with the License. You may obtain a copy of
+ * the License at http://www.mozilla.org/MPL/
+ * 
+ * Software distributed under the License is distributed on an "AS
+ * IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or
+ * implied. See the License for the specific language governing
+ * rights and limitations under the License.
+ * 
+ * The Original Code is MPEG4IP.
+ * 
+ * The Initial Developer of the Original Code is Cisco Systems Inc.
+ * Portions created by Cisco Systems Inc. are
+ * Copyright (C) Cisco Systems Inc. 2001.  All Rights Reserved.
+ */
+
+#include "mp4common.h"
+
+MP4NmhdAtom::MP4NmhdAtom()
+	: MP4Atom("nmhd") 
+{
+	AddVersionAndFlags();
+}
\ No newline at end of file
Index: mp4atom.cpp
===================================================================
--- mp4atom.cpp	(revision 4)
+++ mp4atom.cpp	(working copy)
@@ -120,6 +120,8 @@
 	pAtom = new MP4FreeAtom();
       } else if (ATOMID(type) == ATOMID("ftyp")) {
 	pAtom = new MP4FtypAtom();
+      } else if (ATOMID(type) == ATOMID("ftab")) {
+		  pAtom = new MP4FtabAtom();
       }
       break;
     case 'g':
@@ -176,6 +178,8 @@
     case 'n':
       if (ATOMID(type) == ATOMID("name")) { // iTunes
 	pAtom = new MP4NameAtom();
+      } else if (ATOMID(type) == ATOMID("nmhd")) {
+		  pAtom = new MP4NmhdAtom();
       }
       break;
     case 'r':
@@ -216,6 +220,8 @@
     case 't':
       if (ATOMID(type) == ATOMID("text")) {
 	pAtom = new MP4TextAtom();
+      } else if (ATOMID(type) == ATOMID("tx3g")) {
+	pAtom = new MP4Tx3gAtom();
       } else if (ATOMID(type) == ATOMID("tkhd")) {
 	pAtom = new MP4TkhdAtom();
       } else if (ATOMID(type) == ATOMID("tfhd")) {
Index: mp4.cpp
===================================================================
--- mp4.cpp	(revision 4)
+++ mp4.cpp	(working copy)
@@ -973,6 +973,21 @@
 	return MP4_INVALID_TRACK_ID;
 }
 
+extern "C" MP4TrackId MP4AddSubtitleTrack(
+	MP4FileHandle hFile, MP4TrackId refTrackId)
+{
+	if (MP4_IS_VALID_FILE_HANDLE(hFile)) {
+		try {
+			return ((MP4File*)hFile)->AddSubtitleTrack(refTrackId);
+		}
+		catch (MP4Error* e) {
+			PRINT_ERROR(e);
+			delete e;
+		}
+	}
+	return MP4_INVALID_TRACK_ID;
+}
+
 extern "C" MP4TrackId MP4AddChapterTextTrack(
 	MP4FileHandle hFile, MP4TrackId refTrackId)
 {
Index: atom_tx3g.cpp
===================================================================
--- atom_tx3g.cpp	(revision 0)
+++ atom_tx3g.cpp	(revision 0)
@@ -0,0 +1,55 @@
+/*
+ * The contents of this file are subject to the Mozilla Public
+ * License Version 1.1 (the "License"); you may not use this file
+ * except in compliance with the License. You may obtain a copy of
+ * the License at http://www.mozilla.org/MPL/
+ * 
+ * Software distributed under the License is distributed on an "AS
+ * IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or
+ * implied. See the License for the specific language governing
+ * rights and limitations under the License.
+ * 
+ * The Original Code is MPEG4IP.
+ * 
+ * The Initial Developer of the Original Code is Cisco Systems Inc.
+ * Portions created by Cisco Systems Inc. are
+ * Copyright (C) Cisco Systems Inc. 2001.  All Rights Reserved.
+ */
+
+#include "mp4common.h"
+
+MP4Tx3gAtom::MP4Tx3gAtom() 
+	: MP4Atom("tx3g") 
+{
+	AddReserved("reserved1", 4); /* 0 */
+	AddReserved("reserved2", 2); /* 1 */
+	
+	AddProperty(new MP4Integer16Property("dataReferenceIndex"));/* 2 */
+	
+	AddProperty(new MP4Integer32Property("displayFlags")); /* 3 */
+	AddProperty(new MP4Integer8Property("horizontalJustification")); /* 4 */
+	AddProperty(new MP4Integer8Property("verticalJustification")); /* 5 */
+
+	AddProperty(new MP4Integer8Property("bgColorRed")); /* 6 */
+	AddProperty(new MP4Integer8Property("bgColorGreen")); /* 7 */
+	AddProperty(new MP4Integer8Property("bgColorBlue")); /* 8 */
+	AddProperty(new MP4Integer8Property("bgColorAlpha")); /* 9 */
+
+	AddProperty(new MP4Integer16Property("defTextBoxTop")); /* 10 */
+	AddProperty(new MP4Integer16Property("defTextBoxLeft")); /* 11 */
+	AddProperty(new MP4Integer16Property("defTextBoxBottom")); /* 12 */
+	AddProperty(new MP4Integer16Property("defTextBoxRight")); /* 13 */
+	
+	AddProperty(new MP4Integer16Property("startChar")); /* 14 */
+	AddProperty(new MP4Integer16Property("endChar")); /* 15 */
+	AddProperty(new MP4Integer16Property("fontID")); /* 16 */
+	AddProperty(new MP4Integer8Property("fontFace")); /* 17 */
+	AddProperty(new MP4Integer8Property("fontSize")); /* 18 */
+	
+	AddProperty(new MP4Integer8Property("fontColorRed")); /* 19 */
+	AddProperty(new MP4Integer8Property("fontColorGreen")); /* 20 */
+	AddProperty(new MP4Integer8Property("fontColorBlue")); /* 21 */
+	AddProperty(new MP4Integer8Property("fontColorAlpha")); /* 22 */
+	
+	ExpectChildAtom("ftab", Optional, Many);
+}
Index: atoms.h
===================================================================
--- atoms.h	(revision 4)
+++ atoms.h	(working copy)
@@ -353,6 +353,16 @@
 	void GenerateGmhdType();
 };
 
+class MP4Tx3gAtom : public MP4Atom {
+public:
+	MP4Tx3gAtom();
+};
+
+class MP4FtabAtom : public MP4Atom {
+public:
+	MP4FtabAtom();
+};
+
 class MP4TfhdAtom : public MP4Atom {
 public:
 	MP4TfhdAtom();
@@ -406,4 +416,9 @@
     void Generate();
 };
 
+class MP4NmhdAtom : public MP4Atom {
+public:
+	MP4NmhdAtom();
+};
+
 #endif /* __MP4_ATOMS_INCLUDED__ */
Index: atom_ftab.cpp
===================================================================
--- atom_ftab.cpp	(revision 0)
+++ atom_ftab.cpp	(revision 0)
@@ -0,0 +1,32 @@
+/*
+ * The contents of this file are subject to the Mozilla Public
+ * License Version 1.1 (the "License"); you may not use this file
+ * except in compliance with the License. You may obtain a copy of
+ * the License at http://www.mozilla.org/MPL/
+ * 
+ * Software distributed under the License is distributed on an "AS
+ * IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or
+ * implied. See the License for the specific language governing
+ * rights and limitations under the License.
+ * 
+ * The Original Code is MPEG4IP.
+ * 
+ * The Initial Developer of the Original Code is Cisco Systems Inc.
+ * Portions created by Cisco Systems Inc. are
+ * Copyright (C) Cisco Systems Inc. 2001.  All Rights Reserved.
+ */
+
+#include "mp4common.h"
+
+MP4FtabAtom::MP4FtabAtom()
+	: MP4Atom("ftab") 
+{
+	MP4Integer16Property* pCount = new MP4Integer16Property("entryCount"); /* 0 */
+	AddProperty(pCount);
+
+	MP4TableProperty* pTable = new MP4TableProperty("fontEntries", pCount);  /* 1 */
+	AddProperty(pTable);
+
+	pTable->AddProperty(new MP4Integer16Property("fontID"));  /* 0 */
+	pTable->AddProperty(new MP4StringProperty("name", true));  /* 1 */	
+}
Index: mp4file.cpp
===================================================================
--- mp4file.cpp	(revision 4)
+++ mp4file.cpp	(working copy)
@@ -1948,6 +1948,49 @@
 	return trackId;
 }
 
+MP4TrackId MP4File::AddSubtitleTrack(MP4TrackId refTrackId)
+{
+	// validate reference track id
+	FindTrackIndex(refTrackId);
+	
+	MP4TrackId trackId = 
+	AddTrack(MP4_TEXT_TRACK_TYPE, GetTrackTimeScale(refTrackId));
+	
+	InsertChildAtom(MakeTrackName(trackId, "mdia.minf"), "nmhd", 0);
+	
+	AddChildAtom(MakeTrackName(trackId, "mdia.minf.stbl.stsd"), "tx3g");
+	
+	MP4StringProperty* pHandlerTypeProperty;
+	FindStringProperty(MakeTrackName(trackId, "mdia.hdlr.handlerType"),
+					   (MP4Property**)&pHandlerTypeProperty);
+	pHandlerTypeProperty->SetValue("sbtl");
+	
+	// Hardcoded crap... add the ftab atom and add one font entry
+	MP4Atom* pFtabAtom = AddChildAtom(MakeTrackName(trackId, "mdia.minf.stbl.stsd.tx3g"), "ftab");
+	
+	((MP4Integer16Property*)pFtabAtom->GetProperty(0))->IncrementValue();
+	
+	MP4Integer16Property* pfontID = (MP4Integer16Property*)((MP4TableProperty*)pFtabAtom->GetProperty(1))->GetProperty(0);
+	pfontID->AddValue(1);
+	
+	MP4StringProperty* pName = (MP4StringProperty*)((MP4TableProperty*)pFtabAtom->GetProperty(1))->GetProperty(1);
+	pName->AddValue("Arial");
+	
+	
+	// stsd is a unique beast in that it has a count of the number 
+	// of child atoms that needs to be incremented after we add the text atom
+	MP4Integer32Property* pStsdCountProperty;
+	FindIntegerProperty(
+		MakeTrackName(trackId, "mdia.minf.stbl.stsd.entryCount"),
+		(MP4Property**)&pStsdCountProperty);
+	pStsdCountProperty->IncrementValue();
+
+	/* add the magic "text" atom to the generic media header */
+//	AddChildAtom(MakeTrackName(trackId, "mdia.minf.gmhd"), "text");
+	
+	return trackId;
+}
+
 MP4TrackId MP4File::AddChapterTextTrack(MP4TrackId refTrackId)
 {
 	// validate reference track id
Index: mp4.h
===================================================================
--- mp4.h	(revision 4)
+++ mp4.h	(working copy)
@@ -553,6 +553,10 @@
 	MP4FileHandle hFile, 
 	MP4TrackId refTrackId);
 
+MP4TrackId MP4AddSubtitleTrack(
+	MP4FileHandle hFile,
+	MP4TrackId refTrackId);
+	
 MP4TrackId MP4AddPixelAspectRatio(
 	MP4FileHandle hFile, 
 	MP4TrackId refTrackId,
Last edited by s55 on Wed Jun 25, 2008 10:13 pm, edited 1 time in total.
Reason: Added pastebin code into post incase the pastebin disappears.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by jbrjake »

Cool =)

The libmp4v2 part looks good to go, imo.

At first glance, I didn't like the idea of adding two more files to libhb, but after considering it for a bit, I suppose it's more rational for this than for the iPod atom or the CSV parser.

Update the patches to fix that typo you found, and I'll look at hooking up the CLI, making it use dynamic paths, and throwing in some conditionals so it doesn't run unless the user turns it on.

Thanks, this is a long-requested feature. Will be nice to finally encode anime without having to make a dub/sub decision.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by jbrjake »

Ok, decided to get started without the right timecode calculations.

- Added the new files to the Xcode project so it can compile on Macs
- Finished the mpeg4ip patch (you hadn't included the necessary changes for the Makefile.in and Makefile.am) and fixed the filepaths (HandBrake needs comparisons done at the root level, not inside libmp4v2)
- Integrated the mpeg4ip patch into the build script
- Set up a job->srt boolean to turn it on and a job->srt_file string to hold the filepath, implemented them in muxmp4.c
- Wired up the CLI with an --srt option that takes takes the filepath as its required argument. Short name: 3.

Here's a complete diff:
http://handbrake.pastebin.ca/963911

Code: Select all

Index: test/test.c
===================================================================
--- test/test.c	(revision 1364)
+++ test/test.c	(working copy)
@@ -73,6 +73,7 @@
 static int    vfr           = 0;
 static int    mp4_optimize  = 0;
 static int    ipod_atom     = 0;
+static char * srt      = NULL;
 
 /* Exit cleanly on Ctrl-C */
 static volatile int die = 0;
@@ -235,6 +236,7 @@
 	if( x264opts ) free (x264opts );
 	if( x264opts2 ) free (x264opts2 );
     if (preset_name) free (preset_name);
+    if (srt) free (srt);
 
     fprintf( stderr, "HandBrake has exited.\n" );
 
@@ -901,6 +903,12 @@
             {
                 job->native_language = strdup( native_language );
             }
+            
+            if( srt != NULL && *srt != '\0' )
+            {
+                job->srt = 1;
+                job->srt_file = srt;
+            }
 
             if( job->mux )
             {
@@ -1184,6 +1192,7 @@
     "    -N, --native-language   Select subtitles with this language if it does not\n"
     "          <string>          match the Audio language. Provide the language's\n"
     "                            iso639-2 code (fre, eng, spa, dut, et cetera)\n"
+    "    -3  --srt <string>      Indicate a filepath for loading text subtitles\n"
 	"    -m, --markers           Add chapter markers (mp4 output format only)\n"
 	"\n"
 
@@ -1340,6 +1349,7 @@
             { "subtitle-scan", no_argument,     NULL,    'U' },
             { "subtitle-forced", no_argument,   NULL,    'F' },
             { "native-language", required_argument, NULL,'N' },
+            { "srt",         required_argument, NULL,    '3'},
 
             { "encoder",     required_argument, NULL,    'e' },
             { "aencoder",    required_argument, NULL,    'E' },
@@ -1377,7 +1387,7 @@
         int c;
 
 		c = getopt_long( argc, argv,
-						 "hvuC:f:4i:Io:t:Lc:m::a:6:s:UFN:e:E:2dD:789gpOP::w:l:n:b:q:S:B:r:R:Qx:TY:X:VZ:z",
+						 "hvuC:f:3:4i:Io:t:Lc:m::a:6:s:UFN:e:E:2dD:789gpOP::w:l:n:b:q:S:B:r:R:Qx:TY:X:VZ:z",
                          long_options, &option_index );
         if( c < 0 )
         {
@@ -1500,6 +1510,9 @@
             case 'N':
                 native_language = strdup( optarg );
                 break;
+            case '3':
+                srt = strdup( optarg );
+                break;
             case '2':
                 twoPass = 1;
                 break;
Index: libhb/muxmp4.c
===================================================================
--- libhb/muxmp4.c	(revision 1364)
+++ libhb/muxmp4.c	(working copy)
@@ -9,6 +9,8 @@
 
 #include "hb.h"
 
+#include "srt.h"
+
 void AddIPodUUID(MP4FileHandle, MP4TrackId);
 
 struct hb_mux_object_s
@@ -27,6 +29,9 @@
     MP4TrackId chapter_track;
     int current_chapter;
     uint64_t chapter_duration;
+    
+    /* Subtitle stuff for muxing */
+    MP4TrackId subtitle_track;
 };
 
 struct hb_mux_data_s
@@ -119,7 +124,114 @@
 }
 
 
+void MP4SetTrackYTranslation(const MP4FileHandle file,
+                            const MP4TrackId track,
+                            const u_int16_t translation)
+{
+    uint8_t* val;
+    uint8_t nval[38];
+    uint32_t *ptr32 = (uint32_t*) (nval + 2);
+    uint32_t size;
+    MP4GetTrackBytesProperty(file, track, "tkhd.reserved3", &val, &size);
+    
+    memcpy(nval, val, size);
+    
+    const uint32_t ytranslation = translation * 0x10000;
+    
+#ifdef WORDS_BIGENDIAN
+    ptr32[7] = ytranslation;
+#else
+    /* we need to switch the endianness, as the file format expects big endian */
+    ptr32[7] = ((ytranslation & 0x000000FF) << 24) + ((ytranslation & 0x0000FF00) << 8) + ((ytranslation & 0x00FF0000) >> 8) + ((ytranslation & 0xFF000000) >> 24);
+#endif
+    
+    MP4SetTrackBytesProperty(file, track, "tkhd.reserved3", nval, size);    
+}
+
 /**********************************************************************
+ * MP4CreateSubtitleTrack
+ **********************************************************************
+ * Creates a new subtitle track
+ *********************************************************************/
+MP4TrackId MP4CreateSubtitleTrack(const MP4FileHandle file,
+                                  const MP4TrackId refTrack,
+                                  const uint16_t height,
+                                  const uint16_t width,
+                                  const uint16_t translation,
+                                  const uint16_t language_code)
+{
+    const MP4TrackId subtitleTrack = MP4AddSubtitleTrack(file, refTrack);
+    
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.mdhd.language", language_code);
+    MP4SetTrackFloatProperty(file, subtitleTrack, "tkhd.width", width);
+    MP4SetTrackFloatProperty(file, subtitleTrack, "tkhd.height", height);
+    
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "tkhd.alternate_group", 2);
+    
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.defTextBoxTop", 0);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.defTextBoxLeft", 0);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.defTextBoxBottom", 50);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.defTextBoxRight", 0);
+    
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontID", 1);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontSize", 12);
+    
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.horizontalJustification", 1);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.verticalJustification", 255);
+    
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontColorRed", 255);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontColorGreen", 255);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontColorBlue", 255);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.fontColorAlpha", 255);
+    
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.bgColorRed", 0);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.bgColorGreen", 0);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.bgColorBlue", 0);
+    MP4SetTrackIntegerProperty(file, subtitleTrack, "mdia.minf.stbl.stsd.tx3g.bgColorAlpha", 255);
+    
+    MP4SetTrackYTranslation(file, subtitleTrack, translation);
+    
+    MP4SetTrackTimeScale(file, subtitleTrack, 1000); // Set to ms
+
+    return subtitleTrack;
+}
+
+void MP4WriteSubtitleSample(const MP4FileHandle file,
+                            const MP4TrackId track,
+                            const char* string,
+                            const long duration,
+                            const long offset)
+{
+    if (duration <= 0)
+        return;
+    
+    if (offset) {
+        // Write the offset...
+        u_int8_t empty[2] = {0,0};
+        MP4WriteSample(file,
+                       track,
+                       empty,
+                       2,
+                       duration,
+                       0, true);
+    }
+
+    // .. and write the actual sample
+    const size_t stringLength = strlen(string);
+    u_int8_t buffer[1024];
+    memcpy(buffer+2, string, strlen(string)); // strlen > 1024 -> booom
+    buffer[0] = (stringLength >> 8) & 0xff;
+    buffer[1] = stringLength & 0xff;
+    
+    MP4WriteSample(file,
+                   track,
+                   buffer,
+                   stringLength + 2,
+                   duration,
+                   0, true);    
+}
+
+/**********************************************************************
  * MP4Init
  **********************************************************************
  * Allocates hb_mux_data_t structures, create file and write headers
@@ -183,12 +295,12 @@
             return 0;
         }
 
-		mux_data->track = MP4AddH264VideoTrack( m->file, job->arate,
-		        MP4_INVALID_DURATION, job->width, job->height,
-		        job->config.h264.sps[1], /* AVCProfileIndication */
-		        job->config.h264.sps[2], /* profile_compat */
-		        job->config.h264.sps[3], /* AVCLevelIndication */
-		        3 );      /* 4 bytes length before each NAL unit */
+        mux_data->track = MP4AddH264VideoTrack( m->file, job->arate,
+                MP4_INVALID_DURATION, job->width, job->height,
+                job->config.h264.sps[1], /* AVCProfileIndication */
+                job->config.h264.sps[2], /* profile_compat */
+                job->config.h264.sps[3], /* AVCLevelIndication */
+                3 );      /* 4 bytes length before each NAL unit */
 
 
         MP4AddH264SequenceParameterSet( m->file, mux_data->track,
@@ -196,11 +308,11 @@
         MP4AddH264PictureParameterSet( m->file, mux_data->track,
                 job->config.h264.pps, job->config.h264.pps_length );
 
-		if( job->h264_level == 30 || job->ipod_atom)
-		{
-			hb_log("About to add iPod atom");
-			AddIPodUUID(m->file, mux_data->track);
-		}
+        if( job->h264_level == 30 || job->ipod_atom)
+        {
+            hb_log("About to add iPod atom");
+            AddIPodUUID(m->file, mux_data->track);
+        }
 
     }
     else /* FFmpeg or XviD */
@@ -246,18 +358,18 @@
         MP4SetTrackFloatProperty(m->file, mux_data->track, "tkhd.width", job->width * (width / height));
     }
 
-	/* [Censored] will be used to reference the first audio track when we add a chapter track */
-	MP4TrackId [Censored] = 0;
+    /* [Censored] will be used to reference the first audio track when we add a chapter track */
+    MP4TrackId [Censored] = 0;
 
-	/* add the audio tracks */
+    /* add the audio tracks */
     for( i = 0; i < hb_list_count( title->list_audio ); i++ )
     {
-    	static u_int8_t reserved2[16] = {
-    		0x00, 0x00, 0x00, 0x00,
-    		0x00, 0x00, 0x00, 0x00,
-    		0x00, 0x02, 0x00, 0x10,
-    		0x00, 0x00, 0x00, 0x00,
-	    };
+        static u_int8_t reserved2[16] = {
+            0x00, 0x00, 0x00, 0x00,
+            0x00, 0x00, 0x00, 0x00,
+            0x00, 0x02, 0x00, 0x10,
+            0x00, 0x00, 0x00, 0x00,
+        };
 
         audio = hb_list_item( title->list_audio, i );
         mux_data = malloc( sizeof( hb_mux_data_t ) );
@@ -325,17 +437,39 @@
 
     }
 
-	if (job->chapter_markers)
+    if (job->chapter_markers)
     {
-		/* add a text track for the chapters */
-		MP4TrackId textTrack;
-		textTrack = MP4AddChapterTextTrack(m->file, [Censored]);
+        /* add a text track for the chapters */
+        MP4TrackId textTrack;
+        textTrack = MP4AddChapterTextTrack(m->file, [Censored]);
 
         m->chapter_track = textTrack;
         m->chapter_duration = 0;
         m->current_chapter = job->chapter_start;
-	}
-
+    }
+    
+    if (job->srt)
+    {
+        const uint16_t subtitleHeight = 60;
+        m->subtitle_track = MP4CreateSubtitleTrack(m->file,
+                                                   mux_data->track,
+                                                   subtitleHeight,
+                                                   m->job->width,
+                                                   m->job->height - subtitleHeight,
+                                                   language_code);
+        
+        
+//        char filename[] = job->srt_file;
+        
+        struct srt_entry_s* subtitles = srt_readfile(job->srt_file);
+        while (subtitles) {
+            MP4WriteSubtitleSample(m->file, m->subtitle_track, subtitles->text, subtitles->offset, subtitles->duration);            
+            subtitles = subtitles->next;
+        }
+        
+        srt_free_entries(subtitles);
+    }
+    
     /* Add encoded-by metadata listing version and build date */
     char *tool_string;
     tool_string = (char *)malloc(80);
@@ -387,6 +521,7 @@
                 hb_error("Failed to write to output file, disk full?");
                 *job->die = 1;
             }
+            
             free(sample);
             m->current_chapter++;
             m->chapter_duration += duration;
@@ -465,9 +600,10 @@
             hb_error("Failed to write to output file, disk full?");
             *job->die = 1;
         }
+        
         free(sample);
     }
-
+    
     if (job->areBframes)
     {
            // Insert track edit to get A/V back in sync.  The edit amount is
@@ -507,4 +643,3 @@
     m->job       = job;
     return m;
 }
-
Index: libhb/work.c
===================================================================
--- libhb/work.c	(revision 1364)
+++ libhb/work.c	(working copy)
@@ -337,6 +337,11 @@
         }
     }
 
+    if( job->srt )
+    {
+        hb_log(" + SRT subtitles imported: %s", job->srt_file);
+    }
+    
     if( job->audio_mixdowns[0] == HB_AMIXDOWN_AC3 || job->audio_mixdowns[1] == HB_AMIXDOWN_AC3 )
     {
         /* Hard set correct sample rate for AC3 when libhb
Index: libhb/common.h
===================================================================
--- libhb/common.h	(revision 1364)
+++ libhb/common.h	(working copy)
@@ -252,6 +252,9 @@
     int             subtitle;
     int 			subtitleSmartAdjust;
 
+    int             srt;
+    char *          srt_file;
+    
     /* Muxer settings
          mux:  output file format
          file: file path */
Index: macosx/HandBrake.plist
===================================================================
--- macosx/HandBrake.plist	(revision 1364)
+++ macosx/HandBrake.plist	(working copy)
@@ -9,7 +9,7 @@
 	<key>CFBundleExecutable</key>
 	<string>HandBrake</string>
 	<key>CFBundleGetInfoString</key>
-	<string>0.9.2</string>
+	<string>0.9.3svn1364M</string>
 	<key>CFBundleIconFile</key>
 	<string>HandBrake</string>
 	<key>CFBundleIdentifier</key>
@@ -21,11 +21,11 @@
 	<key>CFBundlePackageType</key>
 	<string>APPL</string>
 	<key>CFBundleShortVersionString</key>
-	<string>0.9.2</string>
+	<string>0.9.3svn1364M</string>
 	<key>CFBundleSignature</key>
 	<string>HB##</string>
 	<key>CFBundleVersion</key>
-	<string>2008021900</string>
+	<string>2008033001</string>
 	<key>NSHumanReadableCopyright</key>
 	<string>HandBrake Devs</string>
 	<key>NSMainNibFile</key>
Index: macosx/HandBrake.xcodeproj/project.pbxproj
===================================================================
--- macosx/HandBrake.xcodeproj/project.pbxproj	(revision 1364)
+++ macosx/HandBrake.xcodeproj/project.pbxproj	(working copy)
@@ -90,6 +90,10 @@
 		593034EC0BBA39A100172349 /* ChapterTitles.m in Sources */ = {isa = PBXBuildFile; fileRef = 593034EA0BBA39A100172349 /* ChapterTitles.m */; };
 		59CBD2370BBB44DA004A3BE3 /* parsecsv.c in Sources */ = {isa = PBXBuildFile; fileRef = 59CBD2360BBB44DA004A3BE3 /* parsecsv.c */; };
 		59CBD2650BBB4D1B004A3BE3 /* ChapterTitles.m in Sources */ = {isa = PBXBuildFile; fileRef = 593034EA0BBA39A100172349 /* ChapterTitles.m */; };
+		7495D9E40DA024CC00CDC473 /* srt.c in Sources */ = {isa = PBXBuildFile; fileRef = 7495D9E20DA024CC00CDC473 /* srt.c */; };
+		7495D9E50DA024CC00CDC473 /* srt.h in Headers */ = {isa = PBXBuildFile; fileRef = 7495D9E30DA024CC00CDC473 /* srt.h */; };
+		7495D9E60DA024CC00CDC473 /* srt.c in Sources */ = {isa = PBXBuildFile; fileRef = 7495D9E20DA024CC00CDC473 /* srt.c */; };
+		7495D9E70DA024CC00CDC473 /* srt.h in Headers */ = {isa = PBXBuildFile; fileRef = 7495D9E30DA024CC00CDC473 /* srt.h */; };
 		A22C85EC0D05D35000C10E36 /* HBPresets.h in Headers */ = {isa = PBXBuildFile; fileRef = A22C85EA0D05D35000C10E36 /* HBPresets.h */; };
 		A22C85ED0D05D35100C10E36 /* HBPresets.m in Sources */ = {isa = PBXBuildFile; fileRef = A22C85EB0D05D35000C10E36 /* HBPresets.m */; };
 		A25289E60D87A27D00461D5B /* enctheora.c in Sources */ = {isa = PBXBuildFile; fileRef = A25289E50D87A27D00461D5B /* enctheora.c */; };
@@ -258,6 +262,8 @@
 		593034E90BBA39A100172349 /* ChapterTitles.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ChapterTitles.h; sourceTree = "<group>"; };
 		593034EA0BBA39A100172349 /* ChapterTitles.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = ChapterTitles.m; sourceTree = "<group>"; };
 		59CBD2360BBB44DA004A3BE3 /* parsecsv.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; name = parsecsv.c; path = ../test/parsecsv.c; sourceTree = SOURCE_ROOT; };
+		7495D9E20DA024CC00CDC473 /* srt.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; name = srt.c; path = ../libhb/srt.c; sourceTree = SOURCE_ROOT; };
+		7495D9E30DA024CC00CDC473 /* srt.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = srt.h; path = ../libhb/srt.h; sourceTree = SOURCE_ROOT; };
 		A22C85EA0D05D35000C10E36 /* HBPresets.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HBPresets.h; sourceTree = "<group>"; };
 		A22C85EB0D05D35000C10E36 /* HBPresets.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = HBPresets.m; sourceTree = "<group>"; };
 		A25289E50D87A27D00461D5B /* enctheora.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; name = enctheora.c; path = ../libhb/enctheora.c; sourceTree = SOURCE_ROOT; };
@@ -432,6 +438,8 @@
 		526FBC8D0B4CA9F90064E04C /* libhb Sources */ = {
 			isa = PBXGroup;
 			children = (
+				7495D9E20DA024CC00CDC473 /* srt.c */,
+				7495D9E30DA024CC00CDC473 /* srt.h */,
 				A25289E50D87A27D00461D5B /* enctheora.c */,
 				B48359A70C82960500E04440 /* lang.c */,
 				EAA526920C3B25D200944FF2 /* stream.c */,
@@ -567,6 +575,7 @@
 			isa = PBXHeadersBuildPhase;
 			buildActionMask = 2147483647;
 			files = (
+				7495D9E70DA024CC00CDC473 /* srt.h in Headers */,
 			);
 			runOnlyForDeploymentPostprocessing = 0;
 		};
@@ -575,6 +584,7 @@
 			buildActionMask = 2147483647;
 			files = (
 				593034EB0BBA39A100172349 /* ChapterTitles.h in Headers */,
+				7495D9E50DA024CC00CDC473 /* srt.h in Headers */,
 			);
 			runOnlyForDeploymentPostprocessing = 0;
 		};
@@ -789,6 +799,7 @@
 				FC8519550C59A02C0073812C /* deinterlace.c in Sources */,
 				FC8519560C59A02C0073812C /* deblock.c in Sources */,
 				FC8519570C59A02C0073812C /* detelecine.c in Sources */,
+				7495D9E60DA024CC00CDC473 /* srt.c in Sources */,
 			);
 			runOnlyForDeploymentPostprocessing = 0;
 		};
@@ -834,6 +845,7 @@
 				FC8519530C59A02C0073812C /* detelecine.c in Sources */,
 				B48359A80C82960500E04440 /* lang.c in Sources */,
 				A25289E60D87A27D00461D5B /* enctheora.c in Sources */,
+				7495D9E40DA024CC00CDC473 /* srt.c in Sources */,
 			);
 			runOnlyForDeploymentPostprocessing = 0;
 		};
Index: contrib/Jamfile
===================================================================
--- contrib/Jamfile	(revision 1364)
+++ contrib/Jamfile	(working copy)
@@ -227,6 +227,7 @@
     LIBMP4V2_PATCH += "$(PATCH) -p1 < ../patch-mpeg4ip.patch && " ;
     LIBMP4V2_PATCH += "$(PATCH) -p1 < ../patch-mpeg4ip-nasm-2.00-configure.patch && " ;
     LIBMP4V2_PATCH += "$(PATCH) -p1 < ../patch-mpeg4ip-ac3.patch && " ;
+    LIBMP4V2_PATCH += "$(PATCH) -p1 < ../patch-mpeg4ip-srt.patch && " ;
     Depends $(<) : $(>) ;
     Depends lib  : $(<) ;
 }
Index: contrib/patch-mpeg4ip-srt.patch
===================================================================
--- contrib/patch-mpeg4ip-srt.patch	(revision 0)
+++ contrib/patch-mpeg4ip-srt.patch	(revision 0)
@@ -0,0 +1,406 @@
+--- mpeg4ip/lib/mp4v2/Makefile.in	2008-03-30 16:58:59.000000000 -0400
++++ mpeg4ip-patched/lib/mp4v2/Makefile.in	2008-03-30 16:56:50.000000000 -0400
+@@ -55,7 +55,7 @@
+ am_libmp4v2_la_OBJECTS = 3gp.lo atom_amr.lo atom_avc1.lo atom_avcC.lo \
+ 	atom_d263.lo atom_damr.lo atom_dref.lo atom_elst.lo \
+ 	atom_enca.lo atom_encv.lo atom_free.lo atom_ftyp.lo \
+-	atom_gmin.lo atom_text.lo atom_ac3.lo \
++	atom_gmin.lo atom_text.lo atom_ac3.lo atom_tx3g.lo atom_ftab.lo atom_nmhd.lo \
+ 	atom_hdlr.lo atom_hinf.lo atom_hnti.lo atom_href.lo \
+ 	atom_mdat.lo atom_mdhd.lo atom_meta.lo atom_mp4a.lo \
+ 	atom_mp4s.lo atom_mp4v.lo atom_mvhd.lo atom_pasp.lo atom_root.lo \
+@@ -83,6 +83,7 @@
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_enca.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_encv.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_free.Plo \
++@AMDEP_TRUE@	./$(DEPDIR)/atom_ftab.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_ftyp.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_gmin.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_hdlr.Plo \
+@@ -96,6 +97,7 @@
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_mp4s.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_mp4v.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_mvhd.Plo \
++@AMDEP_TRUE@	./$(DEPDIR)/atom_nmhd.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_root.Plo ./$(DEPDIR)/atom_rtp.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_pasp.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_s263.Plo ./$(DEPDIR)/atom_sdp.Plo \
+@@ -112,6 +114,7 @@
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_tkhd.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_treftype.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_trun.Plo \
++@AMDEP_TRUE@	./$(DEPDIR)/atom_tx3g.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_udta.Plo ./$(DEPDIR)/atom_url.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_urn.Plo \
+ @AMDEP_TRUE@	./$(DEPDIR)/atom_video.Plo \
+@@ -354,6 +357,7 @@
+ 	atom_enca.cpp \
+ 	atom_encv.cpp \
+ 	atom_free.cpp \
++	atom_ftab.cpp \
+ 	atom_ftyp.cpp \
+ 	atom_gmin.cpp \
+ 	atom_hdlr.cpp \
+@@ -367,6 +371,7 @@
+ 	atom_mp4s.cpp \
+ 	atom_mp4v.cpp \
+ 	atom_mvhd.cpp \
++	atom_nmhd.cpp \
+ 	atom_pasp.cpp \
+ 	atom_root.cpp \
+ 	atom_rtp.cpp \
+@@ -386,6 +391,7 @@
+ 	atom_tkhd.cpp \
+ 	atom_treftype.cpp \
+ 	atom_trun.cpp \
++	atom_tx3g.cpp \
+ 	atom_udta.cpp \
+ 	atom_url.cpp \
+ 	atom_urn.cpp \
+@@ -510,6 +516,7 @@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_enca.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_encv.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_free.Plo@am__quote@
++@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_ftab.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_ftyp.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_gmin.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_hdlr.Plo@am__quote@
+@@ -523,6 +530,7 @@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_mp4s.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_mp4v.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_mvhd.Plo@am__quote@
++@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_nmhd.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_pasp.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_root.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_rtp.Plo@am__quote@
+@@ -541,6 +549,7 @@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_tkhd.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_treftype.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_trun.Plo@am__quote@
++@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_tx3g.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_udta.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_url.Plo@am__quote@
+ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/atom_urn.Plo@am__quote@
+--- mpeg4ip/lib/mp4v2/Makefile.am	2008-03-30 16:58:59.000000000 -0400
++++ mpeg4ip-patched/lib/mp4v2/Makefile.am	2008-03-30 16:47:28.000000000 -0400
+@@ -22,6 +22,7 @@
+ 	atom_encv.cpp \
+ 	atom_free.cpp \
+ 	atom_ftyp.cpp \
++	atom_ftab.cpp \
+ 	atom_gmin.cpp \
+ 	atom_hdlr.cpp \
+ 	atom_hinf.cpp \
+@@ -35,6 +36,7 @@
+ 	atom_mp4v.cpp \
+ 	atom_mvhd.cpp \
+ 	atom_name.cpp \
++	atom_nmhd.cpp \
+ 	atom_pasp.cpp \
+ 	atom_root.cpp \
+ 	atom_rtp.cpp \
+@@ -53,6 +55,7 @@
+ 	atom_tfhd.cpp \
+ 	atom_tkhd.cpp \
+ 	atom_treftype.cpp \
++	atom_tx3g.cpp \
+ 	atom_trun.cpp \
+ 	atom_udta.cpp \
+ 	atom_url.cpp \
+Index: mpeg4ip/lib/mp4v2/mp4file.h
+===================================================================
+--- mpeg4ip/lib/mp4v2/mp4file.h	(revision 4)
++++ mpeg4ip/lib/mp4v2/mp4file.h	(working copy)
+@@ -303,6 +303,7 @@
+ 	MP4TrackId AddHintTrack(MP4TrackId refTrackId);
+ 	MP4TrackId AddTextTrack(MP4TrackId refTrackId);
+ 	MP4TrackId AddChapterTextTrack(MP4TrackId refTrackId);
++	MP4TrackId AddSubtitleTrack(MP4TrackId refTrackId);
+ 
+ 	MP4TrackId AddPixelAspectRatio(MP4TrackId trackId, u_int32_t hSpacing, u_int32_t vSpacing);
+ 
+Index: mpeg4ip/lib/mp4v2/atom_nmhd.cpp
+===================================================================
+--- mpeg4ip/lib/mp4v2/atom_nmhd.cpp	(revision 0)
++++ mpeg4ip/lib/mp4v2/atom_nmhd.cpp	(revision 0)
+@@ -0,0 +1,25 @@
++/*
++ * The contents of this file are subject to the Mozilla Public
++ * License Version 1.1 (the "License"); you may not use this file
++ * except in compliance with the License. You may obtain a copy of
++ * the License at http://www.mozilla.org/MPL/
++ * 
++ * Software distributed under the License is distributed on an "AS
++ * IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or
++ * implied. See the License for the specific language governing
++ * rights and limitations under the License.
++ * 
++ * The Original Code is MPEG4IP.
++ * 
++ * The Initial Developer of the Original Code is Cisco Systems Inc.
++ * Portions created by Cisco Systems Inc. are
++ * Copyright (C) Cisco Systems Inc. 2001.  All Rights Reserved.
++ */
++
++#include "mp4common.h"
++
++MP4NmhdAtom::MP4NmhdAtom()
++	: MP4Atom("nmhd") 
++{
++	AddVersionAndFlags();
++}
+\ No newline at end of file
+Index: mp4atom.cpp
+===================================================================
+--- mpeg4ip/lib/mp4v2/mp4atom.cpp	(revision 4)
++++ mpeg4ip/lib/mp4v2/mp4atom.cpp	(working copy)
+@@ -120,6 +120,8 @@
+ 	pAtom = new MP4FreeAtom();
+       } else if (ATOMID(type) == ATOMID("ftyp")) {
+ 	pAtom = new MP4FtypAtom();
++      } else if (ATOMID(type) == ATOMID("ftab")) {
++		  pAtom = new MP4FtabAtom();
+       }
+       break;
+     case 'g':
+@@ -176,6 +178,8 @@
+     case 'n':
+       if (ATOMID(type) == ATOMID("name")) { // iTunes
+ 	pAtom = new MP4NameAtom();
++      } else if (ATOMID(type) == ATOMID("nmhd")) {
++		  pAtom = new MP4NmhdAtom();
+       }
+       break;
+     case 'r':
+@@ -216,6 +220,8 @@
+     case 't':
+       if (ATOMID(type) == ATOMID("text")) {
+ 	pAtom = new MP4TextAtom();
++      } else if (ATOMID(type) == ATOMID("tx3g")) {
++	pAtom = new MP4Tx3gAtom();
+       } else if (ATOMID(type) == ATOMID("tkhd")) {
+ 	pAtom = new MP4TkhdAtom();
+       } else if (ATOMID(type) == ATOMID("tfhd")) {
+Index: mpeg4ip/lib/mp4v2/mp4.cpp
+===================================================================
+--- mpeg4ip/lib/mp4v2/mp4.cpp	(revision 4)
++++ mpeg4ip/lib/mp4v2/mp4.cpp	(working copy)
+@@ -973,6 +973,21 @@
+ 	return MP4_INVALID_TRACK_ID;
+ }
+ 
++extern "C" MP4TrackId MP4AddSubtitleTrack(
++	MP4FileHandle hFile, MP4TrackId refTrackId)
++{
++	if (MP4_IS_VALID_FILE_HANDLE(hFile)) {
++		try {
++			return ((MP4File*)hFile)->AddSubtitleTrack(refTrackId);
++		}
++		catch (MP4Error* e) {
++			PRINT_ERROR(e);
++			delete e;
++		}
++	}
++	return MP4_INVALID_TRACK_ID;
++}
++
+ extern "C" MP4TrackId MP4AddChapterTextTrack(
+ 	MP4FileHandle hFile, MP4TrackId refTrackId)
+ {
+Index: mpeg4ip/lib/mp4v2/atom_tx3g.cpp
+===================================================================
+--- mpeg4ip/lib/mp4v2/atom_tx3g.cpp	(revision 0)
++++ mpeg4ip/lib/mp4v2/atom_tx3g.cpp	(revision 0)
+@@ -0,0 +1,55 @@
++/*
++ * The contents of this file are subject to the Mozilla Public
++ * License Version 1.1 (the "License"); you may not use this file
++ * except in compliance with the License. You may obtain a copy of
++ * the License at http://www.mozilla.org/MPL/
++ * 
++ * Software distributed under the License is distributed on an "AS
++ * IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or
++ * implied. See the License for the specific language governing
++ * rights and limitations under the License.
++ * 
++ * The Original Code is MPEG4IP.
++ * 
++ * The Initial Developer of the Original Code is Cisco Systems Inc.
++ * Portions created by Cisco Systems Inc. are
++ * Copyright (C) Cisco Systems Inc. 2001.  All Rights Reserved.
++ */
++
++#include "mp4common.h"
++
++MP4Tx3gAtom::MP4Tx3gAtom() 
++	: MP4Atom("tx3g") 
++{
++	AddReserved("reserved1", 4); /* 0 */
++	AddReserved("reserved2", 2); /* 1 */
++	
++	AddProperty(new MP4Integer16Property("dataReferenceIndex"));/* 2 */
++	
++	AddProperty(new MP4Integer32Property("displayFlags")); /* 3 */
++	AddProperty(new MP4Integer8Property("horizontalJustification")); /* 4 */
++	AddProperty(new MP4Integer8Property("verticalJustification")); /* 5 */
++
++	AddProperty(new MP4Integer8Property("bgColorRed")); /* 6 */
++	AddProperty(new MP4Integer8Property("bgColorGreen")); /* 7 */
++	AddProperty(new MP4Integer8Property("bgColorBlue")); /* 8 */
++	AddProperty(new MP4Integer8Property("bgColorAlpha")); /* 9 */
++
++	AddProperty(new MP4Integer16Property("defTextBoxTop")); /* 10 */
++	AddProperty(new MP4Integer16Property("defTextBoxLeft")); /* 11 */
++	AddProperty(new MP4Integer16Property("defTextBoxBottom")); /* 12 */
++	AddProperty(new MP4Integer16Property("defTextBoxRight")); /* 13 */
++	
++	AddProperty(new MP4Integer16Property("startChar")); /* 14 */
++	AddProperty(new MP4Integer16Property("endChar")); /* 15 */
++	AddProperty(new MP4Integer16Property("fontID")); /* 16 */
++	AddProperty(new MP4Integer8Property("fontFace")); /* 17 */
++	AddProperty(new MP4Integer8Property("fontSize")); /* 18 */
++	
++	AddProperty(new MP4Integer8Property("fontColorRed")); /* 19 */
++	AddProperty(new MP4Integer8Property("fontColorGreen")); /* 20 */
++	AddProperty(new MP4Integer8Property("fontColorBlue")); /* 21 */
++	AddProperty(new MP4Integer8Property("fontColorAlpha")); /* 22 */
++	
++	ExpectChildAtom("ftab", Optional, Many);
++}
+Index: mpeg4ip/lib/mp4v2/atoms.h
+===================================================================
+--- mpeg4ip/lib/mp4v2/atoms.h	(revision 4)
++++ mpeg4ip/lib/mp4v2/atoms.h	(working copy)
+@@ -353,6 +353,16 @@
+ 	void GenerateGmhdType();
+ };
+ 
++class MP4Tx3gAtom : public MP4Atom {
++public:
++	MP4Tx3gAtom();
++};
++
++class MP4FtabAtom : public MP4Atom {
++public:
++	MP4FtabAtom();
++};
++
+ class MP4TfhdAtom : public MP4Atom {
+ public:
+ 	MP4TfhdAtom();
+@@ -406,4 +416,9 @@
+     void Generate();
+ };
+ 
++class MP4NmhdAtom : public MP4Atom {
++public:
++	MP4NmhdAtom();
++};
++
+ #endif /* __MP4_ATOMS_INCLUDED__ */
+Index: mpeg4ip/lib/mp4v2/atom_ftab.cpp
+===================================================================
+--- mpeg4ip/lib/mp4v2/atom_ftab.cpp	(revision 0)
++++ mpeg4ip/lib/mp4v2/atom_ftab.cpp	(revision 0)
+@@ -0,0 +1,32 @@
++/*
++ * The contents of this file are subject to the Mozilla Public
++ * License Version 1.1 (the "License"); you may not use this file
++ * except in compliance with the License. You may obtain a copy of
++ * the License at http://www.mozilla.org/MPL/
++ * 
++ * Software distributed under the License is distributed on an "AS
++ * IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or
++ * implied. See the License for the specific language governing
++ * rights and limitations under the License.
++ * 
++ * The Original Code is MPEG4IP.
++ * 
++ * The Initial Developer of the Original Code is Cisco Systems Inc.
++ * Portions created by Cisco Systems Inc. are
++ * Copyright (C) Cisco Systems Inc. 2001.  All Rights Reserved.
++ */
++
++#include "mp4common.h"
++
++MP4FtabAtom::MP4FtabAtom()
++	: MP4Atom("ftab") 
++{
++	MP4Integer16Property* pCount = new MP4Integer16Property("entryCount"); /* 0 */
++	AddProperty(pCount);
++
++	MP4TableProperty* pTable = new MP4TableProperty("fontEntries", pCount);  /* 1 */
++	AddProperty(pTable);
++
++	pTable->AddProperty(new MP4Integer16Property("fontID"));  /* 0 */
++	pTable->AddProperty(new MP4StringProperty("name", true));  /* 1 */	
++}
+Index: mpeg4ip/lib/mp4v2/mp4file.cpp
+===================================================================
+--- mpeg4ip/lib/mp4v2/mp4file.cpp	(revision 4)
++++ mpeg4ip/lib/mp4v2/mp4file.cpp	(working copy)
+@@ -1948,6 +1948,49 @@
+ 	return trackId;
+ }
+ 
++MP4TrackId MP4File::AddSubtitleTrack(MP4TrackId refTrackId)
++{
++	// validate reference track id
++	FindTrackIndex(refTrackId);
++	
++	MP4TrackId trackId = 
++	AddTrack(MP4_TEXT_TRACK_TYPE, GetTrackTimeScale(refTrackId));
++	
++	InsertChildAtom(MakeTrackName(trackId, "mdia.minf"), "nmhd", 0);
++	
++	AddChildAtom(MakeTrackName(trackId, "mdia.minf.stbl.stsd"), "tx3g");
++	
++	MP4StringProperty* pHandlerTypeProperty;
++	FindStringProperty(MakeTrackName(trackId, "mdia.hdlr.handlerType"),
++					   (MP4Property**)&pHandlerTypeProperty);
++	pHandlerTypeProperty->SetValue("sbtl");
++	
++	// Hardcoded crap... add the ftab atom and add one font entry
++	MP4Atom* pFtabAtom = AddChildAtom(MakeTrackName(trackId, "mdia.minf.stbl.stsd.tx3g"), "ftab");
++	
++	((MP4Integer16Property*)pFtabAtom->GetProperty(0))->IncrementValue();
++	
++	MP4Integer16Property* pfontID = (MP4Integer16Property*)((MP4TableProperty*)pFtabAtom->GetProperty(1))->GetProperty(0);
++	pfontID->AddValue(1);
++	
++	MP4StringProperty* pName = (MP4StringProperty*)((MP4TableProperty*)pFtabAtom->GetProperty(1))->GetProperty(1);
++	pName->AddValue("Arial");
++	
++	
++	// stsd is a unique beast in that it has a count of the number 
++	// of child atoms that needs to be incremented after we add the text atom
++	MP4Integer32Property* pStsdCountProperty;
++	FindIntegerProperty(
++		MakeTrackName(trackId, "mdia.minf.stbl.stsd.entryCount"),
++		(MP4Property**)&pStsdCountProperty);
++	pStsdCountProperty->IncrementValue();
++
++	/* add the magic "text" atom to the generic media header */
++//	AddChildAtom(MakeTrackName(trackId, "mdia.minf.gmhd"), "text");
++	
++	return trackId;
++}
++
+ MP4TrackId MP4File::AddChapterTextTrack(MP4TrackId refTrackId)
+ {
+ 	// validate reference track id
+Index: mpeg4ip/lib/mp4v2/mp4.h
+===================================================================
+--- mpeg4ip/lib/mp4v2/mp4.h	(revision 4)
++++ mpeg4ip/lib/mp4v2/mp4.h	(working copy)
+@@ -553,6 +553,10 @@
+ 	MP4FileHandle hFile, 
+ 	MP4TrackId refTrackId);
+ 
++MP4TrackId MP4AddSubtitleTrack(
++	MP4FileHandle hFile,
++	MP4TrackId refTrackId);
++	
+ MP4TrackId MP4AddPixelAspectRatio(
+ 	MP4FileHandle hFile, 
+ 	MP4TrackId refTrackId,
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by jbrjake »

Oh yeah..noticed a couple issues:

When I tried to load from a file on my desktop with spaces in the name and path (escaped with backslashes) they failed to load, timecodes were never parsed. Copied the same file to the working dir and removed the spaces in the name, and it worked fine.

I can't see the subtitles in QuickTime, at all. Subtitles menu is empty. But they show in iTunes. Haven't tried on my phone yet.

QuickTime won't even list the text track in the properties window, unless the file name ends .m4v. But it still doesn't show them in the Subtitles menu.
entropic
Novice
Posts: 51
Joined: Wed Apr 11, 2007 8:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by entropic »

Wow that was quick :D
My obvious error in the srt time calculation is trivial but I dont think it will solve all syncing issuse
Not even worth a diff: srt.c:19 & 20 - change 36000 to 3600000.

As for your problems with quicktime - i found that QT is pretty picky with language codes... i only know for sure that german and english work.
It seems mp4s with subtitles have to be named .m4v too - same as with chapters. In mp4s QT never shows anything except the audio and video tracks...

I've been using QT mostly during all my tests - was not aware that there are cases where itunes is showing more then QT... does itunes show the subtitle menu? or does it just show the subtitles? which language code did your file use? (does it show if you change it to german or english?)

Spaces in srt paths dont work? Too bad that my srt reader is so [Censored]...
Ill try to get some work done tomorrow
saintdev
Enlightened
Posts: 146
Joined: Wed Dec 20, 2006 4:17 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by saintdev »

Here's my thoughts.
MP4CreateSubtitleTrack and MP4WriteSubtitleSample should be added to libmp4v2, not part of muxmp4.c. I'm not sure exactly what MP4SetTrackYTranslation does, but my guess is it should probably be part of libmp4v2 also.
We really should be reading the srt file in a separate thread and passing it along the FIFOs. For several reasons. This way any of the muxers can use the srt info. Also, that way we won't block on file I/O while the other threads are cranking out frames waiting for the muxer to pull them off their FIFO.
A loop would probably be better (and easier) to use in srt_free_entries. Function call overhead FTW!

A minor point you shouldn't use #pragma once. While the GCC preprocessor understands it, other compilers may not. Even though GCC is all that we use, using a wrapper #ifndef is much more portable, and recommended.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by jbrjake »

saintdev wrote:We really should be reading the srt file in a separate thread and passing it along the FIFOs.
Uh, not sure I follow. What entropic's provided front-loads the whole subtitle track at once when the MP4 is first initialized. IMO there is no need for separate threads or FIFOs because it's not running concurrently with the encode...

How could it block I/O?
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by jbrjake »

entropic wrote:My obvious error in the srt time calculation is trivial but I dont think it will solve all syncing issuse
Not even worth a diff: srt.c:19 & 20 - change 36000 to 3600000.
Oh, is that all? I thought the stuff in the libmp4v2 patch would have to be changed too. I changed to 3600000 but they still seem to be out of sync...
As for your problems with quicktime - i found that QT is pretty picky with language codes... i only know for sure that german and english work.
Okay, yeah, that was the problem -- for some reason HB isn't properly setting language codes for some mpeg-2 streams, like the one I used for testing; it comes out as "Undetermined." Once I set the track's language in QTPro's properties, the Subtitle menu populated and I could turn it on.

Also: are the subs supposed to show on a black, translucent backdrop? Because they are...
entropic
Novice
Posts: 51
Joined: Wed Apr 11, 2007 8:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by entropic »

It seems the black background thing is done automatically by QT as soon it recognizes it as subtitles — looks like it's ignoring most of the style properties in the tx3g atom. They are honored as long as the tx3g atoms component sub-type is set to 'text' — but stop working as soon as its set to 'sbtl'...
(Or it might be because the style table sub-atom is missing and its required for sbtl-tx3g tracks and not for text ones?? — weird but i would be surprised and having fun with the ftab atom)
It would really help if I had a video from the iTS with subtitles to see how they do it. (Unfortunately even buying is no option in europe)
Maybe this black background is intended? It sure seems so... Back when I was still playing with the basics I found a way to get rid of it — something weird like setting some garbage defTextBox... surely didn't feel like an apple-intended way to do it... but who knows.
Any chance that somebody who got one can verify that they have this black semi-transparent background too?

Lets see what i can do about the syncing issues that are left… maybe some other little magic flag thats not set correctly... (If apple only would give us some specs on how this is supposed to work...)
Language code wise it seems that only the languages offered in the itunes preferences are accepted (which kinda sucks... i still hope i got this wrong)

Some other things i've been thinking about:
I'd say parsing of the srt file is quick enough to just do it in a pre-encode step - I would even go so far to decouple reading of the subs much more from the rest of libhb. In the end we might want stuff like showing the subs in the UI before the encode is started... maybe even let the user edit them (and set the srt-file's encoding; we have to convert them to unicode after all...)
The muxing code should only deal with decoded subs — no matter where they are coming from.
Maybe one day we really get ocr and I surly don't want this too deep in the system.
Makes sense?

Ben
saintdev
Enlightened
Posts: 146
Joined: Wed Dec 20, 2006 4:17 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by saintdev »

entropic wrote:Some other things i've been thinking about:
I'd say parsing of the srt file is quick enough to just do it in a pre-encode step - I would even go so far to decouple reading of the subs much more from the rest of libhb. In the end we might want stuff like showing the subs in the UI before the encode is started... maybe even let the user edit them (and set the srt-file's encoding; we have to convert them to unicode after all...)
The muxing code should only deal with decoded subs — no matter where they are coming from.
Maybe one day we really get ocr and I surly don't want this too deep in the system.
Makes sense?

Ben
I disagree. We are a dvd ripper, not a sub editor. Use notepad or something similar if you want that. Having the UI parse the subs is IMO a bad idea. With the idea of passing it through a subtitle fifo, this way you could not only handle srt subtitles read from an outside file, but you could also use the same fifo to pass vob (or ass or ssa or usf or....) subtitles to the muxers. You aren't limited to just the one type of subs. Also, that way the subtitle reader would be in a seperate thread, so it wouldn't matter how fast (or slow) it is, it'll only block it's thread while reading from disk, not the whole muxer.

@jbrjake, I didn't say "it blocks I/O" I said "it blocks on I/O" (ie: blocks waiting for fopen/reads).
entropic
Novice
Posts: 51
Joined: Wed Apr 11, 2007 8:38 am

Syncing fixed

Post by entropic »

Just got rid of all the syncing problems it seems.
Revised version of the patch:
http://opensource.entropicgarden.com/ha ... -srt.patch

One interresting thing I just discovered is that the iphone seems to treat <i> tags in subs correctly without any special messing with the tx3g style table... QT however doesn't.
We are a dvd ripper, not a sub editor. Use notepad or something similar if you want that.
I absolutely agree - I was mainly thinking that we might want to show the user the subs and let him choose an encoding.
Alternatively, we could just ignore all the encoding crap and only read unicode files. (Which would be totally ok for me since im converting all my srts to utf8 for years now)
With the idea of passing it through a subtitle fifo, this way you could not only handle srt subtitles read from an outside file, but you could also use the same fifo to pass vob (or ass or ssa or usf or....) subtitles to the muxers. You aren't limited to just the one type of subs.
Not limiting it to one type of subs.. holy [Censored].. i still have a hard time believing that we got any subs at all :shock:
That said, i kinda like your idea even tho it sounds a bit like overkill for now...
Attachments
hb-srt.patch.txt
(527.47 KiB) Downloaded 24 times
Last edited by s55 on Wed Jun 25, 2008 10:15 pm, edited 1 time in total.
Reason: Added file to HB server
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by dynaflash »

Of course it would all be good to have. However, what you are doing entropic is much better than what we currently have. I for one would like to see you continue on your quest. :) If we waited to hit everything out of the ballpark before getting it into HandBrake, we would still be at 0.7.1 especially considering we are all doing this on our spare time. Good stuff entropic :)
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Syncing fixed

Post by jbrjake »

entropic wrote:Just got rid of all the syncing problems it seems.
Revised version of the patch:
http://opensource.entropicgarden.com/ha ... -srt.patch
Did you make any changes to the libmp4 patch? You didn't include it in that link (I'm guessing you didn't run "svn add" on the file before doing "svn diff" ).
Not limiting it to one type of subs.. holy [Censored].. i still have a hard time believing that we got any subs at all :shock:
That said, i kinda like your idea even tho it sounds a bit like overkill for now...
I have to agree... overkill. Perfect's the enemy of good, and all that.
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by dynaflash »

Entropic: as well feel free to join us on our irc channels where we spend most of our time in development.
dodo
Posts: 1
Joined: Thu Apr 10, 2008 5:20 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by dodo »

I've taken the code from entropic and jbrjake and combined them in a correct patch against the latest SVN (1399). The patch is at: http://handbrake.pastebin.ca/979694. I made sure the files srt.[ch] are included as well as the patch for mpeg4ip.

Things that would be interesting is the ability to add multiple srt files, specify the language of the SRT file, and maybe an offset to fix wrong times in the SRT file. I might look and take a crack at that later this week. One thing I did notice was a problem with ripping a chapter, the srt will start at time 0, not the time of the chapter.

Besides this, I love this patch, this is exactly what I was looking for.

** edited to make sure I pointed to the right patch, not the old patch. This should now work with the latest version in SVN.
Last edited by dodo on Thu Apr 10, 2008 3:14 pm, edited 2 times in total.
entropic
Novice
Posts: 51
Joined: Wed Apr 11, 2007 8:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by entropic »

dodo wrote:One thing I did notice was a problem with ripping a chapter, the srt will start at time 0, not the time of the chapter.
Support for this is missing - im already working on it. Still having some syncing issues in this case. Should be ready soon tho
entropic
Novice
Posts: 51
Joined: Wed Apr 11, 2007 8:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by entropic »

A bit cleaned up and with support for encoding of only parts of a title.

Diffed agains r1401
http://opensource.entropicgarden.com/ha ... -srt.patch
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by jbrjake »

*Thread cleaned up, mewling from users deleted.*
entropic
Novice
Posts: 51
Joined: Wed Apr 11, 2007 8:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by entropic »

Next revision.

- Support for multiple text streams
- UI integration

http://opensource.entropicgarden.com/ha ... -srt.patch
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by dynaflash »

Okay, imho this would be a great addition to HB. I am happy to handle the macgui part as far as integrating it into our svn, but know that the libhb dev's want to go over where the libhb functions go etc.
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by eddyg »

mp4v2 changes went into http://code.google.com/p/mp4v2/source/detail?r=23 , http://code.google.com/p/mp4v2/source/detail?r=24

HB changes to follow. We've lost the Mac GUI changes for now.

Cheers, Ed.
entropic
Novice
Posts: 51
Joined: Wed Apr 11, 2007 8:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by entropic »

Now where handbrake is slowly evolving from a "DVD to MPEG-4 converter" to a general transcoding tool I think it would make sense to give this whole subtitle problem another try. First step is that I started switching Muxo over to the code.google.com mp4v2 repository and im in the process to ensure that everything is still working. Let's see where this is going.
rhester
Veteran User
Posts: 2888
Joined: Tue Apr 18, 2006 10:24 pm

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by rhester »

entropic wrote:I started switching Muxo over to the code.google.com mp4v2 repository
Where? I can't find it.

And for that matter, where have you been? :)

Rodney
entropic
Novice
Posts: 51
Joined: Wed Apr 11, 2007 8:38 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by entropic »

Nothing checked in so far. You guys did quite some crazy stuff with mp4v2... like breaking reading of avc1 atoms :?
Anyway.. I just got Muxo kinda working again.
Now i have to decide what i should move from muxo to mp4v2... probably not much since all the basic stuff was already working half a year(?) ago. Meaning i can start again cramming the hole thing into the current hb-svn-head...
I've been pretty busy with my life lately - but i hope i can find some time now to finish the crap i started :D
On the other hand im still trying to decide what to do about Muxo... :roll:

Ben
nightstrm
Veteran User
Posts: 1887
Joined: Fri Mar 23, 2007 5:43 am

Re: [Patch (v0.001)] Subtitle support for mp4 files

Post by nightstrm »

entropic wrote:Nothing checked in so far. You guys did quite some crazy stuff with mp4v2... like breaking reading of avc1 atoms :?
Anyway.. I just got Muxo kinda working again.
Now i have to decide what i should move from muxo to mp4v2... probably not much since all the basic stuff was already working half a year(?) ago. Meaning i can start again cramming the hole thing into the current hb-svn-head...
I've been pretty busy with my life lately - but i hope i can find some time now to finish the crap i started :D
On the other hand im still trying to decide what to do about Muxo... :roll:

Ben
I'd have a talk with eddyg on IRC if you can... I think he has been working the past couple days to cram your code into the current SVN head.

Welcome back btw! :D
Post Reply