Storage Hardware
Forum rules
An Activity Log is required for support requests. Please read How-to get an activity log? for details on how and why this should be provided.
An Activity Log is required for support requests. Please read How-to get an activity log? for details on how and why this should be provided.
Storage Hardware
I've been using HandBreak for some time and just wanted to personally thank all the developers that have put so much time and hard work into developing this program. It really is appreciated! Thank You!
As many on the forum I had been waiting patiently for the AppleTv hardware and now that it's here i'm looking to transfer my extensive DVD collection to a server.
Before investing in a hardware storage soultion I was wonder how others are storing their video, audio and photo collections (for primary use with Apple TV). Since each Apple TV can only hold 40gig I assume most will be placing their collections on some sort of remote server that will be running iTunes. (Streaming not synced)
Could anyone please post their hardware solutions on this thread. i.e. Are you using raid drives, what size, off-the-shelf solutions, is it backed up, using an old pc, windows, mac based, trade-offs....etc.... Also any best practices in configuring itunes to run in this type of environment?
Thank You,
Jim
As many on the forum I had been waiting patiently for the AppleTv hardware and now that it's here i'm looking to transfer my extensive DVD collection to a server.
Before investing in a hardware storage soultion I was wonder how others are storing their video, audio and photo collections (for primary use with Apple TV). Since each Apple TV can only hold 40gig I assume most will be placing their collections on some sort of remote server that will be running iTunes. (Streaming not synced)
Could anyone please post their hardware solutions on this thread. i.e. Are you using raid drives, what size, off-the-shelf solutions, is it backed up, using an old pc, windows, mac based, trade-offs....etc.... Also any best practices in configuring itunes to run in this type of environment?
Thank You,
Jim
-
- Posts: 10
- Joined: Fri Mar 23, 2007 5:22 pm
Good Thread. This is on my mind as well. Right now I've only ripped a few and they're sitting on my G5 in my living room. My plan is to definatley stream, however I do notice that my G5's hard drive is really rev'd up to provide the stream. Durring quiet passages in the movies it is a bit annoying, so I'm considering some sort of NAS solution that I could keep in the basement. Maybe attached to a Airport? not sure yet. Would be nice to have a hardware based raid 5 solution for the NAS box.
I just put in an 80GB drive it was easy took only a few minutes.
Getting a hack so NAS device to work may take a bit more work.
As you need iTunes running to get streaming video and music.
You could get an adapter to go to an external 750 GB, that would look ugly but should be very easy with a quick trip to Fry's.
If you could get an 2.5 " IDE to firewire convertor that would be sweet.
Then it's al all in one box and you bypass networking all together (until you wanted add more content.)
Getting a hack so NAS device to work may take a bit more work.
As you need iTunes running to get streaming video and music.
You could get an adapter to go to an external 750 GB, that would look ugly but should be very easy with a quick trip to Fry's.
If you could get an 2.5 " IDE to firewire convertor that would be sweet.
Then it's al all in one box and you bypass networking all together (until you wanted add more content.)
my storage setup:
Storage Controller: Shuttle with a P4 2.5Ghz, 1Gig of ram
OS: Redhat Linux EL4-update 2
Storage Array: Norco DS-1220 12 drive eSata chassis with 12 350G drives
Storage Controller: Si3124 Controller card
Drives are partiioned as 2 Raid 5 arrays with 2 hot spares.
raid overhead + os overhead + lvm overhead =
Ive got it Netaltalk running so I can mount it via AFP to my G5 mac and have iTunes configured to use the volume as my storage
Storage Controller: Shuttle with a P4 2.5Ghz, 1Gig of ram
OS: Redhat Linux EL4-update 2
Storage Array: Norco DS-1220 12 drive eSata chassis with 12 350G drives
Storage Controller: Si3124 Controller card
Drives are partiioned as 2 Raid 5 arrays with 2 hot spares.
raid overhead + os overhead + lvm overhead =
Code: Select all
Total used free
2.2T 850G 1.3T
I just chain drives to a Mac Mini. I don't place all my media files into the iTunes music folder, so I don't need one large volume across multiple drives (e.g., RAID). I think that's the only iTunes preference applicable to my storage setup.
I keep the AppleTV synced to playlists for the latest, unwatched TV shows (EyeTV, iTunes) and video podcasts. I don't keep it in streaming mode so I don't have to wade through very long lists of content when I usually only want to see new stuff. Switching it to use the Mac as a source (streaming mode) only takes a second when looking for older content (e.g., your DVD backups).
Before I got started with all this, I was concerned about sleeping drives when in streaming mode. (I have one drive that refuses to honor power management settings and will always spin down after five minutes.) It's not a problem, as the Apple TV will simply wait the few seconds until any needed drives on the Mac spin up.
For backup, I use a Windows tower with lots of drive bays. The backups are managed on the Mac using rsync.
Good luck!
I keep the AppleTV synced to playlists for the latest, unwatched TV shows (EyeTV, iTunes) and video podcasts. I don't keep it in streaming mode so I don't have to wade through very long lists of content when I usually only want to see new stuff. Switching it to use the Mac as a source (streaming mode) only takes a second when looking for older content (e.g., your DVD backups).
Before I got started with all this, I was concerned about sleeping drives when in streaming mode. (I have one drive that refuses to honor power management settings and will always spin down after five minutes.) It's not a problem, as the Apple TV will simply wait the few seconds until any needed drives on the Mac spin up.
For backup, I use a Windows tower with lots of drive bays. The backups are managed on the Mac using rsync.
Good luck!
I have my network set up as follows:
I have an AEBS, with a 500GB external drive hooked up to it via Airport Disk. I have a PPC G4 PowerBook, a Core 2 Duo MacBook, and a G4 800 MHz iMac. I have wired the iMac to the AEBS and the rest connect wirelessly. All of them access my media from the Airport Disk external drive.
Initially, I utilized the iTunes on my PowerBook as the conduit for AppleTV. It was quite spotty when watching the better quality video. I thought it was the mere fact that it was wireless, but the clips I would pull from Apple directly though the Apple TV/internet worked seamlessly. I surmised that it was the doubling of wireless access (Apple TV -> PowerBook -> Airport Disk -> Powerbook -> to Apple TV). To test it out, I then set up my Apple TV to utilize the iMac iTunes as the conduit. Since then it has been smooth sailing. Even though my iMac is the slowest Mac I have, the fact that I have now set it up as a dedicated Apple TV conduit has it working like a charm. I currently stream all of my media, I have nothing sync-ed on the Apple TV.
I would use the MacBook, however, my wife has commandeered it for much higher purposes, like shopping........
I have an AEBS, with a 500GB external drive hooked up to it via Airport Disk. I have a PPC G4 PowerBook, a Core 2 Duo MacBook, and a G4 800 MHz iMac. I have wired the iMac to the AEBS and the rest connect wirelessly. All of them access my media from the Airport Disk external drive.
Initially, I utilized the iTunes on my PowerBook as the conduit for AppleTV. It was quite spotty when watching the better quality video. I thought it was the mere fact that it was wireless, but the clips I would pull from Apple directly though the Apple TV/internet worked seamlessly. I surmised that it was the doubling of wireless access (Apple TV -> PowerBook -> Airport Disk -> Powerbook -> to Apple TV). To test it out, I then set up my Apple TV to utilize the iMac iTunes as the conduit. Since then it has been smooth sailing. Even though my iMac is the slowest Mac I have, the fact that I have now set it up as a dedicated Apple TV conduit has it working like a charm. I currently stream all of my media, I have nothing sync-ed on the Apple TV.
I would use the MacBook, however, my wife has commandeered it for much higher purposes, like shopping........
-
- Posts: 8
- Joined: Tue Mar 27, 2007 1:42 am
Thanks for making this thread! I'm definitely at the point (thanks to Apple TV) where I am ready to get rid of actual physical media. I'm just waiting for the new version of HB with some sort of audio solution (5.1, DPLI or DPLII.. whatever.. anything other than stereo) and then I'm going to convert videos and TV show DVDs non-stop. My question to all of you who know a lot about servers is: Is there anything wrong with just filling an old PC with hard drives, installing iTunes and running all of my media from that? I know that true servers would be the best way, but that is all over my head and would be content with just filling up an old PC box. I know it's not ideal, but is it necessarily wrong? Are there any precautions I should take or additions to the PC that should be made if I intend to do this?
I've got an Apple Airport Extreme with Air Disk enabled sharing my Sans Digital 2TB RAID array. It's connected via USB and has a fully managed internal RAID system that the Airport doesnt have to think about. 5 500GB Drives in RAID 5 = 2 TB of useable Raw space, 1.8TB of formatted space.
Sans Digital Box: http://www.sansdigital.com/MR5CT1.html
I use a Mac Mini to share the data with iTunes ... but I sure wish the Airport Extreme could do that directly, or that the Apple TV could actually store its data on a remote system!
Sans Digital Box: http://www.sansdigital.com/MR5CT1.html
I use a Mac Mini to share the data with iTunes ... but I sure wish the Airport Extreme could do that directly, or that the Apple TV could actually store its data on a remote system!
See:Diranged wrote:Airport Express doesnt have eSata and I wanted it to be available to the whole network, even when my Mac Mini is down.
http://www.infrant.com/products/product ... S%20NVPlus
Supposed to be *very* well regarded...there are a whole bunch of glowing reviews on MacInTouch.
I beg to differ... USB 2.0 = 480mb/sec. As it is, I easily max out the 100Mbit port on the Airport Expressbaggss wrote:Ya, that is true, I forgot you said you had it hooked up to that. I just couldn't bring myself to hobble the system with USB though, it's just to slow.Diranged wrote:Airport Express doesnt have eSata and I wanted it to be available to the whole network, even when my Mac Mini is down.
Disagree all you want, but compared to FireWire and eSATA, USB 2 SUCKS. BTW, USB 2 = 480Mb burst not sustained.Diranged wrote:I beg to differ... USB 2.0 = 480mb/sec. As it is, I easily max out the 100Mbit port on the Airport Expressbaggss wrote:Ya, that is true, I forgot you said you had it hooked up to that. I just couldn't bring myself to hobble the system with USB though, it's just to slow.Diranged wrote:Airport Express doesnt have eSata and I wanted it to be available to the whole network, even when my Mac Mini is down.
USB 2.0 Hi-Speed versus FireWire 400
Personal experience on my part bares this out as well. I can see how it might overwhelm the APEx though.The signaling rate of USB 2.0 Hi-Speed mode is 480 Mb/s, while the signaling rate of FireWire 400 (IEEE 1394a, the slower, yet more common variant of firewire as of 2007) is 393.216 Mb/s,[7] which would appear to indicate that USB should be slightly faster but reality is far more complicated than that and other design factors can dwarf a relatively small difference in signaling rate. USB requires more host processing power than FireWire due to the need for the host to provide the arbitration and scheduling of transactions. USB transfer rates are theoretically higher than FireWire due to the need for FireWire devices to arbitrate for bus access. A single FireWire device may achieve a transfer rate for FireWire 400 as high as 41 MB/s, while for USB 2.0 the rate can theoretically be 55 MB/s (for a single device). In a multi-device environment FireWire rapidly loses ground to USB: FireWire's mixed speed networks and long connection chains dramatically affect its performance.
The peer-to-peer nature of FireWire requires devices to arbitrate, which means a FireWire bus must wait until a given signal has propagated to all devices on the bus. The more devices on the bus, the lower is its peak performance. Conversely, for USB the maximum timing model is fixed and is limited only by the host-device branch (not the entire network). Furthermore, the host-centric nature of USB allows the host to allocate more bandwidth to high priority devices instead of forcing them to compete for bandwidth as in FireWire.
All of the above indicates that USB 2 should be faster than firewire 400 but tests have generally shown that for mass storage using normal consumer equipment firewire is faster often by a significant margin.
Just keep in mind for all of you running large raid-0 arays or just big honkin 500G drives.
Do you backup?
can your big aray handle a drive failure?
is it worth your time to re-rip all your content you have on the drives?
my setup gives my 2.1Tb of space and lets me lose 2 drives per array before I lose data, giving me plenty of time to get a replacement drive and rebuild the array .. with 0 data loss.
Also be sure to reemeber this: avg drive life is 5yrs, witht he first yr being the sink of swim stage. Google posted an analysis of thier drive failure rate its a good read
Do you backup?
can your big aray handle a drive failure?
is it worth your time to re-rip all your content you have on the drives?
my setup gives my 2.1Tb of space and lets me lose 2 drives per array before I lose data, giving me plenty of time to get a replacement drive and rebuild the array .. with 0 data loss.
Also be sure to reemeber this: avg drive life is 5yrs, witht he first yr being the sink of swim stage. Google posted an analysis of thier drive failure rate its a good read
Yes firewire-400 is faster than usb-2. I have both and I still use the usb-2 simply because it's fast enough for anything I do.
Ripping a DVD to HD. DVD is the bottleneck.
Compressing DVD to mp4. CPU is the bottleneck.
So if you don't really need fast disk access you may as well buy a cheaper usb-2 drive.
Ripping a DVD to HD. DVD is the bottleneck.
Compressing DVD to mp4. CPU is the bottleneck.
So if you don't really need fast disk access you may as well buy a cheaper usb-2 drive.
I believe we're exclusively discussing storage options for an AppleTV server that streams content. Since the AppleTV's file bitrate bandwidth maxes out at 5mbps (though you can push it near 6mbps) and you can only attach up to 5 AppleTV's to a streaming server the maximum amount of data all the AppleTV's connected to your server could request is 30mbps.
USB2 bursts data at up to 480mbps which is 40x greater than the maximum amount of data your AppleTVs could be requesting. Some people claim USB2 sustained transfer rates of 4MB/s, while this is sort of silly since USB2 isn't a sustained transfer protocol like Firewire even a paltry 4MB/s of bandwidth could handle 5 AppleTV's streaming files at a bitrate higher than Apple claims the AppleTV can handle.
So, for the purposes of this discussion, I don't see any negative side effect to using USB2 drives for server storage.
-Berylium
USB2 bursts data at up to 480mbps which is 40x greater than the maximum amount of data your AppleTVs could be requesting. Some people claim USB2 sustained transfer rates of 4MB/s, while this is sort of silly since USB2 isn't a sustained transfer protocol like Firewire even a paltry 4MB/s of bandwidth could handle 5 AppleTV's streaming files at a bitrate higher than Apple claims the AppleTV can handle.
So, for the purposes of this discussion, I don't see any negative side effect to using USB2 drives for server storage.
-Berylium
All good points, especially in regards to ATV. I'm merely thinking of uploading/backing up with USB 2 and the amount of time required to do so.Berylium wrote:I believe we're exclusively discussing storage options for an AppleTV server that streams content. Since the AppleTV's file bitrate bandwidth maxes out at 5mbps (though you can push it near 6mbps) and you can only attach up to 5 AppleTV's to a streaming server the maximum amount of data all the AppleTV's connected to your server could request is 30mbps.
USB2 bursts data at up to 480mbps which is 40x greater than the maximum amount of data your AppleTVs could be requesting. Some people claim USB2 sustained transfer rates of 4MB/s, while this is sort of silly since USB2 isn't a sustained transfer protocol like Firewire even a paltry 4MB/s of bandwidth could handle 5 AppleTV's streaming files at a bitrate higher than Apple claims the AppleTV can handle.
So, for the purposes of this discussion, I don't see any negative side effect to using USB2 drives for server storage.
-Berylium
I too have been looking for options to backup my Mac systems and store multimedia on the network.
Since switching to the Mac I've really been disappointed with the lack of built-in backup capabilities of Mac OS X and the pathetic third party backup and restore options. Nothing provides basic functionality like NTBACKUP does on Windows. I hope someone could correct me on this as I would like to create backup sets (initially a full system backup followed by differential backups) to disk or directly to DVD.
My plan is to get a network attached storage device. A RAID configuration is most important so that if a drive fails I wont loose data (again, because the available options to backup to DVD suck). Buffalo Technologies has what I'd like, http://www.buffalotech.com/products/net ... rastation/. The terrastation live supports DLNA (though I dont know what the future of this is with Apple) and iTunes/Bonjour sharing.
Since switching to the Mac I've really been disappointed with the lack of built-in backup capabilities of Mac OS X and the pathetic third party backup and restore options. Nothing provides basic functionality like NTBACKUP does on Windows. I hope someone could correct me on this as I would like to create backup sets (initially a full system backup followed by differential backups) to disk or directly to DVD.
My plan is to get a network attached storage device. A RAID configuration is most important so that if a drive fails I wont loose data (again, because the available options to backup to DVD suck). Buffalo Technologies has what I'd like, http://www.buffalotech.com/products/net ... rastation/. The terrastation live supports DLNA (though I dont know what the future of this is with Apple) and iTunes/Bonjour sharing.
jbrjake: Have you used NTBACKUP (http://www.ntbackup.us), MSBACKUP (built in to 16-bit versions of Windows) or other Windows based backup tools? I don't consider simply copying files a very robust or efficient way to backup files. A backup set (or volume) is generally a single file (compressed archive I suppose) that is much easier to manage than entire directory trees and helps retain integrity. The contents of the backup set are usually indexed and cataloged. Yes, this means that data must be restored before it can be accessed (hence the terms backup and restore). These solutions also tend to support full system backup and restore (sometimes even bare metal restore for disaster recovery purposes by booting from CD first and wiping a drive clean before restoring the system and data). I'm not interested in hacking away at custom scripts and prefer a polished graphical user interface for creating and scheduling backup jobs and performing restores.
I haven't used rsync so maybe I'm misunderstanding what it does.
I'm waiting for time machine in Leopard. but seriously... this is a basic feature that should have been available in the OS already (and not an annual .Mac subscription).
I haven't used rsync so maybe I'm misunderstanding what it does.
I'm waiting for time machine in Leopard. but seriously... this is a basic feature that should have been available in the OS already (and not an annual .Mac subscription).
No, I don't do Windows.Bog wrote:jbrjake: Have you used NTBACKUP (http://www.ntbackup.us), MSBACKUP (built in to 16-bit versions of Windows) or other Windows based backup tools?
rsync is The standard system administration tool for doing backups in the *nix world and it's considered quite robust and exceedingly efficient.
There's a Mac GUI for it if you hate the command line that much.