7 HD's inside a mac pro

[samsc]

These guys put 7 HD's inside a mac pro....

http://www.hardmac.com/news/2006-10-04/#5993

[emilianomorgia]

will help speed to use one Hd per layer ?

let's say we copy the content in 7 hd , and we use 7 layers .

Layer one use content from hd 1
Layer two use content from hd 2

and so one ...

Does this help playback speed ?

thanks

[SourceChild]

:rolleyes: And the ideas Blow Me Away! :rolleyes:




These guys put 7 HD's inside a mac pro....

http://www.hardmac.com/news/2006-10-04/#5993 (http://www.hardmac.com/news/2006-10-04/#5993)



Richard, Emiliano
I was blown away by an Epiphany today and these two posts reflect the results of that.
I have been researching the system configuration shown in the article above and have derived the following results.
First of all, By removing the CD drives and putting in the modified HD Drive Bay, we can add three drives of whatever we want. I did some further investigation, and found that I can assemble a rig where drives can be configured 2 x 2 instead allowing for four total additional drives. There would be a requirement for additional fans but looking at the Optical drive shuttle, I am relatively sure that something could be machined to make a four drive modular SCSI bay with added cooling to make sure that the flow-through into the power supply was sufficient to vent the PS heat as well as the new additional heat from the SCSI disks. I am a bit concerned with the power capacity of the PS but I can always check with Apple. If everything is within acceptable ranges then I would propose the following configuration.
Mac Pro Configuration:

;) Apple Configuration:

· 2 x 3.0GHz Dual Core Xeon
· 4 x 1Gb Memory Simms
· Quadro FX 4500 Video Card
;) Buss Configuration:

· PCI Express Slot 1: Video Card
· PCI Express Slot 2: Atto ExpressPCI UL5D
· PCI Express Slot 3: Atto ExpressPCI UL5D
· PCI Express Slot 4: Blackmagic Decklink HD Pro PCIe
;) New Drive Chassis:

· Slot 1: 146Gb 15k Disk on Controller 1a
· Slot 2: 146Gb 15k Disk on Controller 1b
· Slot 3: 146Gb 15k Disk on Controller 2a
· Slot 4: 146Gb 15k Disk on Controller 2b
;) Existing Drive Chassis:

· Slot 1: 500Gb SATA Disk (dual boot partitions)
· Slot 2: 500Gb SATA Disk (RAID 5 disk 1)
· Slot 3: 500Gb SATA Disk (RAID 5 disk 2)
· Slot 4: 500Gb SATA Disk (RAID 5 disk 3)
If we start with a preconfigured Mac Pro, the I?m estimating only an additional $5000 in hardware for capture card, SCSI cards, and SCSI disks to make this a viable solution. This would then be the HD solution for running 4 layers of 720p HD across two outputs or perhaps 2 Layers of 1080p HD on a single output.
:p
We could then run the widescreen configuration to two projectors for out 720p solution or use a Triple Head 2 Go on a single output to three projectors for our 1080p solution.

[SourceChild]

:cool: Another Epiphany :cool:



...let's say we copy the content in 7 hd , and we use 7 layers .

Layer one use content from hd 1
Layer two use content from hd 2


I am very curious about this

What I normally do:

What I do when I create content that I sync, is that I use the convention of numbering files with the same offset and I skip numbers.
Let's say, I have three screens, I would use the following convention.

o 000 (No File Used)
o 001 File1 CT.mov
o 002 File1 RT.mov
o 003 File1 LF.mov
o 004 File2 CT.mov
o 005 File2 RT.mov
o 006 File2 LF.mov
o 007 File3 CT.mov
o 008 File3 RT.mov
o 009 File3 LF.mov
o 010 (No File Used)
o 011 File4 CT.mov
o 012 File4 LF.mov
o 013 File4 RT.mov

Obviously, when I jump each decade, I would start over again at xx1 and continue to xx3 for the next file until I have saturated the decade at xx9.

What I could do:

Considering this new idea of using different drives, I consider this.
SCSI Disk 1:

001 Left Content Group 1
001 File1 LF.mov
002 File2 LF.mov
003 File3 LF.mov
004 File4 LF.mov
SCSI Disk 2:

002 Left Content Group 2
001 File1 LF.mov
002 File2 LF.mov
003 File3 LF.mov
004 File4 LF.mov
SCSI Disk 3:

003 Right Content Group 1
001 File1 RT.mov
002 File2 RT.mov
003 File3 RT.mov
004 File4 RT.mov
SCSI Disk 4:

004 Right Content Group 2
001 File1 RT.mov
002 File2 RT.mov
003 File3 RT.mov
004 File4 RT.mov

:rolleyes: With this convention, I would name the folder on disk 1 001, disk 2 002, etc so that when the Cat scans, it finds its correct folder structure.

:rolleyes: If you also noticed, this convention would have a duplicate folder of each group of content. The idea I have here is that if I wanted to run two layers of content, I could run file1 on layer1 from disk1 and file2 on layer2 from disk2 and then perhaps crossfade again to layer1 for file3 on from disk1 again.

:rolleyes: This way, no single disk ever has to seek more than one MOV file at a time.

[samsc]

Does not work the way you imagine.

The operating system, and quicktime are 'single threaded', meaning adding more discs doesnt do anything.
Frame access is sequential. Not sumultaneous.

And the overall performance is as slow as the slowest disc -

I have tested this. and continue to test - in case apple changes the way things work.
At the moment - it doesnt.

[SourceChild]

Yes Richard,
I found out more about this as I looked into it. Nevertheless, it would still make a plausible solution for having a SCSI based RAID Stripe set internally. However, I am looking into several different SCSI controllers to see about solutions with larger data buffers.
Compared to an xServe RAID running SATA disks, a SCSI raid will always have faster disk seek times but it's a matter of eliminating the controller bottleneck. Obviously this is the advantage of the xServeRAID but I would prefer to find a way to have an encapsulated single machine solution for doing HD files.

Give me your feedback on the use of internal SCSI Raids and what cards you've used. Have you tried a 4 disk stripe or just a typical 2 disk stripe?
One thought is to use two SCSI controllers that will do hardware striping and then use OSX to perform a software stripe across two controllers.

[samsc]

there is only 1 source of PCIe scsi cards for mac? atto?

you want something that works. get an xserve raid. even with just 4 drives.

messing with scsi raids isnt fun. your thinking is getting too complex. with no performance gain.

[tharding]

The xServe is so low fuss. I have the system that Richard describes and I can run 2 Cats from it without a fibre channel router.

The extra features of the xserve that are worth noting are:

1. Fibre channel is so straight forward. One cable to connect which can run substantial distances if required.

2. Dual redundant power supplies, Cooling systems, Battery backup for cache.

I had a G5 die on me the other day and I was able to swap it out in 10 minutes because all I had to do was swap a card and install the Catalyst software. If I had a machine loaded with drives and cards this wouldn't be so simple.

I think you would find that the XServe is very cost competitive when compared to that much SCSI hardware.

cheers

Toby

[RedLightning]

I am in the process of setting up my system and was looking at the option of 1x250gig system drive 3x500gig 10000 RPM SATA drives using Apple disk utility to set up the RAID.
Primarily a non-HD system but I will be buying the Blackmagic Multibridge Pro...

-s

[tharding]

Have a look at SoftRAID. It comes fairly highly recommended as an alternative to the Apple RAID utility.

Cheers

Toby

[GideonKiers]

Hello there,

I'm working on a theatre project which needs a 18,5 meter panoramic projection. There are quite a few approaches to tackling this challenge. Indeed, most of them highly speculative ...

The plan at the moment is to have 4 projectors, driven by two new 2,66 Intel macs and catalyst v4. Each mac 2 GB (4x512mb) ram and an ati x1900 grphx card.

The resolution on the media files should preferably be 1024x768, but could also be 768x576. Codec would be photo-jpeg at 60 or 70 %.

Setup for each output would be pretty standard, 2 layers of video and possibly a masking (grphcs) layer. Hoping to use the new seamless panorama features in v.4.

• First question : Would a 10k sata disk be fast enough to drive those ± 4 layers of video + 2 masking layers for each machine ?

• Second question : Would it be possible to do the 4 outputs from one computer with the ati card and two dualhead2go's ? It would mean ± 8 layers of 768x576 photojpeg 60% video to four outputs via one machine.

• Third question : In case the dualhead2go solution has a chance of working, would it be possible to use one large (3072x576) quicktime for the four outputs ?

• Last question : Not having to render masks into the quicktime(s) would save us _a lot_ of time. This would mean moving one or multiple masks over the four screens in real-time. Would this be possible in the dualhead2go option ?

That's it for now, am sure there are more issues to it though ;)

Thanks !

Gideon

[samsc]

you want to do these things - you have to do a few tests for yourself.
there are too many variables - and some of the answers you will need to find for yourself.

see what works and what doesnt.

do you have a raptor to test?

[tharding]

"• First question : Would a 10k sata disk be fast enough to drive those ± 4 layers of video + 2 masking layers for each machine ?"

Don't forget that if you are using dual outputs as a opposed to the panorama option that "Use Layer" is the most efficient way to go.

That means that if you want the same clip on Output 2 as Output 1 you can "Use Layer" the number of the layer you are playing the clip back on for Output 1. (Hope that made sense!)

This should cut down your disk overheads considerably if you only need 2 layers of video and 1 x masking layer.

In my experience you would struggle getting 4 layers smooth of one Raptor.

Cheers

Toby

[samsc]

In my experience you would struggle getting 4 layers smooth of one Raptor.

Cheers

Toby

mine too.

but sometimes its best for people to find out for themselves.

[GideonKiers]

mm no let me make myself a bit clearer, 4 seperate outputs should make one big panorama. each output therefore would be unique.


That means that if you want the same clip on Output 2 as Output 1 you can "Use Layer" the number of the layer you are playing the clip back on for Output 1. (Hope that made sense!)

[GideonKiers]

fair enough, but unfortunately as usual there's no time or money to test things out really..

so let me rephrase my question. what (non-scsi and non-fibrechannel) harddrive configuration would advisable for the smooth playback of 4 streams of 768 576 photojpeg non-100% clips, from a macpro 2,66 with an ati x1600 card on two seperate outputs, each in a 1024 768 resolution ?


sometimes its best for people to find out for themselves.

[tharding]

Yes but you use one part of the same image do you not? That is how Watchout and other systems work. all you are seeing is a different section of the same image or clip.

When you use the new panorama functions you are essentially doing this. Your mask for instance would be a single file across 4 screens not 4 separate files.

I have a 4 x Raptor RAID that can give me 4 layers of DV but not 1024 x 768 Photo Jpeg without some dropped frames. As Richard says, you really have to try it. I have an Xserve RAID but I still have to check every combination I come up with.

Cheers

Toby

[GideonKiers]

Absolutely yes, each of the 4 projectors does 1/4th of the whole image.

I'm thinking of doing 768x576 photojpeg files, not 1024x768.

The advantage of doing 4 projectors from one machine is that you're working on the whole 'canvas' from one machine, therefore moving a mask around the whole canvas is much easier than if we would do it on two machines.

Maybe I'm misunderstanding you, but I don't really see why 4 layers of DV playback needs a 4 x Raptor RAID though ? I have an old 3.2 catalyst system here that plays 4 dv layers from one 7200 scsi disk without really any problems on a dual 2.5 ppc.

I find it a bit hard to believe that there's no other solution to my harddrive speed problems then buying the xserve raid ...
How about the Fujitsu MAU3147NP (Ultra 320) and an Atto PCI UL5D pci-e card. Would that be sufficient (theoretically) to drive 4 layers of either DV-PAL or 768x576 photjpeg ?


Yes but you use one part of the same image do you not? That is how Watchout and other systems work. all you are seeing is a different section of the same image or clip.

When you use the new panorama functions you are essentially doing this. Your mask for instance would be a single file across 4 screens not 4 separate files.

I have a 4 x Raptor RAID that can give me 4 layers of DV but not 1024 x 768 Photo Jpeg without some dropped frames. As Richard says, you really have to try it. I have an Xserve RAID but I still have to check every combination I come up with.

Cheers

Toby

[samsc]

4 layers of DV playback needs a 4 x Raptor RAID though

please - you need to test with YOUR content - or something like your content - as photojpeg has different performance for each file and different compression settings.
its quite variable. it is very difficult to give general performance guidlines outside dv files - as every file is different.

the performance of dv files is constant because they have a fixed bit rate.

start with something minimal - say a single raptor - no need to do any kindof raid to start with- and see what happens.
tell me what you see. please.

typically - you will not see any performance difference at standard definition - with your files at 768x576 between a single disc -and a raid system.
raid systems are only important when the data rate is the limiting factor - in uncompressed movies.

[samsc]

I find it a bit hard to believe that there's no other solution to my harddrive speed problems then buying the xserve raid ...
How about the Fujitsu MAU3147NP (Ultra 320) and an Atto PCI UL5D pci-e card. Would that be sufficient (theoretically) to drive 4 layers of either DV-PAL or 768x576 photjpeg ?

you dont have to buy an expensive system - you just need to test your files on your system - and see what happens.

photojpeg performance is very different from dv performance.

[GideonKiers]

ok very clear indeed. problem is the system still needs to be bought, so all of this is pure speculative, i'm trying to find out which system to buy.

There is no real need to do photojpeg though, except for quality maybe .. I could also do square pixel dv pal for instance, if that would make the decision for a harddrive system easier (and less expensive ;)). btw: Does catalystv.4 turn the highquality tab on dv quicktimes on automagically, or do i need to save that with the quicktime ?

It's hard to say now what the content will be exactly, since i'm working on a theatre play in which the content is allways changing, with the development of the play. But i guess there would for instance be quite a lot of brightness in the images, and quite a bit of slow moving images over long timespans. Then of course when this production is over, another will be put on stage, might be very different material. So basically the content could be anything.

Has anyone here installed the atto + scsi disk option i pointed out before in a new mac pro yet ?

Thanks for all the help !


you dont have to buy an expensive system - you just need to test your files on your system - and see what happens.

photojpeg performance is very different from dv performance.

[samsc]

what do you have at the moment?

---

scsi and atto seem to work fine in intel mac pro so far.
the drives are easy to fit in optical drive bay.

[GideonKiers]

At the moment we have dual 2.5 ppc late 2004 models.
http://support.apple.com/specs/powermac/Power_Mac_G5_Late_2004.html

One runs pro, the other xpress. The pro has an Atto card and a scsi drive inside, not exactly sure which model. Will have a look later today.


what do you have at the moment?

[samsc]

which scsi drive?

[GideonKiers]

Sorry, haven't been able to check. Hope to do that tomorrow. In the mean time i've had a look at the caldigit drives + cards. The S2VRDuo looks quite promising ? http://www.caldigit.com/S2VRDuo.asp

Should be able to play 4 layers of DV-PAL ? Try to keep the disc tidy to limit seek times ..

Have to send in the specs for the machines they need to buy by the end of this weekend..


which scsi drive?

[GideonKiers]

Hello,

What would be a recommended os x disk speed testing utility, preferably downloadable for free ?

thanks,

[RedLightning]

i used the one included with my Blackmagic card...

-s

[samsc]

i used the one included with my Blackmagic card...

-s

why doesnt that give you any indication of performance?

because this only reads and writes a single large file.
disc performance changes drastically depending on how many files are being read at the same time. not just on how many MB/s it can read.


It gives you no indication of how well it works with catalyst.
Catalyst reads more than one file at the time - it reads one file for each movie playing.

the way to benchmark catalyst is with catalyst - nothing else.

Catalyst is also free when used like this.

Headline decklink blackmagic data transfer rates are no use to you - unless you are capturing a single movie file and writing it to disk.