Video playback with Audio. Sync error?

Hi, I am new to using catalyst with audio. Got a show coming up in four weeks time and using catalyst to playback audio and video triggered from MagicQ PC.

I was advised to play the audio back on a new layer and ether use sync id or trigger both the layers at the same time.

In testing today i found that:
Triggering the layers together made the video out of sync with the audio by the time the clip completed (only about 12frames, but noticeable).
-
Syncing the layers by id kept the video and audio in sync but the layers hud had some funny jumping going on. I dont think this upset the video performance.

Videos of 'jumping'
http://www.youtube.com/watch?v=R7skttA_AgA
http://www.youtube.com/watch?v=8vylwZCvBCc
-
Finally!
Playing the video with audio mode 70 again kept both video and audio in sync. I did however loose the FPS reading on the layer window.


Please could you advise me on the best option to playback video and audio using catalyst.

Video files are 820x1024 AIC 25FPS. Audio: Linear PCM 48Hz, 16-Bit.
Edited in FCP and exported using QT.

Catalyst server:
OSX 10.6.6 / 4.40_m329 / DV dongle
MacPro 4,1 2.26 8 Core / 6GB / ATI HD 4870 / OCZ-Vertex SSD (playback)

Thank you for helping!!
P

Dont know who advised playback separately - but using playmode 70-76 onwards is probably the simplest way of playing a movie with audio.

The FPS counter will disappear, because catalyst prioritises audio playback when using these playmodes (70-76) - i.e. it plays back at the same frame rate as the clip with audio was encoded - no matter what your system frame rate is set to. This is to ensure that the audio playback is flawless and sounds like it was intended.

So the best way to playback audio with a video file is to encode both together as one movie (eg AIC codec) and use playmode 70-76


However - you should also check the spec of your graphics card and read this post:

samsc.com/showthread.php?t=1908

ATI 4870 / 5770 / 5870 have driver issues is 10.6.6/10.6.7 - and you get very poor performance when using catalyst.
You should change your graphics card to an Nvidia card which does not have the same problems.
The Nvidia Quadro 4000 is the best performing official card available if using Snow Leopard (as of May 2011)

Do get back to me if you need more help

Simon

Cheers Simon,
Will continue testing in that play mode and look into the graphics card situation. 10.6.6 is running on my server (snug in one of your cases too!!) don't know if this problem still exists?

yes - problem re graphics cards still exists....

Strongly suggest changing the graphics card...
To give an example: ATI card tests only resulted in ONE layer of HD content playback, compared to Nvidia Quadro 4000 with 8 layers of 1920x1080 25fps (AIC) playback...

The ATI drivers are broken in 10.6.x

(May 2011)

Quote:

---

Originally Posted by pip

I was advised to play the audio back on a new layer and ether use sync id or trigger both the layers at the same time.

---

I'm not sure who advised you of this but there is only one advantage to placing audio on a separate file. That reason is to be able to adjust the inframe or outframe amounts of the video file to adjust the offset of frames between layers.

This is an advanced technique that I have suggested to people in the past as a workaround but this is not a solution that someone should use when creating new shows.

If you must play audio with video, follow Simon's statement and render them together as one file, verifying they are synced in post before loading them to the Catalyst.

Just found that running the audio from the same layer causes the playback to jitter and a 'spike' in the mov 1 play reading in the stats hud...

Attached video: http://www.youtube.com/watch?v=wkmajfu0wdA

This does not happen when using an non-audio play mode. I take it this could be a graphics card problem? Will look at replacing the cards tomorrow.

possibly - but what are the specs of your movie?
what codec and data rate - this could also be a factor?

s

Just a note about the FPS counter and audio playback....

I am currently doing a show through South America and we are running content that is 29.97.

Yes, I am running content on Catalyst and no, I did not want 29.97 but we are also using the content on playback decks as well so 30fps was not an option.

The content has to run frame accurate with the timecode of the show because the content was created exactly to the music of the set. In fact, we placed the music on the content as reference.

Since the show timecode exceeds the total allowable length of time that a Catalyst frame counter can recognize, I have had to program the presets to be triggered from the SAM cuelist. Basically meaning the cuelist starts the file and then it free runs until the next SAM cue fades it out.

I know this is a bit of a tangent but my point is simple.

In order to make the files play frame accurate (roughly) to the timecode, I needed to use the audio playback mode because it will use the audio as a time base reference for playing the files.

Yes, I drop frames this way and my whole team is aware of it. However, when the vocalist is shown in the content, lip syncing the same lyrics she's actually singing then the audience is more likely to notice voice slip than a few missed frames.

Not the correct way to do things, but since day to day we might have 50hz or 60hz, power and PAL or NTSC, it's the only way to cover all bases.

Take these thoughts as insight but try not to use this method if a better alternative is available.

Thank you both for your comments. In reply to Simon the mov specs are as attached...

Ok, quick update. I have now installed Catalyst 4.40 onto two of my MacPro and my office i7 MacBook Pro and tried each with the file i am experiencing problems with.

When playing the file in mode 70, all the macs play with a stutter so this narrows it down i guess to the way the video has been rendered in post..?

Or i have three bad macs! :confused:

try this - rerender the clip to 832 x 1024 and let me know if that helps


You may find that as the horizontal pixel count is not a multiple of 16 - the content may run like a dog.

Nev.

Quote:

---

Originally Posted by NevBull

try this - rerender the clip to 832 x 1024 and let me know if that helps

Nev.

---

ah, good eye Nev.

Thanks Nev, that stopped the clip from dropping frames every second or so as before. Runs much more smoothly.

The only thing it still does, which i don't understand the stats window, there is still a frame or so dropped when this pink bar (in image below) jumps from top image to bottom image. The bar then slowly decreases and then jumps back up every 10-15sec. On doing so the video glitches.

do you have any video inputs enabled? I have seen similar issues when several inputs where enabled... interestingly it seems to be a Snow Leopard thing...

Okay, I have a few thoughts on this subject. I have had these problems on many occasions and have yet to find a solid solution for it.

Here is what I understand:

1. Running the video with an audio play mode when they are merged together in the same file can cause playback to be less than smooth. This is because as Todd mentions, the playback rate is driven by the audio portion of the file for sync, not the video. The Mac OS will always treat the audio as a priority.

2. Running the audio and video on separate files on different layers will potentially yield the best result for smooth video playback but then this is where we run into the "lip sync" problem. I typically run the audio from a different server dedicated just for that purpose.

Here is what I have discovered since:

1. Running the audio and video independent of each other works wonderfully if the media was created at a specific frame rate and your display is running perfectly at that rate. ie: When I have the media at 30fps and connected to monitors that are at a perfect 60Hz refresh rate, I can achieve perfect sync.

2. 60Hz is not always that. On most systems (Spider, Encore, Etc.) or even large displays and projectors, the true refresh rate will be 59.94. Even when your display settings are set to 60Hz in both the system settings as well as Catalyst, your output frequency will be adjusted to that of the display your connected to thanks to EDID management through DVI. This is easily verifiable when you look in the Stats window in the HUD.

Here is my plan to fix the problem next time around:

1. Use a DVI detective to capture the EDID from the monitors that work and force the video card to stay at those settings when connected to other systems.

2. Continue to run audio and video independently so that the video playback will be as smooth as possible.

3. Be more cognizant of the refresh rates that the Catalyst is capable of.

The playback pro doesn't seem to have this problem. I am really curious to know what they do to correct this and provide solid, clean playback. That is a Mac based system too for anyone that doesn't already know and is responsible for taking over when Catalyst has failed me in the past with this.

There are a couple of products on the market that are actually designed to deal specifically with this problem. The DVI parrot is one that comes to mind that is designed to "prevent display misunderstandings" as their literature says.

Was up last night testing different mov files (photo jpeg, AIC... 25/30fps) on different versions of catalyst (4.10 - 4.40) running 10.5.8 + 10.6.7! Then got some sleep.

Im still finding dropped frames when using the audio play mode as mentioned in this thread. Looking at the stats its the middle 'pink' bar on the layers play graph that spikes (as mention before) causes the dropped frames and stutter.. be good to learn what this represents?

I also split video/audio on another layer and

• a) triggered both play setting at the same time... video was one second behind by the end of the clip
• b) synced them by local ID
• c) ran Catalyst on another MacPro (no dongle) running just the audio and remote synced the two together. MacPro with the DV dongle was the master.

I did notice the screens i have been using are fixed to 60Hz and the cinema display 59.XX.

Will be using a TH2G with LX41 and Lighthouse R7 LED come the show in a weeks time!

Not looked into Playback Pro yet, will do, but did run the 25/30FPS versions on Qlab and that showed no problem. Least i have a backup, but need the functions of Catalyst.

If anyone wants to see if its the content i personally am using... i have cropped a section if it would help??

http://bradeng.com/tmp/001%20Test%20Cropped.zip

Screen recording... Note at 15FPS!

http://bradeng.com/tmp/scr_rec.mov

Quote:

---

Originally Posted by pip

Just found that running the audio from the same layer causes the playback to jitter and a 'spike' in the mov 1 play reading in the stats hud...

Attached video: http://www.youtube.com/watch?v=wkmajfu0wdA

This does not happen when using an non-audio play mode. I take it this could be a graphics card problem? Will look at replacing the cards tomorrow.

---

'spiking' is normal - this is audio being buffered

what screens and what screen sizes?
what catalyst playback frame rate?
screen 1 div 2? screen 2 div 2

-

'spiking' is normal

the usual problem is not drop frames at all - but screen refresh rates that do not match the audio/video frame rates
the 'visual' stutter isnt what people think - there isnt any stuttering - the screen just isnt displaying the image because the movie frames display times dont match screen refresh times

when doing audio - you have to work differently -

the frame rate of movie needs to match frame rate of screens -

triple heads are particularily obnoxious for this

-

screens dont run at 60Hz or exactly 75Hz -

Quote:

---

Originally Posted by pip

Was up last night testing different mov files (photo jpeg, AIC... 25/30fps) on different versions of catalyst (4.10 - 4.40) running 10.5.8 + 10.6.7! Then got some sleep.

Im still finding dropped frames when using the audio play mode as mentioned in this thread. Looking at the stats its the middle 'pink' bar on the layers play graph that spikes (as mention before) causes the dropped frames and stutter.. be good to learn what this represents?

I also split video/audio on another layer and

• a) triggered both play setting at the same time... video was one second behind by the end of the clip
• b) synced them by local ID
• c) ran Catalyst on another MacPro (no dongle) running just the audio and remote synced the two together. MacPro with the DV dongle was the master.

I did notice the screens i have been using are fixed to 60Hz and the cinema display 59.XX.

Will be using a TH2G with LX41 and Lighthouse R7 LED come the show in a weeks time!

Not looked into Playback Pro yet, will do, but did run the 25/30FPS versions on Qlab and that showed no problem. Least i have a backup, but need the functions of Catalyst.

---

the solution to perfect audio and video requires

a very different approach - audio takes total priority- not video

you run a single layer only - and run the catalyst video frame playback rate at 60fps or 75fps - not 25 or 30fps - see attachment
( lock to screen 1 divide 1 or lock to screen 2 divide 1 )

dont use triple heads -

only use a single output from computer

never have mismatched monitor sizes on different outputs

-

i think you will find that playback pro is basically - 1 or 2 layers - with only 1 output
not triple heads- not 32 layers - not mixed screen outputs etc
not all the fx -

Quote:

---

Originally Posted by SvenCRen

Okay, I have a few thoughts on this subject. I have had these problems on many occasions and have yet to find a solid solution for it.

Here is what I understand:

1. Running the video with an audio play mode when they are merged together in the same file can cause playback to be less than smooth. This is because as Todd mentions, the playback rate is driven by the audio portion of the file for sync, not the video. The Mac OS will always treat the audio as a priority.

2. Running the audio and video on separate files on different layers will potentially yield the best result for smooth video playback but then this is where we run into the "lip sync" problem. I typically run the audio from a different server dedicated just for that purpose.

Here is what I have discovered since:

1. Running the audio and video independent of each other works wonderfully if the media was created at a specific frame rate and your display is running perfectly at that rate. ie: When I have the media at 30fps and connected to monitors that are at a perfect 60Hz refresh rate, I can achieve perfect sync.

2. 60Hz is not always that. On most systems (Spider, Encore, Etc.) or even large displays and projectors, the true refresh rate will be 59.94. Even when your display settings are set to 60Hz in both the system settings as well as Catalyst, your output frequency will be adjusted to that of the display your connected to thanks to EDID management through DVI. This is easily verifiable when you look in the Stats window in the HUD.

Here is my plan to fix the problem next time around:

1. Use a DVI detective to capture the EDID from the monitors that work and force the video card to stay at those settings when connected to other systems.

2. Continue to run audio and video independently so that the video playback will be as smooth as possible.

3. Be more cognizant of the refresh rates that the Catalyst is capable of.

The playback pro doesn't seem to have this problem. I am really curious to know what they do to correct this and provide solid, clean playback. That is a Mac based system too for anyone that doesn't already know and is responsible for taking over when Catalyst has failed me in the past with this.

There are a couple of products on the market that are actually designed to deal specifically with this problem. The DVI parrot is one that comes to mind that is designed to "prevent display misunderstandings" as their literature says.

---

Ah ha! Ok used the divide settings as mentions above and that is a fantastic improvement! :)

Richard, if you don't mind explaining: if you use a frame playback rate of 30fps, monitors 60Hz, and use the 1/1 setting (60fps) do i then need to re render the content to 60fps? Sorry I don't really understand what the 1/1 or 2/1 do.

Same for 25fps..
Set monitors to 50/75Hz and use setting?

Despite never seeing the stat hud move so fast there where no drop frames when quickly testing this morning. Will continue testing.

For the show i was going to use the TH2G outputting two 800x600 windows (one to ImagePro/R7 LED, other to LX41). If this is not a good idea then will look at one output and two ImagePros for locating the content.

Thank you! :)

you dont rerender - playback movie rate stays the same

but it does run my playback engine at 60fps or 50 or 75fps - which means the threshold for overloading graphics card is much lower- much less time to decode and render things on screen

so you have to reduce number of layers by at least half - on older graphics cards - it doesnt work so well at all.

output screen video timings are critical with audio

dont mix screen output sizes

-

if you want no drop frames playing audio - the frame rate at all stages of playback and screens and output screens needs to match exactly- not just on mac but in entire video system.
not 50/60 exact - but 59.94Hz 60.02Hz 75.03Hz exact

using well defined PAL -720p and 1080p frame rates - NOT vga or dvi frame rates or triple heads - is usually best - vga/dvi timing are off by quite a lot for audio. ( 1/1000)

800x600 is ok at 75Hz - but not through a triple head

Quote:

---

Originally Posted by pip

Ah ha! Ok used the divide settings as mentions above and that is a fantastic improvement! :)

Richard, if you don't mind explaining: if you use a frame playback rate of 30fps, monitors 60Hz, and use the 1/1 setting (60fps) do i then need to re render the content to 60fps? Sorry I don't really understand what the 1/1 or 2/1 do.

Same for 25fps..
Set monitors to 50/75Hz and use setting?

Despite never seeing the stat hud move so fast there where no drop frames when quickly testing this morning. Will continue testing.

For the show i was going to use the TH2G outputting two 800x600 windows (one to ImagePro/R7 LED, other to LX41). If this is not a good idea then will look at one output and two ImagePros for locating the content.

Thank you! :)

---

the reason why these things happen - is that when play audio is used-
the real movie frame rate is used-

the real movie frame rate might be exactly 30fps or 25fps -

DOES NOT match the vga or dvi timings-

vga/dvi timings are different at different screen sizes-

they are 60.02 or 75.03 or 59.94 - very rarely exactly 75Hz or 60Hz

so your movie is playing back NOT locked to the screen outputs-

There arent any drop frames - or any stuttering - the frames are not being output locked to the screen output rate - so at certain times - usually every 30 secs dont have any chance to get displayed- as the movie playback goes out of sync with the screen rate


you want to do audio and dropped frames are critical-

you need to use single or dual output - at exact same size-
do not use triple heads-
do not mix monitor resolutions-

use 1920x1080 or 1280x720 or 800x600 similar-

run catalyst playback at 60fps or 50fps or 75fps -

this give more of a chance for frames which dont line up with the screen refresh rate to get displayed.

Richard - a question:
In a show scenario with some content with audio and some without

- How do you change the playback from 'lock to screen div 1' back to a rate that works for the rest of the non-audio content using a non-audio playmode?

Do you have to go and change it in the catalyst control window or via the web server?
or
should you just use the speed control to slow playback of the non-audio files?

S

Quote:

---

Originally Posted by Mr_P

Richard - a question:
In a show scenario with some content with audio and some without

- How do you change the playback from 'lock to screen div 1' back to a rate that works for the rest of the non-audio content using a non-audio playmode?

Do you have to go and change it in the catalyst control window or via the web server?
or
should you just use the speed control to slow playback of the non-audio files?

S

---

look in the safari activity log when you access the pull down menu in cat web server-

changes the frame rate via web commands
http://xxxyourcomputer.local.:8080/c...=61,0,500,31,0
http://xxxyourcomputer.local.:8080/c...=61,0,500,32,0
...

-

the performance is far lower at 60Hz- its really only for emergencies when audio syncing is THE most important thing- or you have one or two layers
normal shows will very rarely run well at 60Hz - if at all.