Five months after ATi promised this in Catalyst 10.4:

H.264 Level 5.1 support
ATI Catalyst can now playback 5.1 level streams on all support players

My test file, a 1920x1080 L5.0 and 9 ref frames

Still doesn't work.