I am having exact issues as well. I have tried multiple distros including Ubuntu, Fedora, Debian and NixOS and the behaviour is same.

Basically, the sound stutters in the beginning for about 5-30 seconds and eventually starts playing normally. If you pause an audio/video for over 10 seconds and you play again, you get same behaviour.

The mic behaves same way and it becomes pretty much useless.

I do however notice that the audio works fine in some distros with older kernels (like Ubuntu 22.04). However, GPU acceleration is missing in those distros.

Considering that USB cameras work fine, I suspect that USB headset/speakers/mic might work.

It has been a blocker for me and I hope there are some workarounds.