I must say that's a bit surprising, usually it's the other way around.
Anyway, the first piece of advice I can give you is that most of the times learners don't develop their listening skills by doing listening exercises. They do so by listening to you when you're explaining, presenting, or giving feedback. With this rationale in mind, a couple of simple things like encouraging note taking and pausing frequently to ask questions (and from there providing prompt feeedback) can go a long way.
I think it's important to make listening tasks less intimidating.
I'd firstly suggest putting more effort into pre-listening activities: activating learners' background knowledge of the topic, asking them to predict content, pre-teaching some vocabulary. Secondly, you can divide the audio recording into smaller pieces and adjust the tasks so that they start with an easier one, slowly building up the level of difficulty.