Forum rules
An Activity Log is required for support requests. Please read How-to get an activity log? for details on how and why this should be provided.
Please replace this text with the contents of your log file between the two code tags - OR - provide a pastebin URL in place of these 3 lines.
If you are unable to do so, please state why so we can help you.
Handbrake is designed for converting video, and does audio because it is found in video tracks. So no, it doesn't "just do audio".
There are several open source projects that can do "just audio", including ffmpeg. They're better fits for what you are looking for, because they can concentrate on just the audio.