Skip to content

[Whisper] Audio format errors on valid file #333

Closed
Closed
[Whisper] Audio format errors on valid file#333
Assignees
mpokrass
Labels
bugSomething isn't working

Description

@muddi900
muddi900
opened on Mar 22, 2023

Describe the bug

Hello

I am trying to integrate the whisper API into my Flask app. However I get the following error when I input the received file from the flask endpoint, I get the following error:

openai.error.InvalidRequestError: Invalid file format. Supported formats: ['m4a', 'mp3', 'webm', 'mp4', 'mpga', 'wav', 'mpeg']

However, loading the file in the interactive console works fine.

In [16]: r = openai.Audio.transcribe('whisper-1',open('../Downloads/sample.mp3','rb'))

In [17]: r
Out[17]:
<OpenAIObject at 0x192993c6750> JSON: {
  "text": "This episode is actually a co-production with another podcast called Digital Folklore, which is hosted by Mason Amadeus and Perry Carpenter. We've been doing a lot of our research together and our brainstorming sessions have been so thought-provoking, I wanted to bring them on so we could discuss the genre of analog horror together. So, why don't you guys introduce yourselves so we know who's who? Yeah, this is Perry Carpenter and I'm one of the hosts of Digital Folklore. And I'm Mason Amadeus and I'm the other host of Digital Folklore. And tell me, what is Digital Folklore? Yeah, so Digital Folklore is the evolution of folklore, you know, the way that we typically think about it. And folklore really is the product of basically anything that humans create that doesn't have a centralized canon. But when we talk about digital folklore, we're talking about..."
}

To Reproduce

  1. Create a Flask App.
  2. Add an end point that receives an valid audio file.
  3. pass the bytes data of the file to openai.Audio.transcribe method through 'request.files[fileName].stream.read()`.

Code snippets

The end point code:


with tempfile.TemporaryFile() as temp_file:
    temp_file.write(audio_file)
    transcript_read = openai.Audio.transcribe("whisper-1", temp_file)
return transcript_read

the FFprobe info of the file:

ffprobe version 4.4.1-full_build-www.gyan.dev Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 11.2.0 (Rev1, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mp3, from '.\Downloads\sample.mp3':
  Metadata:
    title           : Monsters in the Static
    comment         : We look at the subgenre of analog horror, where something sinister might be lurking in the horizontal lines and vertical holds of those old VHS tapes.
    lyrics-ENG      : <p>In the subgenre of analog horror, there’s something sinister or supernatural lurking in the horizontal lines and vertical holds in those old VHS tapes. Filmmaker <a href="https://wnuf.bigcartel.com/">Chris LaMartina</a> explains why he wanted his mov
    album           : Imaginary Worlds
    genre           : Podcast
    date            : 2020
    encoder         : Lavf58.76.100
  Duration: 00:00:50.05, start: 0.025057, bitrate: 128 kb/s
  Stream #0:0: Audio: mp3, 44100 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc58.13

OS

Windows 11

Python version

Python v10.5

Library version

0.27.2

Activity

  1. muddi900
    added
    bugSomething isn't working
    on Mar 22, 2023

  2. sumeyyeyegen commented on Mar 24, 2023

    @sumeyyeyegen
    sumeyyeyegen
    on Mar 24, 2023

    were you able to find a solution? i am getting the same error. it works very well in jupyter notebook app. but I keep getting this error in the hugging face application.

  3. muddi900 commented on Mar 24, 2023

    @muddi900
    muddi900
    on Mar 24, 2023 via email
    Author

  4. IsaacKnowles commented on Mar 27, 2023

    @IsaacKnowles
    IsaacKnowles
    on Mar 27, 2023

    I've run into this today as well, but with the webm audio format, also using Flask. A web app is recording a brief spoken audio clip with mimeType 'audio/webm;codecs=opus' and when I save a copy of the recorded audio to a file, it is submitted to the API and correctly transcribed without issue. However, if I submit the request via my Flask app, I get the same error as @muddi900.

    ffprobe info:

    ffprobe version 5.1.2 Copyright (c) 2007-2022 the FFmpeg developers
  built with Apple clang version 14.0.0 (clang-1400.0.29.202)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/5.1.2_6 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, matroska,webm, from 'sample.webm':
  Metadata:
    encoder         : Chrome
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)

    OS: Mac 12.6.3
    Python version: 3.10.1
    openai version: 0.27.2

  5. danielfaust commented on Mar 28, 2023

    @danielfaust
    danielfaust
    on Mar 28, 2023

    I was getting this error on some files, and while looking at them, I noticed that they are lacking a proper mp3 header. My solution was to let them run through ffmpeg once before uploading them, by using the acodec='copy' parameter so that the actual audio content of the mp3 file does not get modified:

    ffmpeg \
    .input(path) \
    .output('_temp.mp3', acodec='copy') \
    .overwrite_output() \
    .run()

    and then I'm uploading _temp.mp3 instead of path.

  6. muddi900 commented on Mar 28, 2023

    @muddi900
    muddi900
    on Mar 28, 2023 via email
    Author
  7. hallacy
    assigned
    mpokrass
    on Apr 6, 2023

  8. sparkle666 commented on Apr 13, 2023

    @sparkle666
    sparkle666
    on Apr 13, 2023

    Also got same error when loading audio files locally.

    audio2 = open("Greeti.mp3", "rb")                   
sub = openai.Audio.transcribe("whisper-1", audio2, response_format = "text")

    'Greetings.\n'

    Changing the response formats to a different string returns errors.

    sub = openai.Audio.transcribe("whisper-1", audio2, response_format = "srt")

    openai.error.InvalidRequestError: Invalid file format. Supported formats: ['m4a', 'mp3', 'webm', 'mp4', 'mpga', 'wav', 'mpeg']

  9. grp06 commented on Apr 14, 2023

    @grp06
    grp06
    on Apr 14, 2023

    I fought with this for a long time. Finally got it working by not using the MediaRecorder() API on the frontend. I switched to using

      const startRecording = () => {
    setIsRecording(true)
    navigator.mediaDevices.getUserMedia({ audio: true }).then((stream) => {
      const options = {
        type: 'audio',
        mimeType: 'audio/mp3',
        numberOfAudioChannels: 1,
        recorderType: RecordRTC.StereoAudioRecorder,
        checkForInactiveTracks: true,
        timeSlice: 5000,
        ondataavailable: (blob) => {
          socket.emit('audio', { buffer: blob })
        },
      }

      const recordRTC = new RecordRTC(stream, options)
      setRecorder(recordRTC)
      recordRTC.startRecording()
    })
  }

    and it worked immediately.

  10. zieen commented on Apr 21, 2023

    @zieen
    zieen
    on Apr 21, 2023

    how to dell it in server side ? it complain about the missing name of file?

  11. amoschoomy commented on May 1, 2023

    @amoschoomy
    amoschoomy
    on May 1, 2023

    any updates on this? the request fails inside an endpoint, but works when on local files

  12. calummackervoy commented on May 10, 2023

    @calummackervoy
    calummackervoy
    on May 10, 2023

    I have a variation on the solution using RecordRTC which was posted above. It shows how to use start/stop, reset the audio channel, send with Ajax request (multi-part form data)

    if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
        navigator.mediaDevices.getUserMedia({ audio: true })
        .then((stream) => {
            const options = {
                type: 'audio',
                mimeType: 'audio/mp3',
                numberOfAudioChannels: 1,
                recorderType: RecordRTC.StereoAudioRecorder
            }
 
            const recordRTC = new RecordRTC(stream, options);

            $(document).on("click", "#record-button", () => {
                let recordButton = $("#record-button");
                // already recording, hit stop
                if(recordButton.attr("recording") === "true") {
                    recordButton.attr("recording", false)
                    recordButton.html("REC");

                    recordRTC.stopRecording(async () => {
                        let blob = await recordRTC.getBlob();
                        var form = new FormData();
                        form.append("file", blob);
                        $.ajax({
                            type: "POST",
                            data: form,
                            url: "",
                            processData: false,
                            contentType: false,
                            success: function (data) {
                                // ...
                                recordRTC.reset();
                            },
                            error: (err) => {
                                // ...
                                recordRTC.reset();
                            }
                        });
                    });
                }
                // not recording, hit play
                else {
                    //mediaRecorder.start();
                    recordButton.attr("recording", true);
                    recordButton.html("STOP");

                    recordRTC.startRecording();
                }
            });
        })
        // Error callback
        .catch((err) => {
            console.error(`The following getUserMedia error occurred: ${err}`);
        });
}

  13. odusseys commented on Aug 3, 2023

    @odusseys
    odusseys
    on Aug 3, 2023

    Having the same issue with a mp3 file, written by ffmpeg, lame mp3

  14. cquintero4told commented on Aug 4, 2023

    @cquintero4told
    cquintero4told
    on Aug 4, 2023

    Try with this code:

    with tempfile.NamedTemporaryFile(suffix='.mp3') as temp_file:
    temp_file.write(audio_file)
    temp_file.flush()
    temp_file.seek(0)

    transcript_read = openai.Audio.transcribe("whisper-1", temp_file)

  15. ilyak1990 commented on Sep 2, 2023

    @ilyak1990
    ilyak1990
    on Sep 2, 2023

    For me specifically it was on iPhone, I was saving a valid .wav file (was working when I tested it) then I used a file type detector tool to find out it was actually some other file format that apple was saving it to, you can either convert to and from file types using node library ffmpeg or for iphone specifically save it as a .m4a file instead of .wav

  16. andora2 commented on Sep 11, 2023

    @andora2
    andora2
    on Sep 11, 2023

    was fighting with similar problems.
    Works currently on iPhone 13(iOS 17) with MediaRecorder,

    Client:

    ...
const recorder = new MediaRecorder(stream, { mimeType: 'audio/mp4' });
...

    Server:

    ...
const formData = new FormData();
formData.append('file', buffer, { filename: "audio.mp4", contentType:  "audio/mp4" });
...
const response = await axios.post('https://api.openai.com/v1/audio/transcriptions', formData, {
    headers: {
        'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
        ...formData.getHeaders(),
    },
});

    For Chrome & co. I go with 'audio/webm' on client and 'audio/mp3' on server.

    Don't really expect this to be bulletproof, but for the time being its stable enough for my needs.

    BR,
    Adrian

  17. thiswillbeyourgithub commented on Nov 8, 2023

    @thiswillbeyourgithub
    thiswillbeyourgithub
    on Nov 8, 2023
    Contributor

    Hi, I think I found a viable workaround. The issue seems to be in the way the BufferReader is reading files. Bypassing the bufferreader fixed it for me.

    Can't investigate more because of the outtage unfortunately. Might be related to #727

    In this line if instead of file=buffer_reader I put file=open(args.file, "rb") the function returns normally instead of 400 error.

    edit: found a fix and submitted a PR in #733

  18. rattrayalex commented on Nov 10, 2023

    @rattrayalex
    rattrayalex
    on Nov 10, 2023
    Contributor

    OpenAI audio endpoints generally require a filename with extension be provided in the upload, which is used to determine the file type.

    This is made more convenient with the new v1 of the SDK, where you can pass a pathlib.Path to the API:

    from pathlib import Path
from openai import OpenAI

openai = OpenAI()

speech_file_path = Path(__file__).parent / "Downloads" / "sample.mp3"


openai.audio.transcriptions.create(model='whisper-1', file=file)
  19. rattrayalex
    closed this as completedon Nov 10, 2023
  20. safa0
    added a commit that references this issue on Apr 27, 2025

    Fix parallel_tool_calls when False (openai#333)

    923a354
  21. karpetrosyan
    added a commit that references this issue on Jul 9, 2025

    revert custom code applied to a generated branch (openai#333)

    ac3a4cf
  22. wagner-austin
    added a commit that references this issue on Jan 13, 2026

    Use MP3 output format for audio chunks to fix OpenAI compatibility

    594c12c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions