Windows Media Foundation Recording Audio

Windows Media Foundation recording audio

I apologize for the late response, and I hope you can still find this valuable. I recently completed a project similar to yours (recording webcam video along with a selected microphone to a single video file with audio). The key is to creating an aggregate media source.

// http://msdn.microsoft.com/en-us/library/windows/desktop/dd388085(v=vs.85).aspx
HRESULT CreateAggregateMediaSource(IMFMediaSource *videoSource,
                                   IMFMediaSource *audioSource,
                                   IMFMediaSource **aggregateSource)
{
    *aggregateSource = nullptr;
    IMFCollection *pCollection = nullptr;

    HRESULT hr = ::MFCreateCollection(&pCollection);

    if (S_OK == hr)
        hr = pCollection->AddElement(videoSource);

    if (S_OK == hr)
        hr = pCollection->AddElement(audioSource);

    if (S_OK == hr)
        hr = ::MFCreateAggregateSource(pCollection, aggregateSource);

    SafeRelease(&pCollection);
    return hr;
}

When configuring the sink writer, you will add 2 streams (one for audio and one for video).
Of course, you will also configure the writer correctly for the input stream types.

HRESULT        hr                  = S_OK;
IMFMediaType  *videoInputType      = nullptr;
IMFMediaType  *videoOutputType     = nullptr;
DWORD          videoOutStreamIndex = 0u;
DWORD          audioOutStreamIndex = 0u;
IMFSinkWriter *writer              = nullptr;

// [other create and configure writer]

if (S_OK == hr))
    hr = writer->AddStream(videoOutputType, &videoOutStreamIndex);    

// [more configuration code]

if (S_OK == hr)
    hr = writer->AddStream(audioOutputType, &audioOutStreamIndex);

Then when reading the samples, you will need to pay close attention to the reader streamIndex, and sending them to the writer appropriately. You will also need to pay close attention to the format that the codec expects. For instance, IEEE float vs PCM, etc. Good luck, and I hope it is not too late.

Windows Media Foundation Enumerate audio devices

Sample code for Windows Media Foundation enumerate audio devices, the device capture struct

struct CaptureDeviceParam
{
    /// <summary>
    /// The array of devices.
    /// </summary>
    IMFActivate **ppDevices;
    /// <summary>
    /// Device count.
    /// </summary>
    UINT32      count;
    /// <summary>
    /// Device selection.
    /// </summary>
    UINT32      selection;
};

And the enum device method.

        /// <summary>
        /// Get the audio capture devices.
        /// </summary>
        /// <param name="param">The capture device param.</param>
        void MediaCapture::GetAudioCaptureDevices(CaptureDeviceParam *param)
        {
            HRESULT hr = S_OK;
            IMFAttributes *pAttributes = NULL;

            // Initialize an attribute store to specify enumeration parameters.
            hr = MFCreateAttributes(&pAttributes, 1);

            // Ask for source type = audio capture devices
            if (SUCCEEDED(hr))
            {
                // Set the device attribute.
                hr = pAttributes->SetGUID(
                    MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE,
                    MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_AUDCAP_GUID
                );
            }

            // Enumerate devices.
            if (SUCCEEDED(hr))
            {
                // Enumerate the device list.
                hr = MFEnumDeviceSources(pAttributes, &(*param).ppDevices, &(*param).count);
            }

            // Safe release.
            SafeRelease(&pAttributes);
        }

GetAudioCaptureDevices is a static method in the 'MediaCapture' class, which can be called anytime.

Raw Audio File to AAC using Windows Media Foundation on Windows 7

The hr return code equals 3222091460

Those are HRESULT codes. Use this "ShowHresult" tool to have them conveniently decoded for you. The code means 0xC00D36C4 MF_E_UNSUPPORTED_BYTESTREAM_TYPE "The byte stream type of the given URL is unsupported."

The problem is basically that there is no support for these raw files, .WAV is a good source for raw audio - the file holds both format descriptor and the payload.

You can obviously read data from the raw audio file yourself and compress into AAC using Media Foundation's AAC Encoder via its IMFTransform interface. This is reasonably easy and you have AAC data on the output to e.g. write into raw .AAC.

Alternate options to Media Foundation is DirectShow (there are suitable codecs, though I thought it might be not so easy to start), libfaac, FFmpeg's libavcodec (available under LGPL, not GPL).

Capturing Audio In Media Foundation - ReadSample Never Returns

MFStartup and MFShutdown should only be called once for the duration of your presented code, for example, remove them everywhere and change main like this:

int main()
{
    MFStartup(MF_VERSION, MFSTARTUP_NOSOCKET);
    stub_function();
    MFShutdown();
}

Windows media foundation use raw image to encode video

Found PushSource in the Windows SDK samples, which does this.

Access raw audio/video frames from Windows.Media.Core.MediaSource

This API is a part of UWP Media Player API. You are not supposed to read video/audio samples using MediaSource because it is designated to be a wrapper over Media Foundation media sources of different kinds with uniform Media Player playback item management.

The only thing MediaSource is useful for is to wrap sources and schedule them for Media Player playback. Media Player is capable to access underlying Media Foundation sources behind the scene.

MediaStreamSource property is, in particular, null because this media source instance is not based on MediaStreamSource implementation. If you created media source supplying MediaStreamSource you would respectively have the property provided you access to originally supplied implementation layer. You have a source of a different type, so your non-null property is likely to be IRandomAccessStream.

To retrieve individual samples you need something like Media Foundation Media Sources, Media Foundation Source Reader, Windows.Media.Capture.Frames for cameras etc.

---

Related Topics