Image Sequence to Video Stream

Image sequence to video stream?

Well, this answer comes a bit late, but since I have noticed some activity with my original question lately (and the fact that there was not provided a working solution) I would like to give you what finally worked for me.

I'll split my answer into three parts:

Background
Problem
Solution

Background

(this section is not important for the solution)

My original problem was that I had a lot of images (i.e. a huge amount), images that were individually stored in a database as byte arrays. I wanted to make a video sequence with all these images.

My equipment setup was something like this general drawing:
Sample Image

The images depicted growing tomato plants in different states. All images were taken every 1 minute under daytime.

/*pseudo code for taking and storing images*/
while (true)
{
    if (daylight)
    {
        //get an image from the camera
        //store the image as byte array to db
    }
    //wait 1 min
}

I had a very simple db for storing images, there were only one table (the table ImageSet) in it:
Sample Image

Problem

I had read many articles about ffmpeg (please see my original question) but I couldn't find any on how to go from a collection of images to a video.

Solution

Finally, I got a working solution!
The main part of it comes from the open source project AForge.NET. In short, you could say that AForge.NET is a computer vision and artificial intelligence library in C#.
(If you want a copy of the framework, just grab it from http://www.aforgenet.com/)

In AForge.NET, there is this VideoFileWriter class (a class for writing videofiles with help of ffmpeg). This did almost all of the work. (There is also a very good example here)

This is the final class (reduced) which I used to fetch and convert image data into a video from my image database:

public class MovieMaker
{

    public void Start()
    {
        var startDate = DateTime.Parse("12 Mar 2012");
        var endDate = DateTime.Parse("13 Aug 2012");

        CreateMovie(startDate, endDate);
    }    
    

    /*THIS CODE BLOCK IS COPIED*/

    public Bitmap ToBitmap(byte[] byteArrayIn)
    {
        var ms = new System.IO.MemoryStream(byteArrayIn);
        var returnImage = System.Drawing.Image.FromStream(ms);
        var bitmap = new System.Drawing.Bitmap(returnImage);

        return bitmap;
    }

    public Bitmap ReduceBitmap(Bitmap original, int reducedWidth, int reducedHeight)
    {
        var reduced = new Bitmap(reducedWidth, reducedHeight);
        using (var dc = Graphics.FromImage(reduced))
        {
            // you might want to change properties like
            dc.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
            dc.DrawImage(original, new Rectangle(0, 0, reducedWidth, reducedHeight), new Rectangle(0, 0, original.Width, original.Height), GraphicsUnit.Pixel);
        }

        return reduced;
    }

    /*END OF COPIED CODE BLOCK*/

    private void CreateMovie(DateTime startDate, DateTime endDate)
    {
        int width = 320;
        int height = 240;
        var framRate = 200;

        using (var container = new ImageEntitiesContainer())
        {
            //a LINQ-query for getting the desired images
            var query = from d in container.ImageSet
                        where d.Date >= startDate && d.Date <= endDate
                        select d;

            // create instance of video writer
            using (var vFWriter = new VideoFileWriter())
            {
                // create new video file
                vFWriter.Open("nameOfMyVideoFile.avi", width, height, framRate, VideoCodec.Raw);

                var imageEntities = query.ToList();

                //loop throught all images in the collection
                foreach (var imageEntity in imageEntities)
                {
                    //what's the current image data?
                    var imageByteArray = imageEntity.Data;
                    var bmp = ToBitmap(imageByteArray);
                    var bmpReduced = ReduceBitmap(bmp, width, height);

                    vFWriter.WriteVideoFrame(bmpReduced);
                }
                vFWriter.Close();
            }
        }

    }
}

Update 2013-11-29 (how to) (Hope this is what you asked for @Kiquenet?)

Download AForge.NET Framework from the downloads page (Download full ZIP archive and you will find many interesting Visual Studio solutions with projects, like Video, in the AForge.NET Framework-2.2.5\Samples folder...)
Namespace: AForge.Video.FFMPEG (from the documentation)
Assembly: AForge.Video.FFMPEG (in AForge.Video.FFMPEG.dll) (from the documentation) (you can find this AForge.Video.FFMPEG.dll in the AForge.NET Framework-2.2.5\Release folder)

If you want to create your own solution, make sure you have a reference to AForge.Video.FFMPEG.dll in your project. Then it should be easy to use the VideoFileWriter class. If you follow the link to the class you will find a very good (and simple example). In the code, they are feeding the VideoFileWriter with Bitmap image in a for-loop

Generating video from a sequence of images in C#

http://electron.mit.edu/~gsteele/ffmpeg/

http://www.codeproject.com/Articles/7388/A-Simple-C-Wrapper-for-the-AviFile-Library

http://ffmpeg.org/ffmpeg.html -> search for For creating a video from many images:

All the links are from this question on SO

Link related to FFMPEG in .net (From this question);

FFMpeg.NET

FFMpeg-Sharp

FFLib.NET

http://ivolo.mit.edu/post/Convert-Audio-Video-to-Any-Format-using-C.aspx

Other resources

Expression Encoder

VLC

Convert image sequence to MP4 file using .Net Core and C# on Azure Storage Blobs

Per my experience, the solution to convert image sequence to a video streaming or file is normally to use some popular libraries like ffmpeg or libav to do that.

If you search for some keywords like C# ffmpeg images video or C# libav images video, you will get much useful content to help realizing it, as below, there are few threads you can refer to.

Image sequence to video stream?
Converting Images into Video using C#
tomaszzmuda/Xabe.FFmpeg

And then, you can get the video streaming or file to upload to Azure Blob Storage as a block blob, please follow the offical tutorial Quickstart: Azure Blob storage client library for .NET to know how to use Azure Storage SDK for .NET standard and the key class in C# is CloudBlockBlob Class which functions enter link description hereBeginUploadFromFile for video file or BeginUploadFromStream for video stream.

But there is an issue about your app deployment. Due to the limitation of Azure Web App sandbox about Win32k.sys (User32/GDI32) Restrictions as the figure below, you can not deploy any app used GDI system call to Azure WebApp.

Sample Image

So a normal recommended Azure service for your app is Azure VM.

Update:

If you considered to use dotNet Core to realize it, I suggested that you can try to use Azure App Service on Linux which be based on Docker and no limits for GDI. Also, Azure Batch is the other choice for realizing your needs in any language. And I wish you will complete your evaluation.

Streaming video using stream of images

I think this thread may be of help

http://ubuntuforums.org/showthread.php?t=665607

It explains how to use ffmpeg and ffserver for live streaming while the converted video is generated. This works even for infinite streams.

when you stream your video using ffserver, you may want to set a reverse-proxy on your main webserver so that your users do not need to type a port number for the video.

ffmpeg can read from image sequences, use %d to represent the incrementing number. Just use the sequence as your input and the ffserver stream as your output

How to create video file from Images sequence file?

Look up the documentation, find the VideoFileWriter Class, look at it's members, see the WriteVideoFrame method(s), read the line: "Write new video frame into currently opened video file.". Bingo. Half way there.

If you can't connect the dots when reading the methods' signature (WriteVideoFrame(Bitmap)) or don't understand how to use the Open() overloads or Close() method (why wouldn't you, the last two are pretty common for File I/O) you can always Google "VideoFileWriter WriteVideoFrame example", find code, go from there. The meat of the function there is:

VideoFileWriter writer = new VideoFileWriter();
writer.Open("myfile.avi", width, height, 25, VideoCodec.MPEG4, 1000000);
    // ... here you'll need to load your bitmaps
    writer.WriteVideoFrame(image);
}
writer.Close();

So something like this should probably work:

using (VideoFileWriter writer = new VideoFileWriter()) 
{
    writer.Open(@"d:\myfile.avi", 640, 480, 25, VideoCodec.MPEG4);
    foreach (var file in Directory.GetFiles(@"d:\foo\bar", "*.jpg"))
    {
        writer.WriteVideoFrame(Bitmap.FromFile(file) as Bitmap);
    }
    writer.Close();
}

Which, with a few minutes of fiddling around, gave me something like this:

var size = new Size(1600, 1200);                    // The desired size of the video
var fps = 25;                                       // The desired frames-per-second
var codec = VideoCodec.MPEG4;                       // Which codec to use
var destinationfile = @"d:\myfile.avi";             // Output file
var srcdirectory = @"d:\foo\bar";                   // Directory to scan for images
var pattern = "*.jpg";                              // Files to look for in srcdirectory
var searchoption = SearchOption.TopDirectoryOnly;   // Search Top Directory Only or all subdirectories (recursively)?

using (var writer = new VideoFileWriter())          // Initialize a VideoFileWriter
{
    writer.Open(destinationfile, size.Width, size.Height, fps, codec);              // Start output of video
    foreach (var file in Directory.GetFiles(srcdirectory, pattern, searchoption))   // Iterate files
    {
        using (var original = (Bitmap)Image.FromFile(file))     // Read bitmap
        using (var resized = new Bitmap(original, size))        // Resize if necessary
            writer.WriteVideoFrame(resized);                    // Write frame
    }
    writer.Close();                                 // Close VideoFileWriter
}                                                   // Dispose VideoFileWriter

This resizes images; should not all images in the sequence be the same. If they are you can skip that step simply by changing

using (var original = (Bitmap)Image.FromFile(file))     // Read bitmap
using (var resized = new Bitmap(original, size))        // Resize if necessary
    writer.WriteVideoFrame(resized);                    // Write frame

to:

using (var mybitmap = (Bitmap)Image.FromFile(file))     // Read bitmap
    writer.WriteVideoFrame(mybitmap);                   // Write frame

Also make sure you add the correct using statements; you will, at a minimum, need the following for above examples:

using AForge.Video.FFMPEG;
using System.Drawing;
using System.IO;

Also you'll need to reference the DLL's as described here:

... you should have created a project, added a reference to the AForge.Video.FFMPEG library, set the target platform to x86 and the target framework version to 3.5 or lower now. If so, it can go on.
... we need a few more dlls from the AForge archive. You can find these in the folder “Externals\ffmpeg” inside the AForge archive. All files in this folder have to be copied to the output folder of your Visual Studio project. (After we changed the target architecture this now should be “YourProjectFolder\bin\x86\Debug”.)

If it still doesn't work then tell us what happens, exact error messages etc.

Image Sequence to Video Stream