Load Image with Opencv Mat C++

How to read an image into a Mat object using opencv c++ and android studio ndk?

Since this was my first time using Android Studio there was much to learn. It took me a while but here is the answers to the questions that I posted.

Question 1: Is the specified imagePath = "//drawable//ring.png" correct to access the ring.png file ?

This is most definitely not the correct path to use when accessing images for the purpose of image processing etc.The drawable folder can still be used to update ex. an image view by setting the src of the image view to the image

 imageView.setImageResource(R.drawable.ring);

When working with images and Mat objects, I found it best to use the Android debugging bridge to copy the files to the SD card of the device. This link will provided you with the necessary steps to install the adb https://www.howtogeek.com/125769/how-to-install-and-use-abd-the-android-debug-bridge-utility/

When the images are done copying to the SD card. The file path can be found by making use of the built in Java functionEnvironment.getExternalStorageDirectory(). Which looks something like /storage/emulated/0/YOUR_file it depends on the location which was used to copy the files to.

Useful tip: Download ES File Explorer for the device to help navigate through the external or internal storage.

Question 2: Is there any permissions needed allowing the ndk to access res folders ?
The method which I used didn't need any permissions. However at the moment the NDK side cannot directly read an image from the SD card, the image must be passed from the Java side by making use of assets or by passing the address of the image which was read into the Mat object(Java side).

Read and write permission is needed in order to access the SD card. This must be set in the manifest.xml and must be correctly implemented in the code. There are many great tutorials on YouTube.

Question 3: Is there any similar methods to assign an image to a Mat object?
This question seems redundant now, there are many ways to skin a cat.

In short, I think it is easier to stick to the Java side when using Opencv4Android and some form of image processing is needed.To get you started in java here is a small snippet from my code.

Mat image;
String imageInSD = Environment.getExternalStorageDirectory()+"/Pictures/Data/Birds/"(ImageFolders[i])+"/"+String.valueOf(id)+".png";

image = Imgcodecs.imread(imageInSD,Imgcodecs.IMREAD_COLOR);

Good luck!!

OpenCv Fastest way to make Mat Image from 1 D Array in C++

You can create an empty Mat and set the data to be your array.

unsigned char* array = ...
img = cv::Mat(1944,2592, CV_8U);
img.data = array;

cv::imshow('My image', img);
cv::waitKey()

A faster method would be to directly create the Mat with your data. This way, no data pointer is allocated when creating the object. It might also be safer as you're not manipulating internal data pointers directly.

img = cv::Mat(1944, 2592, CV_8U, array);

Opencv create new image using cv::Mat

First, you need the following information to create the image:

1. Width: 301 pixels
2. Height: 260 pixels
3. Each pixel value (intensity) is 0 ~ 255: an 8-bit unsigned integer
4. Supports all RGB colors: 3 channels
5. Initial color: black = (B, G, R) = (0, 0, 0)

You can create the Image using cv::Mat:

Mat grHistogram(260, 301, CV_8UC3, Scalar(0, 0, 0));

The 8U means the 8-bit Usigned integer, C3 means 3 Channels for RGB color, and Scalar(0, 0, 0) is the initial value for each pixel. Similarly,

line(grHistrogram,pt1,pt2,Scalar(255,255,255),1,8,0);

is to draw a line on grHistogram from point pt1 to point pt2. The color of line is white (255, 255, 255) with 1-pixel thickness, 8-connected line, and 0-shift.

Sometimes you don't need a RGB-color image, but a simple grayscale image. That is, use one channel instead of three. The type can be changed to CV_8UC1 and you only need to specify the intensity for one channel, Scalar(0) for example.

Back to your problem,

Why this happening since, both create a Mat image structure?

Because you need to specify the type of the Mat. Is it a color image CV_8UC3 or a grayscale image CV_8UC1? They are different. Your program may not work as you think if you use Scalar(255) on a CV_8UC3 image.

What is the purpose of const cv:Scalar &_s ?

cv::Scalar is use to specify the intensity value for each pixel. For example, Scalar(255, 0, 0) is blue and Scalar(0, 0, 0) is black if type is CV_8UC3. Or Scalar(0) is black if it's a CV_8UC1 grayscale image. Avoid mixing them together.

Loading image from file as given type into an OpenCV Mat?

As @Miki pointed out in the comments the answer was getting the channe type to match for the image and template. I ended up changing my bufferedImageToMat function.

private Mat bufferedImageToMat(BufferedImage inBuffImg) 
{
    BufferedImage image = new BufferedImage(inBuffImg.getWidth(), inBuffImg.getHeight(), BufferedImage.TYPE_4BYTE_ABGR);
    Graphics2D g2d= image.createGraphics();
    g2d.drawImage(inBuffImg, 0, 0, null);
    g2d.dispose();

    Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3);
    byte[] data = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
    mat.put(0, 0, data);

    return mat;
}

My templates are read in as CvType.CV_8UC3, so it was just a matter of creating a Mat from the screen image with this type!

How can I load an Image using Opencv and pass it to Unity (through a C++ dll)?

I found the cause of the crash which was later on when I wanted to get the values which I casted as the wrong type of values (Vecd instead of the correct Vec3b).
Additionally, I was also not using "new cv::Mat()" when testing and that cause the Mat to be discarded by the C++ (and the C# pointing to random stuff).

For anyone that would like to do something similare here is my code (even if it' is heavily inspired by ArucoUnity/ArucoUnityPlugin by NormandErwan).

C++

With the following somewhere (for me it was in PatternRecognition.hpp)

#ifdef PATTERNRECOGNITIONTOCLONEDLL_EXPORTS
#define PATTERNRECOGNITION_API __declspec(dllexport)
#else
#define PATTERNRECOGNITION_API __declspec(dllimport)
#endif

mat.hpp

#include <opencv2/core.hpp>
#include "PatternRecognition.hpp"

using namespace CloneToTangible;

extern "C" {

    const string pluginFolderPath = "./Assets/Plugin/";

    PATTERNRECOGNITION_API cv::Mat* mat_new();
    PATTERNRECOGNITION_API cv::Mat* mat_loadPath(const char* path);

    PATTERNRECOGNITION_API void mat_delete(cv::Mat* matPtr);

    PATTERNRECOGNITION_API Vec3b* mat_at(cv::Mat* mat, uint posX, uint posY);

    PATTERNRECOGNITION_API int mat_rows(cv::Mat* mat);
    PATTERNRECOGNITION_API int mat_cols(cv::Mat* mat);

    PATTERNRECOGNITION_API int mat_type(cv::Mat* mat);
}

mat.cpp

#include "pch.h"
#include "mat.hpp"
#include <opencv2\imgcodecs.hpp>

cv::Mat* mat_new()
{
    Mat* matPtr = new cv::Mat();
    //DebugLog(("[C++] Mat Loaded -> at " + std::to_string((int)matPtr) + "  (" + std::to_string(matPtr->rows) + "," + std::to_string(matPtr->cols) + ") " + std::to_string(matPtr->at<Vec3b>(0, 0).val[1])).c_str());
    return matPtr;
}

cv::Mat* mat_loadPath(const char* path)
{
    //DebugLog(("[C++] Loading mat by Path -> " + pluginFolderPath + string(path)).c_str());
    Mat* matPtr = new cv::Mat();
    *matPtr = cv::imread(pluginFolderPath + string(path), cv::IMREAD_COLOR);
    //DebugLog(("[C++] Mat Loaded -> at " + std::to_string((int)matPtr) + "  (" + std::to_string(matPtr->rows) + "," + std::to_string(matPtr->cols) + ") " + std::to_string(matPtr->at<Vec3b>(0, 0).val[1])).c_str());
    return matPtr;
}

void mat_delete(cv::Mat* matPtr)
{
    delete matPtr;
}

Vec3b* mat_at(cv::Mat* mat, uint posX, uint posY)
{
    return new Vec3b(mat->at<Vec3b>(posX,posY));
}

int mat_rows(cv::Mat* mat)
{
    //DebugLog(("Returning rows for " + std::to_string((int)&mat) + " which is " + std::to_string(mat->rows)).c_str());
    return mat->rows;
}

int mat_cols(cv::Mat* mat)
{
    //DebugLog(("Returning cols for " + std::to_string((int)&mat) + " which is " + std::to_string(mat->cols)).c_str());
    return mat->cols;
}

int mat_type(cv::Mat* mat)
{
    return mat->type();
}

vec3b.hpp

#include <opencv2/core.hpp>
#include "PatternRecognition.hpp"

extern "C" {

    PATTERNRECOGNITION_API cv::Vec3b* cv_Vec3b_new(unsigned char v0, unsigned char v1, unsigned char v2);

    PATTERNRECOGNITION_API void cv_Vec3b_delete(cv::Vec3b* vec3dPtr);

    PATTERNRECOGNITION_API unsigned char cv_Vec3b_get(cv::Vec3b* vec3dPtr, int i);

    PATTERNRECOGNITION_API void cv_Vec3b_set(cv::Vec3b* vec3dPtr, int i, unsigned char value);

    PATTERNRECOGNITION_API void cv_Vec3b_setAll(cv::Vec3b* vec3dPtr, unsigned char v0, unsigned char v1, unsigned char v2);;
}

vec3b.cpp

#include "pch.h"
#include "vec3b.hpp"

cv::Vec3b* cv_Vec3b_new(unsigned char v0, unsigned char v1, unsigned char v2)
{
    return new cv::Vec3b(v0, v1, v2);
}

void cv_Vec3b_delete(cv::Vec3b* vec3bPtr)
{
    int n = (*vec3bPtr)[0];
    delete vec3bPtr;
}

unsigned char cv_Vec3b_get(cv::Vec3b* vec3bPtr, int i)
{
    unsigned char value = 0;
    try {
        value = (*vec3bPtr)[i];
    }
    catch (const cv::Exception& e) {
        return value;
    }
    return value;
}

void cv_Vec3b_set(cv::Vec3b* vec3bPtr, int i, unsigned char value)
{
    try {
        (*vec3bPtr)[i] = value;
    }
    catch (const cv::Exception& e) {
    }
}

void cv_Vec3b_setAll(cv::Vec3b* vec3bPtr, unsigned char v0, unsigned char v1, unsigned char v2)
{
    try {
        (*vec3bPtr)[0] = v0;
        (*vec3bPtr)[1] = v1;
        (*vec3bPtr)[2] = v2;
    }
    catch (const cv::Exception& e) {
    }
}

C#

Mat.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;
using UnityEngine;

class Mat : HandleCppPtr
{
    public int cols
    {
        get { return mat_cols(CppPtr); }
    }
    public int rows
    {
        get { return mat_rows(CppPtr); }
    }
    public int type
    {
        get { return mat_type(CppPtr); }
    }

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern IntPtr mat_new();

    [DllImport("PatternRecognitionToCloneDLL")]
    public static extern IntPtr mat_loadPath(string path);

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern void mat_delete(IntPtr matPtr);

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern IntPtr mat_at(IntPtr matPtr, uint x, uint y);

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern int mat_cols(IntPtr matPtr);

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern int mat_rows(IntPtr matPtr);

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern int mat_type(IntPtr matPtr);

    public Mat(string path, DeleteResponsibility deleteResponsibility = DeleteResponsibility.True) : base(mat_loadPath(path), deleteResponsibility)
    {
        Debug.Log("[C#] Mat Instantiation : Mat(string path) ");
    }

    public Mat(DeleteResponsibility deleteResponsibility = DeleteResponsibility.True) : base(mat_new(), deleteResponsibility)
    {
        Debug.Log("[C#] Mat Instantiation : Mat() ");
    }
    public Mat(IntPtr cppPtr, DeleteResponsibility deleteResponsibility = DeleteResponsibility.True) : base(cppPtr, deleteResponsibility)
    {

    }

    public Vec3b At(uint x, uint y)
    {
        if (x < rows && y < cols)
            return new Vec3b(mat_at(this.CppPtr, x, y));
        else
            return new Vec3b();
    }

    protected override void DeleteCppPtr()
    {
        //Debug.Log("[C#] Mat Destruction : DeleteCppPtr(), before mat_delete() call ");
        mat_delete(this.CppPtr);
        //Debug.Log("[C#] Mat Destruction : DeleteCppPtr(), after mat_delete() call ");
    }

    public override string ToString()
    {
        string message = "Mat->";
        int rows = mat_rows(this.CppPtr);
        Debug.Log("[C#] Mat has " + rows + " rows");
        int cols = mat_cols(this.CppPtr);
        Debug.Log("[C#] Mat has " + cols + " cols");
        for (uint x = 0; x < rows; x++)
        {
            message += "\n {";
            for (uint y = 0; y < cols; y++)
            {
                message += At(x, y).ToString();
            }
            message += "}";
        }
        return message;
    }

}

Vec3b.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;

public class Vec3b : HandleCppPtr
{

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern IntPtr cv_Vec3b_new(byte v0, byte v1, byte v2);

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern void cv_Vec3b_delete(IntPtr vec3bPtr);

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern byte cv_Vec3b_get(IntPtr vec3bPtr, int i);

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern void cv_Vec3b_set(IntPtr vec3bPtr, int i, byte value);

    [DllImport("PatternRecognitionToCloneDLL")]
    static extern void cv_Vec3b_setAll(IntPtr vec3bPtr, byte v0, byte v1, byte v2);

    public Vec3b(byte v0 = 0, byte v1 = 0, byte v2 = 0) : base(cv_Vec3b_new(v0, v1, v2))
    {
    }

    public Vec3b(IntPtr cppPtr) : base(cppPtr)
    {
    }

    protected override void DeleteCppPtr()
    {
        cv_Vec3b_delete(this.CppPtr);
    }

    public int Get(int i)
    {
        return cv_Vec3b_get(this.CppPtr, i);
    }

    public void Set(int i, byte value)
    {
        cv_Vec3b_set(this.CppPtr, i, value);
    }

    public void Set(byte v0, byte v1, byte v2)
    {
        cv_Vec3b_setAll(this.CppPtr, v0, v1, v2);
    }

    public override string ToString()
    {
        string message = "Vec3b->";
        message += "(" + Get(0) + "," + Get(1) + "," + Get(2) + ")";
        return message;
    }

}

HandleCppPtr.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;
using UnityEngine;

public enum DeleteResponsibility
{
    True,
    False
}
public abstract class  HandleCppPtr
{

    public HandleCppPtr(IntPtr cppPtr, DeleteResponsibility deleteResponsibility = DeleteResponsibility.True)
    {
        CppPtr = cppPtr;
        this.deleteResponsibility = deleteResponsibility;
    }

    ~HandleCppPtr()
    {
        if(this.deleteResponsibility == DeleteResponsibility.True)
        {
            //Debug.Log("[C# ]Deleting HandleCppPtr object -> " + this.ToString());
            DeleteCppPtr();
        }
        else
        {
            //Debug.Log("[C# ] NOT Deleting HandleCppPtr object because deleteResp is False -> " + this.ToString());
        }
    }

    public IntPtr CppPtr
    {
        get { return handle.Handle; }
        set { handle = new HandleRef(this, value); }
    }
    public DeleteResponsibility deleteResponsibility;
    HandleRef handle;
    protected abstract void DeleteCppPtr();
}

---

Related Topics