Getting Correct Image Rotation

Getting correct Image rotation

If the pictures contains exif data the PropertyItems should include the orientation tag.

It encodes the rotation/flipping necessary to display the image correctly:

PropertyTagOrientation
Image orientation viewed in terms of rows and columns.
Tag 0x0112
1 - The 0th row is at the top of the
visual image, and the 0th column is the visual left side.

2 - The 0th
row is at the visual top of the image, and the 0th column is the
visual right side.

3 - The 0th row is at the visual bottom of the
image, and the 0th column is the visual right side.

4 - The 0th row
is at the visual bottom of the image, and the 0th column is the visual
left side.

5 - The 0th row is the visual left side of the image, and
the 0th column is the visual top.

6 - The 0th row is the visual right
side of the image, and the 0th column is the visual top.

7 - The 0th
row is the visual right side of the image, and the 0th column is the
visual bottom.

8 - The 0th row is the visual left side of the image,
and the 0th column is the visual bottom.

Here is a function to retrieve a PropertyItem:

PropertyItem getPropertyItemByID(Image img, int Id)
{
  return 
    img.PropertyItems.Select(x => x).FirstOrDefault(x => x.Id == Id);
}

Here is an example of using the GDI+ RotateFlip method to adjust an image on the fly:

void Rotate(Bitmap bmp)
{
    PropertyItem pi = bmp.PropertyItems.Select(x => x)
                         .FirstOrDefault(x => x.Id == 0x0112);
    if (pi == null) return; 

    byte o = pi.Value[0];

    if (o==2) bmp.RotateFlip(RotateFlipType.RotateNoneFlipX);
    if (o==3) bmp.RotateFlip(RotateFlipType.RotateNoneFlipXY);
    if (o==4) bmp.RotateFlip(RotateFlipType.RotateNoneFlipY);
    if (o==5) bmp.RotateFlip(RotateFlipType.Rotate90FlipX);
    if (o==6) bmp.RotateFlip(RotateFlipType.Rotate90FlipNone);
    if (o==7) bmp.RotateFlip(RotateFlipType.Rotate90FlipY);
    if (o==8) bmp.RotateFlip(RotateFlipType.Rotate90FlipXY);
}

It changes the image to the correctly rotated version..

I have tested to values with this nice set of sample images.

Note: The code will only work if the images actually contain the orientation tag. If they don't, maybe because they are scans, then it will do nothing.

Note 2 You wrote I checked the original image rotation. This is not so simple: The explorer will display the images already rotated, so here they all look right and even inspecting the properties doesn't reveal the orientation!

Usually, when no exif data are present, the PropertyTagOrientation tag is present but only has the default value of 1..

Update:
If the image doesn't have the PropertyTagOrientation here is how you can add one:

    using System.Runtime.Serialization;
    ..

    pi = (PropertyItem)FormatterServices
        .GetUninitializedObject(typeof(PropertyItem));

    pi.Id = 0x0112;   // orientation
    pi.Len = 2;
    pi.Type = 3;
    pi.Value = new byte[2] { 1, 0 };

    pi.Value[0] = yourOrientationByte;

    yourImage.SetPropertyItem(pi);

Kudos to @ne1410s's excellent answer here!.

Note that adding PropertyItems to an image does not add exif data; the two are different tag sets!

How to adjust the orientation of a image in order to show proper preview of the image?

How about just rotating the image with CSS?

reader.onload = function (e) {
    $(elm).css({
        "background-image":"url('"+e.target.result+"')",
        "transform": "rotate(90deg)"
    });
}

Edit: Well, after a brief dive into the world of EXIF data, I think I might have a solution for you. I borrowed Ali's getOrientation() function, as @Nick suggested, and then I just corrected the orientation with CSS as follows:

function correctOrientation(element, orientation){
    switch (orientation) {
      case 2: $(element).css("transform", "scaleX(-1)");
        break;
      case 3: $(element).css("transform", "rotate(180deg)");
        break;
      case 4: $(element).css("transform", "rotate(180deg) scaleX(-1)");
        break;
      case 5: $(element).css("transform", "rotate(-90deg) scaleX(-1)");
        break;
      case 6: $(element).css("transform", "rotate(90deg)");
        break;
      case 7: $(element).css("transform", "rotate(90deg) scaleX(-1)");
        break;
      case 8: $(element).css("transform", "rotate(-90deg)");
        break;
      default: break;
    }
}

fiddle

And if you need example files to test it, get them here.

Edit: Placing the uploaded image in an <img> instead of the div's background-image:
https://jsfiddle.net/ynzvtLe2/2/

Is there a way i can detect the image orientation and rotate the image to the right angle?

This is an interesting problem, i have tried with many approaches to correct orientation of document images but all of them have got different exceptions.
I am sharing one of the approaches based on text orientation. For text region detection i am using gradient map of input image.

All other implementation details are commented in the code.

Please note that this only works if all the text present in image have same orientation.

#Document image orientation correction
#This approach is based on text orientation

#Assumption: Document image contains all text in same orientation

import cv2
import numpy as np

debug = True

#Display image
def display(img, frameName="OpenCV Image"):
    if not debug:
        return
    h, w = img.shape[0:2]
    neww = 800
    newh = int(neww*(h/w))
    img = cv2.resize(img, (neww, newh))
    cv2.imshow(frameName, img)
    cv2.waitKey(0)

#rotate the image with given theta value
def rotate(img, theta):
    rows, cols = img.shape[0], img.shape[1]
    image_center = (cols/2, rows/2)
    
    M = cv2.getRotationMatrix2D(image_center,theta,1)

    abs_cos = abs(M[0,0])
    abs_sin = abs(M[0,1])

    bound_w = int(rows * abs_sin + cols * abs_cos)
    bound_h = int(rows * abs_cos + cols * abs_sin)

    M[0, 2] += bound_w/2 - image_center[0]
    M[1, 2] += bound_h/2 - image_center[1]

    # rotate orignal image to show transformation
    rotated = cv2.warpAffine(img,M,(bound_w,bound_h),borderValue=(255,255,255))
    return rotated

def slope(x1, y1, x2, y2):
    if x1 == x2:
        return 0
    slope = (y2-y1)/(x2-x1)
    theta = np.rad2deg(np.arctan(slope))
    return theta

def main(filePath):
    img = cv2.imread(filePath)
    textImg = img.copy()

    small = cv2.cvtColor(textImg, cv2.COLOR_BGR2GRAY)

    #find the gradient map
    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
    grad = cv2.morphologyEx(small, cv2.MORPH_GRADIENT, kernel)

    display(grad)

    #Binarize the gradient image
    _, bw = cv2.threshold(grad, 0.0, 255.0, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
    display(bw)

    #connect horizontally oriented regions
    #kernal value (9,1) can be changed to improved the text detection
    kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9, 1))
    connected = cv2.morphologyEx(bw, cv2.MORPH_CLOSE, kernel)
    display(connected)

    # using RETR_EXTERNAL instead of RETR_CCOMP
    # _ , contours, hierarchy = cv2.findContours(connected.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
    contours, hierarchy = cv2.findContours(connected.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) #opencv >= 4.0

    mask = np.zeros(bw.shape, dtype=np.uint8)
    #display(mask)
    #cumulative theta value
    cummTheta = 0
    #number of detected text regions
    ct = 0
    for idx in range(len(contours)):
        x, y, w, h = cv2.boundingRect(contours[idx])
        mask[y:y+h, x:x+w] = 0
        #fill the contour
        cv2.drawContours(mask, contours, idx, (255, 255, 255), -1)
        #display(mask)
        #ratio of non-zero pixels in the filled region
        r = float(cv2.countNonZero(mask[y:y+h, x:x+w])) / (w * h)

        #assume at least 45% of the area is filled if it contains text
        if r > 0.45 and w > 8 and h > 8:
            #cv2.rectangle(textImg, (x1, y), (x+w-1, y+h-1), (0, 255, 0), 2)

            rect = cv2.minAreaRect(contours[idx])
            box = cv2.boxPoints(rect)
            box = np.int0(box)
            cv2.drawContours(textImg,[box],0,(0,0,255),2)

            #we can filter theta as outlier based on other theta values
            #this will help in excluding the rare text region with different orientation from ususla value 
            theta = slope(box[0][0], box[0][1], box[1][0], box[1][1])
            cummTheta += theta
            ct +=1 
            #print("Theta", theta)
            
    #find the average of all cumulative theta value
    orientation = cummTheta/ct
    print("Image orientation in degress: ", orientation)
    finalImage = rotate(img, orientation)
    display(textImg, "Detectd Text minimum bounding box")
    display(finalImage, "Deskewed Image")

if __name__ == "__main__":
    filePath = 'D:\data\img6.jpg'
    main(filePath)

Here is Image with detected text regions, from this we can see that some of the text regions are missing. Text orientation detection plays the key role here in overall document orientation detection so based on document type a few small tweaks should be made in the text detection algorithm to make this approach work better.

Image with detected text regions

Here is the final image with correct orientation
Final deskewed image

Please suggest modifications in this approaches to make it more robust.

Incorrect image rotation in img tag

I found the problem. The images that don't show properly are uploaded from mobile devices such as Samsung and iPhone. So the EXIF orientation is that of the mobile.

To see the exact exif of the image, you can test it here http://exif.regex.info/exif.cgi. Even if you open the image directly in browsers such as Chrome or Firefox, you will see the image with the right orientation, but if you load the image using the html img tag, it will show the exif orientation.

So the best solution is to check exif orientation before uploading image, and then convert it to the right orientation by using a php function.

function image_fix_orientation($path)
{
    $image = imagecreatefromjpeg($path);
    $exif = exif_read_data($path);

    if (empty($exif['Orientation']))
    {
        return false;
    }

    switch ($exif['Orientation'])
    {
        case 3:
            $image = imagerotate($image, 180, 0);
            break;
        case 6:
            $image = imagerotate($image, - 90, 0);
            break;
        case 8:
            $image = imagerotate($image, 90, 0);
            break;
    }

    imagejpeg($image, $path);

    return true;
}

// For JPEG image only
image_fix_orientation('/path/to/image.jpg');

Problem with image orientation after upload

This is the solution that works for me now. The image orientation correction code is in lines 50-70. There may be a better solution, but this is the only thing I have managed to do, to work properly:

<?php
if(isset($_FILES['image'])){
    $errors= array();
    foreach($_FILES['image']['tmp_name'] as $key => $tmp_name ){
        $file_name =$_FILES['image']['name'][$key];
        $file_size =$_FILES['image']['size'][$key];
        $file_tmp =$_FILES['image']['tmp_name'][$key];
        $file_type=$_FILES['image']['type'][$key];      

            // Remove encoding problem
            $file_name = Normalizer::normalize($file_name); 
            setlocale(LC_ALL,'bs_BA.UTF-8');        

            // get file extension
            $fileType = strtolower(pathinfo($file_name, PATHINFO_EXTENSION)); 

            $temp = pathinfo($file_name, PATHINFO_EXTENSION);
            $name = str_replace($temp, '', $file_name);

            // get filename without extension
            $fileNewName = pathinfo($name, PATHINFO_FILENAME);
            $watermarkImagePath = 'watermark.png'; 
            $folderPath = "a/";
            $sourceProperties = getimagesize($file_tmp);
            $imageType = $sourceProperties[2];

            // Resize code
            switch ($imageType) {
            case IMAGETYPE_PNG:
                $imageResourceId = imagecreatefrompng($file_tmp); 
                $targetLayer = imageResize($imageResourceId,$sourceProperties[0],$sourceProperties[1]);
                imagepng($targetLayer,$folderPath. $fileNewName. ".jpg");
                break;
            case IMAGETYPE_GIF:
                $imageResourceId = imagecreatefromgif($file_tmp); 
                $targetLayer = imageResize($imageResourceId,$sourceProperties[0],$sourceProperties[1]);
                imagegif($targetLayer,$folderPath. $fileNewName. ".jpg");
                break;
            case IMAGETYPE_JPEG:
                $imageResourceId = imagecreatefromjpeg($file_tmp); 
                $targetLayer = imageResize($imageResourceId,$sourceProperties[0],$sourceProperties[1]);
                imagejpeg($targetLayer,$folderPath. $fileNewName. ".jpg");

                break;
            default:
                echo "Invalid Image type.";
                exit;
                break;
            }               

        // Image Orientation correction                         

        $targetFilePath = $folderPath . $file_name;                 
        $exif = exif_read_data($file_tmp);
        if ($exif['Orientation']==3 OR $exif['Orientation']==6 OR $exif['Orientation']==8) {
            $imageResource = imagecreatefromjpeg($targetFilePath); 
            switch ($exif['Orientation']) { 
            case 3:
            $image = imagerotate($imageResource, 180, 0);
            break;
            case 6:
            $image = imagerotate($imageResource, -90, 0);
            break;
            case 8:
            $image = imagerotate($imageResource, 90, 0);
            break;
        } 
        imagejpeg($image, $targetFilePath);
        imagedestroy($imageResource);
        imagedestroy($image);
        }

        // watermark code

                $watermarkImg = imagecreatefrompng($watermarkImagePath); 

                if(preg_match('/[.](jpg)$/i', $file_name)) {  
                $im = imagecreatefromjpeg($targetFilePath);  
              } else if (preg_match('/[.](jpeg)$/i', $file_name)) {  
                $im = imagecreatefromjpeg($targetFilePath);  
              } else if (preg_match('/[.](png)$/i', $file_name)) {  
                $im = imagecreatefrompng($targetFilePath);  
              }  else if (preg_match('/[.](gif)$/i', $file_name)) {  
                $im = imagecreatefromgif($targetFilePath);  
              }  

                $marge_right = 1; 
                $marge_bottom = 1; 

                $sx = imagesx($watermarkImg); 
                $sy = imagesy($watermarkImg); 

                imagecopy($im, $watermarkImg, imagesx($im) - $sx - $marge_right, imagesy($im) - $sy - $marge_bottom, 0, 0, imagesx($watermarkImg), imagesy($watermarkImg)); 

                imagejpeg($im, $targetFilePath,70); 
                imagedestroy($im);     

    }
     echo ' Successful upload';
}
function imageResize($imageResourceId,$width,$height) {
if($width > $height){
    $targetWidth=1000;
    $targetHeight=($height/$width)*$targetWidth;
} else {
    $targetHeight=1000;
$targetWidth=($width/$height)*$targetHeight;}
    $targetLayer=imagecreatetruecolor($targetWidth,$targetHeight);
    imagecopyresampled($targetLayer,$imageResourceId,0,0,0,0,$targetWidth,$targetHeight, $width,$height);
    return $targetLayer;
}

?>
<div class="sender">
<form action="" method="POST" enctype="multipart/form-data">
    <input type="file" name="image[]" multiple/>
    <input type="submit" value="Send"/>
</form></div>

Getting Correct Image Rotation