Get Pixel Data as Array from Uiimage/Cgimage in Swift

iOS: How to get pixel data array from CGImage in Swift

Per the link provided in your question, you can get the pixelData by doing

extension UIImage {
    func pixelData() -> [UInt8]? {
        let size = self.size
        let dataSize = size.width * size.height * 4
        var pixelData = [UInt8](repeating: 0, count: Int(dataSize))
        let colorSpace = CGColorSpaceCreateDeviceRGB()
        let context = CGContext(data: &pixelData,
                                width: Int(size.width),
                                height: Int(size.height),
                                bitsPerComponent: 8,
                                bytesPerRow: 4 * Int(size.width),
                                space: colorSpace,
                                bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue)
        guard let cgImage = self.cgImage else { return nil }
        context?.draw(cgImage, in: CGRect(x: 0, y: 0, width: size.width, height: size.height))

        return pixelData
    }
 }

However, as a developer, the concerning objects here are the bitmapInfo and colorSpace. Your image may be getting distorted or colored differently depending on the information provided. The exact solution will be dependent upon how you obtained the image and what color schemes were provided from the image. You may just need to play with the variables.

I've never had an issue using CGColorSpaceCreateDeviceRGB() as my colorSpace but I have had to alter my bitmapInfo many times as my images were coming in as a different value.

Here is the location to reference the different types of bitmaps. More than likely though, you only need a variation of the CGImageAlphaInfo which can be located here.

If necessary, you can change the colorSpace. The default CGcolorSpace webpage can be found here. However, you could probably get away with one of the default ones located here

Re: Get pixel data as array from UIImage/CGImage in swift

You probably forgot the CGImageAlphaInfo parameter. For color images, if you assume bytesPerPixel to be 4, you need to set either RGBA (or ARGB) when creating the context. Following is an example for RGBA without the alpha channel.

// RGBA format
let ctx = CGBitmapContextCreate(&data, pixelsWide, pixelsHigh, 8,   
    bitmapBytesPerRow, colorSpace, CGImageAlphaInfo.NoneSkipLast.rawValue)

According to the documentation, you have these options:

enum CGImageAlphaInfo : UInt32 {
    case None
    case PremultipliedLast
    case PremultipliedFirst
    case Last
    case First
    case NoneSkipLast
    case NoneSkipFirst
    case Only
}

Pixel Array to UIImage in Swift

Note: This is a solution for iOS creating a UIImage. For a solution for macOS and NSImage, see this answer.

Your only problem is that the data types in your PixelData structure need to be UInt8. I created a test image in a Playground with the following:

public struct PixelData {
    var a: UInt8
    var r: UInt8
    var g: UInt8
    var b: UInt8
}

var pixels = [PixelData]()

let red = PixelData(a: 255, r: 255, g: 0, b: 0)
let green = PixelData(a: 255, r: 0, g: 255, b: 0)
let blue = PixelData(a: 255, r: 0, g: 0, b: 255)

for _ in 1...300 {
    pixels.append(red)
}
for _ in 1...300 {
    pixels.append(green)
}
for _ in 1...300 {
    pixels.append(blue)
}

let image = imageFromARGB32Bitmap(pixels: pixels, width: 30, height: 30)

---

Update for Swift 4:

I updated imageFromARGB32Bitmap to work with Swift 4. The function now returns a UIImage? and guard is used to return nil if anything goes wrong.

func imageFromARGB32Bitmap(pixels: [PixelData], width: Int, height: Int) -> UIImage? {
    guard width > 0 && height > 0 else { return nil }
    guard pixels.count == width * height else { return nil }

    let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
    let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue)
    let bitsPerComponent = 8
    let bitsPerPixel = 32

    var data = pixels // Copy to mutable []
    guard let providerRef = CGDataProvider(data: NSData(bytes: &data,
                            length: data.count * MemoryLayout<PixelData>.size)
        )
        else { return nil }

    guard let cgim = CGImage(
        width: width,
        height: height,
        bitsPerComponent: bitsPerComponent,
        bitsPerPixel: bitsPerPixel,
        bytesPerRow: width * MemoryLayout<PixelData>.size,
        space: rgbColorSpace,
        bitmapInfo: bitmapInfo,
        provider: providerRef,
        decode: nil,
        shouldInterpolate: true,
        intent: .defaultIntent
        )
        else { return nil }

    return UIImage(cgImage: cgim)
}

---

Making it a convenience initializer for UIImage:

This function works well as a convenience initializer for UIImage. Here is the implementation:

extension UIImage {
    convenience init?(pixels: [PixelData], width: Int, height: Int) {
        guard width > 0 && height > 0, pixels.count == width * height else { return nil }
        var data = pixels
        guard let providerRef = CGDataProvider(data: Data(bytes: &data, count: data.count * MemoryLayout<PixelData>.size) as CFData)
            else { return nil }
        guard let cgim = CGImage(
            width: width,
            height: height,
            bitsPerComponent: 8,
            bitsPerPixel: 32,
            bytesPerRow: width * MemoryLayout<PixelData>.size,
            space: CGColorSpaceCreateDeviceRGB(),
            bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue),
            provider: providerRef,
            decode: nil,
            shouldInterpolate: true,
            intent: .defaultIntent)
        else { return nil }
        self.init(cgImage: cgim)
    }
}

Here is an example of its usage:

// Generate a 500x500 image of randomly colored pixels

let height = 500
let width = 500

var pixels: [PixelData] = .init(repeating: .init(a: 0, r: 0, g: 0, b: 0), count: width * height)
for index in pixels.indices {
    pixels[index].a = 255
    pixels[index].r = .random(in: 0...255)
    pixels[index].g = .random(in: 0...255)
    pixels[index].b = .random(in: 0...255)
}
let image = UIImage(pixels: pixels, width: width, height: height)

Getting RGBA values for all pixels of CGImage Swift

You can access the underlying pixels in a vImage buffer to do this.

For example, given an image named cgImage, use the following code to populate a vImage buffer:

guard
    let format = vImage_CGImageFormat(cgImage: cgImage),
    let buffer = try? vImage_Buffer(cgImage: cgImage,
                                    format: format) else {
        exit(-1)
    }

let rowStride = buffer.rowBytes / MemoryLayout<Pixel_8>.stride / format.componentCount

Note that a vImage buffer's data may be wider than the image (see: https://developer.apple.com/documentation/accelerate/finding_the_sharpest_image_in_a_sequence_of_captured_images) which is why I've added rowStride.

To access the pixels as a single buffer of interleaved values, use:

do {
    let n = rowStride * Int(buffer.height) * format.componentCount
    let start = buffer.data.assumingMemoryBound(to: Pixel_8.self)
    let ptr = UnsafeBufferPointer(start: start, count: n)
    
    print(Array(ptr)[ 0 ... 15]) // prints the first 15 interleaved values
}

To access the pixels as a buffer of Pixel_8888 values, use (make sure that format.componentCount is 4:

do {
    let n = rowStride * Int(buffer.height)
    let start = buffer.data.assumingMemoryBound(to: Pixel_8888.self)
    let ptr = UnsafeBufferPointer(start: start, count: n)
    
    print(Array(ptr)[ 0 ... 3]) // prints the first 4 pixels
}

How to get pixel data from a UIImage (Cocoa Touch) or CGImage (Core Graphics)?

FYI, I combined Keremk's answer with my original outline, cleaned-up the typos, generalized it to return an array of colors and got the whole thing to compile. Here is the result:

+ (NSArray*)getRGBAsFromImage:(UIImage*)image atX:(int)x andY:(int)y count:(int)count
{
    NSMutableArray *result = [NSMutableArray arrayWithCapacity:count];

    // First get the image into your data buffer
    CGImageRef imageRef = [image CGImage];
    NSUInteger width = CGImageGetWidth(imageRef);
    NSUInteger height = CGImageGetHeight(imageRef);
    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
    unsigned char *rawData = (unsigned char*) calloc(height * width * 4, sizeof(unsigned char));
    NSUInteger bytesPerPixel = 4;
    NSUInteger bytesPerRow = bytesPerPixel * width;
    NSUInteger bitsPerComponent = 8;
    CGContextRef context = CGBitmapContextCreate(rawData, width, height,
                    bitsPerComponent, bytesPerRow, colorSpace,
                    kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
    CGColorSpaceRelease(colorSpace);

    CGContextDrawImage(context, CGRectMake(0, 0, width, height), imageRef);
    CGContextRelease(context);

    // Now your rawData contains the image data in the RGBA8888 pixel format.
    NSUInteger byteIndex = (bytesPerRow * y) + x * bytesPerPixel;
    for (int i = 0 ; i < count ; ++i)
    {
        CGFloat alpha = ((CGFloat) rawData[byteIndex + 3] ) / 255.0f;
        CGFloat red   = ((CGFloat) rawData[byteIndex]     ) / alpha;
        CGFloat green = ((CGFloat) rawData[byteIndex + 1] ) / alpha;
        CGFloat blue  = ((CGFloat) rawData[byteIndex + 2] ) / alpha;
        byteIndex += bytesPerPixel;

        UIColor *acolor = [UIColor colorWithRed:red green:green blue:blue alpha:alpha];
        [result addObject:acolor];
    }

  free(rawData);

  return result;
}

Image get pixel RGBA

If you're printing it through NSLog, you can specify the format as such:

    UIImage *image = [UIImage imageNamed:@"Aurora"];
    CGImageRef cgImage = image.CGImage;
    CFDataRef data = CGDataProviderCopyData(CGImageGetDataProvider(cgImage));
    uint8_t *dataPtr = (uint8_t *) CFDataGetBytePtr(data);

    uint8_t components[4] = {
        dataPtr[0],
        dataPtr[1],
        dataPtr[2],
        dataPtr[3]
    };
    for (int i = 0; i < 4; i++) {
        NSLog(@"index %i val %hhu", i, components[i]);
    }

Which gives this output for me (for my example image):

index 0 val 20
index 1 val 39
index 2 val 55
index 3 val 255

If you're print it out on the lldb (using p/po/frame variable) there are certain default type formatters applied. You can see them using this command: type summary list

You can add your own custom summary using this command: type summary add

https://lldb.llvm.org/use/variable.html#type-summary

In this instance, you should be able to do something like this:

type summary add --summary-string "0=${var[0]%u}, 1=${var[1]%u}, 2=${var[2]%u}, 3=${var[3]%u}" 'uint8_t [4]'

Your defined type summary will be listed under the "default" category of type summary list.

Category: default
-----------------------
uint8_t [4]: `0=${var[0]%u}, 1=${var[1]%u}, 2=${var[2]%u}, 3=${var[3]%u}`

Then when you breakpoint and po components it will print out using the custom type summary defined:

(lldb) po components
0=20, 1=39, 2=55, 3=255

You can delete the type summary using the type summary delete <summary name> (i.e type summary delete "uint8_t [4]") or delete all custom type summaries using type summary clear.

There is most likely an easier way to do what you're trying to in the rest of the documentation from the link above, but using some of the custom format stuff in the document wasn't working right away for me.

---

Related Topics