Convert an Uiimage in a Texture

Convert an UIImage in a texture

I haven't test the following but i will decompose the conversion in 3 steps:

1. Extract info for your image:

   UIImage* image = [UIImage imageNamed:@"imageToApplyAsATexture.png"];
   CGImageRef imageRef = [image CGImage];
   int width = CGImageGetWidth(imageRef);
   int height = CGImageGetHeight(imageRef);
   

2. Allocate a textureData with the above properties:

   GLubyte* textureData = (GLubyte *)malloc(width * height * 4); // if 4 components per pixel (RGBA)
   
   CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
   NSUInteger bytesPerPixel = 4;
   NSUInteger bytesPerRow = bytesPerPixel * width;
   NSUInteger bitsPerComponent = 8;  
   CGContextRef context = CGBitmapContextCreate(textureData, width, height,
                                                bitsPerComponent, bytesPerRow, colorSpace,
                                                kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
   
   CGColorSpaceRelease(colorSpace);
   
   CGContextDrawImage(context, CGRectMake(0, 0, width, height), imageRef);
   CGContextRelease(context);
   

3. Set-up your texture:

   GLuint textureID;    
   glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
   glGenTextures(1, &textureID);
   
   glBindTexture(GL_TEXTURE_2D, textureID);
   glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, textureData);
   

EDIT:

Read this tut; everything is explained from the conversion of one image to a texture and applying a texture in an iOS environment.

UIImage to MTLTexture when downloaded as PNG

In Metal coordinates are reversed. However, you now have a much simpler way to load textures with MTKTextureLoader:

import MetalKit

let textureLoader = MTKTextureLoader(device: device)
let texture: MTLTexture = textureLoader.newTextureWithContentsOfURL(filePath, options: nil)

This will create a new texture for you with the appropriate coordinates using the image located at filePath. If you don't want to use a NSURL you also have the newTextureWithData and newTextureWithCGImage options.

In UIImage to Texture conversion, Texture is flipped vertically

This two lines fixed the problem

CGAffineTransform flipVertical = CGAffineTransformMake(1, 0, 0, -1, 0,height);
CGContextConcatCTM(context, flipVertical);

iOS, SpriteKit convert SKTextureAtlas/SKTexture into UIImage

Ok, After weekend I finally got it working.
Important part is

guard let render = view.texture(from: sprite) else { return UIImage() }
return UIImage(cgImage: render.cgImage())

which I got thanks to this answer

Full working code:

func image(texture: SKTexture) -> UIImage {
    let view = SKView(frame:CGRect(x: 0, y: 0, width: texture.size().width, height: texture.size().height))
    let scene = SKScene(size: texture.size())
    let sprite  = SKSpriteNode(texture: texture)
    sprite.position = CGPoint(x: view.frame.midX, y: view.frame.midY)
    scene.addChild(sprite)
    view.presentScene(scene)

    guard let render = view.texture(from: sprite) else { return UIImage() }
    return UIImage(cgImage: render.cgImage())
}

The reason I was looking to make it work is that I'm writing game with SpriteKit however UI is done with SwiftUI.

How to get UIImage from texture of an EAGLView?

http://www.bit-101.com/blog/?p=1861

MTLTexture displays UIImages incorrectly if converted to textures

There are two separate concerns here: matching the pixel format between your render pipeline state and your framebuffer, and matching the pixel format between your image data and the pixel format of the texture you create from it.

In the first case, it's sensible to use MTLPixelFormatBGRA8Unorm. This has to be set on your CAMetalLayer (or MTKView) as well as your render pipeline state's primary color attachment. If they don't agree, you'll get the exception you mentioned. This configuration looks something like this:

metalView.colorPixelFormat = MTLPixelFormatBGRA8Unorm;
// and elsewhere...
pipelineStateDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormatBGRA8Unorm;

In the latter case, what matters is that the component ordering of your image data matches the format you choose for your texture. The colorspace you get from your UIImage is sRGB, so you need to specify MTLPixelFormatRGBA8Unorm as your texture's pixel format (where you currently have MTLPixelFormatBGRA8Unorm in your texture creation code), or swap the bytes in the red and blue channels manually.

The pixel formats of your view and the texture you create from the image don't have to be the same. Metal swizzles colors when you sample textures and when you write to the framebuffer, so within your shaders, you can always assume an RGBA ordering.

Add UIImage as texture to a Plane in RealityKit

Currently, you cannot use UIImage or CIImage as a shader's texture in RealityKit 2.0. In both versions of RealityKit, the texture must be loaded via the String type parameter of the load() method.

RealityKit 2.0

To assign a texture to a shader in RealityKit 2.0 use the following approach:

let mesh: MeshResource = .generatePlane(width: 0.45, depth: 0.45)
        
var material = SimpleMaterial()
material.color = .init(tint: .white.withAlphaComponent(0.999),
                    texture: .init(try! .load(named: "texture.png")))
material.metallic = .float(1.0)
material.roughness = .float(0.0)

let model = ModelEntity(mesh: mesh, materials: [material])

RealityKit 1.0

To assign a texture to a shader in RealityKit 1.0 use this approach:

let scene = try! Experience.loadMyScene()

var material = SimpleMaterial()
material.baseColor = try! .texture(.load(named: "texture.png"))
material.metallic = MaterialScalarParameter(floatLiteral: 1.0)
material.roughness = MaterialScalarParameter(floatLiteral: 0.0)
material.tintColor = UIColor.white

let mesh: MeshResource = .generatePlane(width: 0.45, depth: 0.45)
let component = ModelComponent(mesh: mesh, materials: [material])

scene.myFavoriteScene?.children[0].components.set(component)
arView.scene.anchors.append(scene)

CGImage

Nonetheless, you can create a texture resource from an in-memory Core Graphics image:

static func generate(from: CGImage, 
                 withName: String?, 
                  options: TextureResource.CreateOptions) -> TextureResource

Also, you can use a URL parameter:

material.color.texture = .init(try! .load(contentsOf: url))  // RealityKit 2.0

---

Related Topics