Why Does Swift Return an Unexpected Pointer When Converting an Optional String into an Unsafepointer

Swift flatMap gives unexpected result while using with optional array

The issue is that for the purposes of map and flatMap, optionals are collections of 0 or 1 elements. You can directly call map and flatMap on optionals without unwrapping them:

let foo: Int? = 5
foo.map { $0 * $0 } // Int? = 25; "collection of one element"
let bar: Int? = nil
bar.map { $0 * $0 } // Int? = nil; "collection of zero elements"

To illustrate in more familiar terms your current situation, you're looking at the equivalent of this:

class Person {
    let cars: [[String]]
}

If you had a var persons: [Person] and called persons.flatMap { $0.cars }, the resulting type of this operation would unquestionably be [[String]]: you start out with three layers of collections and you end up with two.

This is also effectively what is happening with [String]? instead of [[String]].

In the case that you are describing, I would advise dropping the optional and using empty arrays. I'm not sure that the distinction between a nil array and an empty array is truly necessary in your case: my interpretation is that a nil array means that the person is incapable of owning a car, whereas an empty array means that the person is capable of owning a car but doesn't have any.

If you cannot drop the optional, then you will need to call flatMap twice to flatten two layers instead of only one:

persons.flatMap { $0.cars }.flatMap { $0 }

Why its impossible to cast `(()- Void)` to `(()- Void)?`

When a closure is wrapped inside an optional, that closure "escapes". If you don't know what "escapes" means, read this answer of mine.

The non-optional closure in the second getData, however, is not @escaping. So you are trying to make a non-escaping closure escape! The error message is kind of confusing here, but if you pass .some(callback) rather than callback directly, it becomes clear what's happening:

Converting non-escaping parameter 'callback' to generic parameter 'Wrapped' may allow it to escape

So you should mark callback in your second getData as @escaping:

func getData(id: CLong, callback: @escaping ((Dictionary<String, Any>) -> Void) ) {

Why does calling removeAtIndex: on a string invert the result if called within a string literal?

This has nothing to do with whether you call the function inside of a
string literal or not. The String method

public mutating func removeAtIndex(i: Index) -> Character

removes the character at the given index from the string and
returns that character as the function result.

var user = "@pdxCorey"
let firstChar = user.removeAtIndex(user.startIndex)

print(user)      // pdxCorey
print(firstChar) // @

In the first case you are printing the value of user after removing the
first character, that gives "pdxCorey".

In the second case you are printing the return value of removeAtIndex()
which is the removed character "@".

UnsafeMutablePointer.pointee and didSet properties

Any time you see a construct like UnsafeMutablePointer(&intObservingThing.anInt), you should be extremely wary about whether it'll exhibit undefined behaviour. In the vast majority of cases, it will.

First, let's break down exactly what's happening here. UnsafeMutablePointer doesn't have any initialisers that take inout parameters, so what initialiser is this calling? Well, the compiler has a special conversion that allows a & prefixed argument to be converted to a mutable pointer to the 'storage' referred to by the expression. This is called an inout-to-pointer conversion.

For example:

func foo(_ ptr: UnsafeMutablePointer<Int>) {
  ptr.pointee += 1
}

var i = 0
foo(&i)
print(i) // 1

The compiler inserts a conversion that turns &i into a mutable pointer to i's storage. Okay, but what happens when i doesn't have any storage? For example, what if it's computed?

func foo(_ ptr: UnsafeMutablePointer<Int>) {
  ptr.pointee += 1
}

var i: Int {
  get { return 0 }
  set { print("newValue = \(newValue)") }
}
foo(&i)
// prints: newValue = 1

This still works, so what storage is being pointed to by the pointer? To solve this problem, the compiler:

1. Calls i's getter, and places the resultant value into a temporary variable.
2. Gets a pointer to that temporary variable, and passes that to the call to foo.
3. Calls i's setter with the new value from the temporary.

Effectively doing the following:

var j = i // calling `i`'s getter
foo(&j)
i = j     // calling `i`'s setter

It should hopefully be clear from this example that this imposes an important constraint on the lifetime of the pointer passed to foo – it can only be used to mutate the value of i during the call to foo. Attempting to escape the pointer and using it after the call to foo will result in a modification of only the temporary variable's value, and not i.

For example:

func foo(_ ptr: UnsafeMutablePointer<Int>) -> UnsafeMutablePointer<Int> {
  return ptr
}

var i: Int {
  get { return 0 }
  set { print("newValue = \(newValue)") }
}
let ptr = foo(&i)
// prints: newValue = 0
ptr.pointee += 1

ptr.pointee += 1 takes place after i's setter has been called with the temporary variable's new value, therefore it has no effect.

Worse than that, it exhibits undefined behaviour, as the compiler doesn't guarantee that the temporary variable will remain valid after the call to foo has ended. For example, the optimiser could de-initialise it immediately after the call.

Okay, but as long as we only get pointers to variables that aren't computed, we should be able to use the pointer outside of the call it was passed to, right? Unfortunately not, turns out there's lots of other ways to shoot yourself in the foot when escaping inout-to-pointer conversions!

To name just a few (there are many more!):

• A local variable is problematic for a similar reason to our temporary variable from earlier – the compiler doesn't guarantee that it will remain initialised until the end of the scope it's declared in. The optimiser is free to de-initialise it earlier.

  For example:

  func bar() {
    var i = 0
    let ptr = foo(&i)
    // Optimiser could de-initialise `i` here.
  
    // ... making this undefined behaviour!
    ptr.pointee += 1
  }
  

• A stored variable with observers is problematic because under the hood it's actually implemented as a computed variable that calls its observers in its setter.

  For example:

  var i: Int = 0 {
    willSet(newValue) {
      print("willSet to \(newValue), oldValue was \(i)")
    }
    didSet(oldValue) {
      print("didSet to \(i), oldValue was \(oldValue)")
    }
  }
  

  is essentially syntactic sugar for:

  var _i: Int = 0
  
  func willSetI(newValue: Int) {
    print("willSet to \(newValue), oldValue was \(i)")
  }
  
  func didSetI(oldValue: Int) {
    print("didSet to \(i), oldValue was \(oldValue)")
  }
  
  var i: Int {
    get {
      return _i
    }
    set {
      willSetI(newValue: newValue)
      let oldValue = _i
      _i = newValue
      didSetI(oldValue: oldValue)
    }
  }
  

• A non-final stored property on classes is problematic as it can be overridden by a computed property.

And this isn't even considering cases that rely on implementation details within the compiler.

For this reason, the compiler only guarantees stable and unique pointer values from inout-to-pointer conversions on stored global and static stored variables without observers. In any other case, attempting to escape and use a pointer from an inout-to-pointer conversion after the call it was passed to will lead to undefined behaviour.

---

Okay, but how does my example with the function foo relate to your example of calling an UnsafeMutablePointer initialiser? Well, UnsafeMutablePointer has an initialiser that takes an UnsafeMutablePointer argument (as a result of conforming to the underscored _Pointer protocol which most standard library pointer types conform to).

This initialiser is effectively same as the foo function – it takes an UnsafeMutablePointer argument and returns it. Therefore when you do UnsafeMutablePointer(&intObservingThing.anInt), you're escaping the pointer produced from the inout-to-pointer conversion – which, as we've discussed, is only valid if it's used on a stored global or static variable without observers.

So, to wrap things up:

var intObservingThing = IntObservingThing(anInt: 0)
var otherPtr = UnsafeMutablePointer(&intObservingThing.anInt)
// "I was just set to 0."

otherPtr.pointee = 20

is undefined behaviour. The pointer produced from the inout-to-pointer conversion is only valid for the duration of the call to UnsafeMutablePointer's initialiser. Attempting to use it afterwards results in undefined behaviour. As matt demonstrates, if you want scoped pointer access to intObservingThing.anInt, you want to use withUnsafeMutablePointer(to:).

I'm actually currently working on implementing a warning (which will hopefully transition to an error) that will be emitted on such unsound inout-to-pointer conversions. Unfortunately I haven't had much time lately to work on it, but all things going well, I'm aiming to start pushing it forwards in the new year, and hopefully get it into a Swift 5.x release.

In addition, it's worth noting that while the compiler doesn't currently guarantee well-defined behaviour for:

var normalThing = NormalThing(anInt: 0)
var ptr = UnsafeMutablePointer(&normalThing.anInt)
ptr.pointee = 20

From the discussion on #20467, it looks like this will likely be something that the compiler does guarantee well-defined behaviour for in a future release, due to the fact that the base (normalThing) is a fragile stored global variable of a struct without observers, and anInt is a fragile stored property without observers.

Swift getaddrinfo

Memory allocated by getaddrinfo (e.g. by malloc) will not be given to any other
dynamic memory allocation function in the same running process
until released by freeaddrinfo (e.g. by free).
Therefore the Swift runtime will not trample on that memory (if we
assume that it has no programming errors such as wrong pointer calculations).

Also struct addrinfo is a value type, so

result.append(ai!)

will append a copy of the pointed-to structure to the array.

But there is still a problem. Some members of struct addrinfo
are pointers

public var ai_canonname: UnsafeMutablePointer<Int8>! /* canonical name for hostname */
public var ai_addr: UnsafeMutablePointer<sockaddr>! /* binary address */

which may point into the memory allocated by getaddrinfo and therefore
invalid after freeaddrinfo, and dereferencing them after your
function returns causes undefined behaviour.

Therefore you must either postpone the freeaddrinfo until the
address list is not needed anymore, or copy the information.
This is a bit cumbersome because ai_addr may point to a
IPv4 or IPv6 socket address structure which have different length.

The following code demonstrates how the address list can be copied
to an array of sockaddr_storage structures (which are large enough
to hold any IP address). This code has not been thoroughly tested,
so use it with care.

public func getaddrinfo(node: String, service: String, hints: addrinfo?) throws -> [sockaddr_storage] {
    var err: Int32
    var res: UnsafeMutablePointer<addrinfo>?
    if var hints = hints {
        err = getaddrinfo(node, service, &hints, &res)
    } else {
        err = getaddrinfo(node, service, nil, &res)
    }
    if err == EAI_SYSTEM {
        throw SystemError.getaddrinfo(err, errno)
    }
    if err != 0 {
        throw SystemError.getaddrinfo(err, nil)
    }
    defer {
        freeaddrinfo(res)
    }
    guard let firstAddr = res else {
        return []
    }

    var result = [sockaddr_storage]()
    for addr in sequence(first: firstAddr, next: { $0.pointee.ai_next }) {
        var sockAddr = sockaddr_storage()
        memcpy(&sockAddr, addr.pointee.ai_addr, Int(addr.pointee.ai_addrlen))
        result.append(sockAddr)
    }
    return result
}

Remarks:

• If you are only interested in a single address family then
  you can replace sockaddr_storage by sockaddr_in or
  sockaddr_in6.

• I have made the node and service parameters non-optional.
  The reason is that Swift currently has a Bug when passing more
  than one optional String to a C function, see
  Why does Swift return an unexpected pointer when converting an optional String into an UnsafePointer?.

• sequence() is used here to traverse the linked list instead of
  a while loop.

How to handle UnsafePointer Unmanaged CFArray ?

I am a little bit guessing (because I have no experience with the Accessibility
functions), but from the function declaration it should work like this:

let element: AXUIElementRef = ...

var ptr : Unmanaged<CFArray>? = nil
let error = AXUIElementCopyAttributeNames(element, &ptr)
if error == AXError(kAXErrorSuccess) {
    let names = ptr!.takeRetainedValue() // gives a CFArray
    // ...
}

Update for Swift 3 (untested):

let element: AXUIElement = ...

var cfArray: CFArray?
let error = AXUIElementCopyAttributeNames(element, &cfArray)
if error == .success, let names = cfArray as? [String] {
    // names is [String] array ...
}

---

Related Topics