Xctest and Asynchronous Testing in Xcode 6

XCTest and asynchronous testing in Xcode 6

The sessions video is perfect, basically you want to do something like this

func testFetchNews() {
    let expectation = self.expectationWithDescription("fetch posts")

    Post.fetch(.Top, completion: {(posts: [Post]!, error: Fetcher.ResponseError!) in
        XCTAssert(true, "Pass")
        expectation.fulfill()
    })

    self.waitForExpectationsWithTimeout(5.0, handler: nil)
}

How can I get XCTest to wait for async calls in setUp before tests are run?

There are two techniques for running asynchronous tests. XCTestExpectation and semaphores. In the case of doing something asynchronous in setUp, you should use the semaphore technique:

override func setUp() {
    super.setUp()

    // Fill out a database with data. I can make this call do anything, here
    // it returns a block.

    let data = getData()

    let semaphore = DispatchSemaphore(value: 0)

    db.overwriteDatabase(data) {

        // do some stuff

        semaphore.signal()
    }

    semaphore.wait()
}

Note, for that to work, this onDone block cannot run on the main thread (or else you'll deadlock).

---

If this onDone block runs on the main queue, you can use run loops:

override func setUp() {
    super.setUp()

    var finished = false

    // Fill out a database with data. I can make this call do anything, here
    // it returns a block.

    let data = getData()

    db.overwriteDatabase(data) {

        // do some stuff

        finished = true
    }

    while !finished {
        RunLoop.current.run(mode: .default, before: Date.distantFuture)
    }
}

This is a very inefficient pattern, but depending upon how overwriteDatabase was implemented, it might be necessary

Note, only use this pattern if you know that onDone block runs on the main thread (otherwise you'll have to do some synchronization of finished variable).

Testing asynchronous call in unit test in iOS

Try KIWI framework. It's powerful and might help you with other kinds of tests.

XCTest and NSRunLoop for async tests

There are a few misconceptions regarding your example code:

First off, if the completion handler will be executed on a different thread than where the call-site executes, the compiler will create code that is "undefined behavior". This is due to modifying the variable returnedModel in thread "A" and reading the value in thread "M". This is a classic "data race" which produces undefined behavior (please read more in the C and C++ specification).

The __block modifier may alleviate this issue, but I don't believe clang takes special actions here. In the worst case, the thread reading the value (the main thread) never "sees" an update of the value performed through the handler or it reads "garbage".

Another problem with this approach requires a more thorough understanding how Run Loops do actually work. In your example, in the worst case, the run loop's method runMode:beforeDate: will only return when the timeout expired - that is after 10 secs. It may return earlier only if there was an event processed on this mode - possibly unrelated to the test code.

So in short, this approach isn't really suited to accomplish the task. But other "flavors" may indeed work.

According your questions:

Q1: No.

The reason is probably, that XCTest is actually quite old (its just another name for SenTest), and code at the time where it was invented had probably no such fancy stuff like "asynchronous operations", "blocks" and "completion handlers". So there's no built-in solution for this task.

Q2: I don't quite understand this questions. But we might assume that "matchers" (aka "assert something") use exceptions when the test fails. Those require to be executed on the main thread, where there is a catch handler implemented by the underlying test implementation. Maybe XCTest doesn't use exceptions - however, other Unit Test libraries may indeed use exceptions - for example "Cedar". That means, if you execute a completion handler on some queue, and a matcher throws an exception, it MUST be executed on the main thread. (bummer).

Q3: Perhaps the exception issue? But I have no idea. Possibly there's another issue. You may provide more information.

The other "side" effects may be "race conditions" or other issues. But unless you provide more detailed info I'm guessing ;)

Whether or not there is a need to "test async" really depends on what you are actually testing:

For example, if you use a well known third party network library which has a completion handler, would you really want to test whether the handler will be invoked? (Probably not, since you wouldn't want to actually test the network library).

But if you implemented your own asynchronous operation which reports the result via a completion handler, you actually may want to test whether a completion handler will be invoked.

Asynchronous Performance Tests with XCTest

With some help from Apple, I have a solution. Silly oversight on my part as this is very easy to solve. To get to work, all you need to do is put the creating of the expectation object (clsQueryReturnedExpectation) inside the measureMetrics block so it is created afresh each time the performance test is run.

PFCLSClient *theClient = [[PFCLSClient alloc] init];

[self measureMetrics:@[XCTPerformanceMetric_WallClockTime] automaticallyStartMeasuring:YES forBlock: ^{
    XCTestExpectation *clsQueryReturnedExpectation = [self expectationWithDescription:@"clsQuery returned"];  
    [theClient getStructureOfType:clsImageTypeSVG ForID:idString success: ^(NSDictionary *structureInfo) {
       [clsQueryReturnedExpectation fulfill];
    } failure: ^(NSError *error) {
       XCTFail();
       [clsQueryReturnedExpectation fulfill];
    }];

   [self waitForExpectationsWithTimeout:5 handler: ^(NSError *error) {
        [self stopMeasuring];
   }];
}];

How to test a sendAsynchronousRequest: on XCTest

This looks like exactly what you need:

XCAsyncTestCase: Asynchronous capable SenTestCase subclass.

Basically, you should write your test like this:

- (void)testConnection
{
    [[Conector sharedInstance] performAsynchronousRequestWithServerRequest:_srvRequest completionHandler:^(RequestAsynchronousStatus finishStatus, NSData *data) {
        if (finishStatus == RequestAsynchronousOK)
        {
            _data = data;
            [self notify:XCTAsyncTestCaseStatusSucceeded];
            NSLog(@"Data OK");
        }
    }];

    [self waitForTimeout:10];

    XCTAssertNotNil(_data, @"Data was nil");
}

Notice the waitForTimeout: call and the notify: calls. 10 seconds should be enough for the test, though it would depend on the request itself.

You could even get more specific and wait for a certain status, like so:

[self waitForStatus: XCTAsyncTestCaseStatusSucceeded timeout:10];

This way, if the connection fails to notify the XCTAsyncTestCaseStatusSucceeded status, the wait call will timeout and your test would fail (as it should).

Swift ASYNC Event Handling With Xcode UI Test

Stephen is right, the UI interruption monitor should work nicely. I recommend adding it to your test set up so it runs for all of your tests.

class UITests: XCTestCase {
    let app = XCUIApplication()

    override func setUp() {
        super.setUp()
        addUIInterruptionMonitorWithDescription("Alert") { (alert) -> Bool in
            alert.buttons["OK"].tap()
            return true
        }
        app.launch()
    }

    func testFoo() {
        // example test
    }
}

---

Related Topics