Nsurlsession With Nsblockoperation and Queues

NSURLSession with NSBlockOperation and queues

The harshest criticisms of synchronous network requests are reserved for those who do it from the main queue (as we know that one should never block the main queue). But you're doing it on your own background queue, which addresses the most egregious problem with synchronous requests. But you're losing some wonderful features that asynchronous techniques provide (e.g. cancelation of requests, if needed).

I'll answer your question (how to make NSURLSessionDataTask behave synchronously) below, but I'd really encourage you to embrace the asynchronous patterns rather than fighting them. I'd suggest refactoring your code to use asynchronous patterns. Specifically, if one task is dependent upon another, simply put the initiation of the dependent task in the completion handler of the prior task.

If you have problems in that conversion, then post another Stack Overflow question, showing us what you tried, and we can try to help you out.

---

If you want to make an asynchronous operation synchronous, a common pattern is to use a dispatch semaphore so your thread that initiated the asynchronous process can wait for a signal from the completion block of the asynchronous operation before continuing. Never do this from the main queue, but if you're doing this from some background queue, it can be a useful pattern.

You can create a semaphore with:

dispatch_semaphore_t semaphore = dispatch_semaphore_create(0);

You can then have the completion block of the asynchronous process signal the semaphore with:

dispatch_semaphore_signal(semaphore);

And you can then have the code outside of the completion block (but still on the background queue, not the main queue) wait for that signal:

dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);

So, with NSURLSessionDataTask, putting that all together, that might look like:

[queue addOperationWithBlock:^{

    dispatch_semaphore_t semaphore = dispatch_semaphore_create(0);

    NSURLSession *session = [NSURLSession sharedSession]; // or create your own session with your own NSURLSessionConfiguration
    NSURLSessionTask *task = [session dataTaskWithURL:url completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
        if (data) {
            // do whatever you want with the data here
        } else {
            NSLog(@"error = %@", error);
        }

        dispatch_semaphore_signal(semaphore);
    }];
    [task resume];

    // but have the thread wait until the task is done

    dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);

    // now carry on with other stuff contingent upon what you did above
]);

With NSURLConnection (now deprecated), you have to jump through some hoops to initiate requests from a background queue, but NSURLSession handles it gracefully.

---

Having said that, using block operations like this means that the operations won't respond to cancellation events (while they're running, at least). So I generally eschew this semaphore technique with block operations and just wrap the data tasks in asynchronous NSOperation subclass. Then you enjoy the benefits of operations, but you can make them cancelable, too. It's more work, but a much better pattern.

For example:

//
//  DataTaskOperation.h
//
//  Created by Robert Ryan on 12/12/15.
//  Copyright © 2015 Robert Ryan. All rights reserved.
//

@import Foundation;
#import "AsynchronousOperation.h"

NS_ASSUME_NONNULL_BEGIN

@interface DataTaskOperation : AsynchronousOperation

/// Creates a operation that retrieves the contents of a URL based on the specified URL request object, and calls a handler upon completion.
///
/// @param  request                    A NSURLRequest object that provides the URL, cache policy, request type, body data or body stream, and so on.
/// @param  dataTaskCompletionHandler  The completion handler to call when the load request is complete. This handler is executed on the delegate queue. This completion handler takes the following parameters:
///
/// @returns                           The new session data operation.

- (instancetype)initWithRequest:(NSURLRequest *)request dataTaskCompletionHandler:(void (^)(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error))dataTaskCompletionHandler;

/// Creates a operation that retrieves the contents of a URL based on the specified URL request object, and calls a handler upon completion.
///
/// @param  url                        A NSURL object that provides the URL, cache policy, request type, body data or body stream, and so on.
/// @param  dataTaskCompletionHandler  The completion handler to call when the load request is complete. This handler is executed on the delegate queue. This completion handler takes the following parameters:
///
/// @returns                           The new session data operation.

- (instancetype)initWithURL:(NSURL *)url dataTaskCompletionHandler:(void (^)(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error))dataTaskCompletionHandler;

@end

NS_ASSUME_NONNULL_END

and

//
//  DataTaskOperation.m
//
//  Created by Robert Ryan on 12/12/15.
//  Copyright © 2015 Robert Ryan. All rights reserved.
//

#import "DataTaskOperation.h"

@interface DataTaskOperation ()

@property (nonatomic, strong) NSURLRequest *request;
@property (nonatomic, weak) NSURLSessionTask *task;
@property (nonatomic, copy) void (^dataTaskCompletionHandler)(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error);

@end

@implementation DataTaskOperation

- (instancetype)initWithRequest:(NSURLRequest *)request dataTaskCompletionHandler:(void (^)(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error))dataTaskCompletionHandler {
    self = [super init];
    if (self) {
        self.request = request;
        self.dataTaskCompletionHandler = dataTaskCompletionHandler;
    }
    return self;
}

- (instancetype)initWithURL:(NSURL *)url dataTaskCompletionHandler:(void (^)(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error))dataTaskCompletionHandler {
    NSURLRequest *request = [NSURLRequest requestWithURL:url];
    return [self initWithRequest:request dataTaskCompletionHandler:dataTaskCompletionHandler];
}

- (void)main {
    NSURLSessionTask *task = [[NSURLSession sharedSession] dataTaskWithRequest:self.request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
        self.dataTaskCompletionHandler(data, response, error);
        [self completeOperation];
    }];

    [task resume];
    self.task = task;
}

- (void)completeOperation {
    self.dataTaskCompletionHandler = nil;
    [super completeOperation];
}

- (void)cancel {
    [self.task cancel];
    [super cancel];
}

@end

Where:

//
//  AsynchronousOperation.h
//

@import Foundation;

@interface AsynchronousOperation : NSOperation

/// Complete the asynchronous operation.
///
/// This also triggers the necessary KVO to support asynchronous operations.

- (void)completeOperation;

@end

And

//
//  AsynchronousOperation.m
//

#import "AsynchronousOperation.h"

@interface AsynchronousOperation ()

@property (nonatomic, getter = isFinished, readwrite)  BOOL finished;
@property (nonatomic, getter = isExecuting, readwrite) BOOL executing;

@end

@implementation AsynchronousOperation

@synthesize finished  = _finished;
@synthesize executing = _executing;

- (instancetype)init {
    self = [super init];
    if (self) {
        _finished  = NO;
        _executing = NO;
    }
    return self;
}

- (void)start {
    if ([self isCancelled]) {
        self.finished = YES;
        return;
    }

    self.executing = YES;

    [self main];
}

- (void)completeOperation {
    self.executing = NO;
    self.finished  = YES;
}

#pragma mark - NSOperation methods

- (BOOL)isAsynchronous {
    return YES;
}

- (BOOL)isExecuting {
    @synchronized(self) {
        return _executing;
    }
}

- (BOOL)isFinished {
    @synchronized(self) {
        return _finished;
    }
}

- (void)setExecuting:(BOOL)executing {
    @synchronized(self) {
        if (_executing != executing) {
            [self willChangeValueForKey:@"isExecuting"];
            _executing = executing;
            [self didChangeValueForKey:@"isExecuting"];
        }
    }
}

- (void)setFinished:(BOOL)finished {
    @synchronized(self) {
        if (_finished != finished) {
            [self willChangeValueForKey:@"isFinished"];
            _finished = finished;
            [self didChangeValueForKey:@"isFinished"];
        }
    }
}

@end

NSBlockOperation, NSOperationQueue and Blocks

The issue is that NSBlockOperation is for synchronous blocks. It will be finished as soon as its block(s) have finished executing. If its block(s) fire off asynchronous methods, those will run independently.

For example, when your syncUserInfoOperation's block is executed, it fires off [dataSync syncUserInfo:...] and then considers itself done; it doesn't wait for any of the completion handlers to fire, or anything like that.

A good solution to this is to create your own NSOperation subclasses. You'd probably want to create one for each of your data sync types to make it easier to setup dependencies, etc., but that's up to you. You can read all about how to do that here (be sure to read the section on "Configuring Operations for Concurrent Execution").

You could also make a generic NSOperation subclass that takes a block that can be run asynchronously. The main issue with that is it makes it much harder to handle things like canceling the operation, which you probably still want.

How to execute multiple NSURLSessionDataTask serially?

Are you using those in main queue? Do not run the callback blocks in the same queue as dispatch_semaphore_wait, because dispatch_semaphore_wait will block the queue and the callback will not be executed, which will cause dead lock.

NSURLSession Delegate Queue

It looks like, despite what the getter for delegateQueue says, the NSURLSession is indeed using your NSOperationQueue. I added KVO for the "operations" property on the queue:

[queue addObserver:self forKeyPath:@"operations" options:NSKeyValueObservingOptionNew context:NULL];

And added the following method:

- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context
{
    NSLog(@"%@%@", object, change);
    NSOperationQueue *queue = (NSOperationQueue *)object;

    NSLog(@"The queue in KVO: %@", queue);
    NSLog(@"Max op Count in KVO: %i", queue.maxConcurrentOperationCount);

    NSLog(@"%@", self.session.delegateQueue);

}

And it prints:

2013-09-29 07:45:13.751 sotest1[17214:1403] {name = 'NSOperationQueue 0x895e0c0'}
2013-09-29 07:45:13.757 sotest1[17214:4207] {name = 'NSOperationQueue 0x98a0940'}{
    kind = 1;
    new =     (
        ""
    );
}
2013-09-29 07:45:13.757 sotest1[17214:4207] The queue in KVO: {name = 'NSOperationQueue 0x98a0940'}
2013-09-29 07:45:13.757 sotest1[17214:4207] Max op Count in KVO: 5

So you can see the delegate does indeed get processed by your queue, despite the fact that the getter says otherwise. Weird.

Btw, the way you're doing it is also exactly AFNetworking does it, which is generally a good sign: https://github.com/AFNetworking/AFNetworking/blob/master/AFNetworking/AFURLSessionManager.m#L287

Deadlock inside NSURLSession delegate queue

After commenting most of my code I isolated the code causing the problem and it does not related at all to alamofire or NSURLSession.

I have in my own code a call to objc_sync_enter on an array (of objects), it always has a matching objc_sync_exit call on the same array. after changing this call to be on self instead of this array, the deadlock inside NSBlockOperation is gone. It may be related to the fact that an array is not an object but a struct. So if you experience very strange deadlock in your code, I suggest that before you try anything else, make sure you don't have calls of objc_sync_enter on structs.

Learning NSBlockOperation

I am not an expert in NSOperation or NSOperationQueues but I think below code is a bit better although I think it has some caveats still. Probably enough for some purposes but is not a general solution for concurrency:

- (NSOperation *)executeBlock:(void (^)(void))block
                      inQueue:(NSOperationQueue *)queue
                   completion:(void (^)(BOOL finished))completion
{
    NSOperation *blockOperation = [NSBlockOperation blockOperationWithBlock:block];
    NSOperation *completionOperation = [NSBlockOperation blockOperationWithBlock:^{
        completion(blockOperation.isFinished);
    }];
    [completionOperation addDependency:blockOperation];

    [[NSOperationQueue currentQueue] addOperation:completionOperation];
    [queue addOperation:blockOperation];
    return blockOperation;
}

Now lets use it:

- (void)tryIt
{
    // Create and configure the queue to enqueue your operations
    backgroundOperationQueue = [[NSOperationQueue alloc] init];

    // Prepare needed data to use in the operation
    NSMutableString *string = [NSMutableString stringWithString:@"tea"];
    NSString *otherString = @"for";

    // Create and enqueue an operation using the previous method
    NSOperation *operation = [self executeBlock:^{
        NSString *yetAnother = @"two";
        [string appendFormat:@" %@ %@", otherString, yetAnother];
    }
    inQueue:backgroundOperationQueue 
    completion:^(BOOL finished) {
        // this logs "tea for two"
        NSLog(@"%@", string);
    }];

    // Keep the operation for later uses
    // Later uses include cancellation ...
    [operation cancel]; 
}

Some answers to your questions:

1. Cancelation. Usually you subclass NSOperation so you can check self.isCancelled and return earlier. See this thread, it is a good example. In current example you cannot check if the operation has being cancelled from the block you are supplying to make an NSBlockOperation because at that time there is no such an operation yet. Cancelling NSBlockOperations while the block is being invoked is apparently possible but cumbersome. NSBlockOperations are for specific easy cases. If you need cancellation you are better subclassing NSOperation :)

2. I don't see a problem here. Although note two things. a)I changed the method do to run the completion block in current queue b)a queue is required as a parameter. As @Mike Weller said, you should better supply background queue so you don't need to create one per each operation and can choose what queue to use to run your stuff :)

3. I think yes, you should make string atomic. One thing you should not forget is that if you supply several operations to the queue they might not run in that order (necessarily) so you could end up with a very strange message in your string. If you need to run one operation at a time serially you can do: [backgroundOperation setMaxConcurrentOperationCount:1]; before start enqueuing your operations.
   There is a reading-worthy note in the docs though:

   Additional Operation Queue Behaviors
   An operation queue executes its queued operation objects based on their priority and readiness. If all of the queued operation objects have the same priority and are ready to execute when they are put in the queue—that is, their isReady method returns YES—they are executed in the order in which they were submitted to the queue. For a queue whose maximum number of concurrent operations is set to 1, this equates to a serial queue. However, you should never rely on the serial execution of operation objects. Changes in the readiness of an operation can change the resulting execution order.

4. I think after reading these lines you know :)

How to know when a for loop with NSURLSessionDataTasks is complete

There are a number of options. The fundamental issue is that these individual data tasks run asynchronously, so you need some way to keep track of these asynchronous tasks and establish some dependency on their completion.

There are several possible approaches:

1. The typical solution is to employ a dispatch group. Enter the group before you start the request with dispatch_group_enter, leave the group with dispatch_group_leave inside the completion handler, which is called asynchronously, and then, at the end of the loop, supply a dispatch_group_notify block that will be called asynchronously when all of the "enter" calls are offset by corresponding "leave" calls:

   - (void)findStationsByRoute {
       dispatch_group_t group = dispatch_group_create();
   
       for (NSString *stopID in self.allRoutes) {
           NSString     *urlString = [NSString stringWithFormat:@"http://truetime.csta.com/developer/api/v1/stopsbyroute?route=%@", stopID];
           NSURL        *url       = [NSURL URLWithString:urlString];
           NSURLRequest *request   = [NSURLRequest requestWithURL:url];
   
           dispatch_group_enter(group);   // enter group before making request
   
           NSURLSessionDataTask *task = [self.session dataTaskWithRequest:request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
               NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *)response;
               if(httpResponse.statusCode == 200){
                   NSError *jsonError;   // Note, do not initialize this with [[NSError alloc]init];
                   NSDictionary *stopLocationDictionary = [NSJSONSerialization JSONObjectWithData:data options:0 error:&jsonError];
                   NSArray *stopDirectionArray = [stopLocationDictionary objectForKey:@"direction"];
                   for (NSDictionary *stopDictionary in stopDirectionArray) {
                       NSArray *stop = [stopDictionary objectForKey:@"stop"];
                       [self.arrayOfStops addObject:stop];
                   }
               }
   
               dispatch_group_leave(group);  // leave group from within the completion handler
           }];
   
           [task resume];
       }
   
       dispatch_group_notify(group, dispatch_get_main_queue(), ^{
           // do something when they're all done
       });
   }
   

2. A more sophisticated way to handle this is to wrap the NSSessionDataTask in a NSOperation subclass and you can then use dependencies between your data task operations and your final completion operation. You'll want to ensure your individual data task operations are "concurrent" ones (i.e. do not issue isFinished notification until the asynchronous data task is done). The benefit of this approach is that you can set maxConcurrentOperationCount to constrain how many requests will be started at any given time. Generally you want to constrain it to 3-4 requests at a time.

   Note, this can also address timeout issues from which the dispatch group approach can suffer from. Dispatch groups don't constrain how many requests are submitted at any given time, whereas this is easily accomplished with NSOperation.

   For more information, see the discussion about "concurrent operations" in the Operation Queue section of the Concurrency Programming Guide.

   For an example of wrapping NSURLSessionTask requests in asynchronous NSOperation subclass, see a simple implementation the latter half NSURLSession with NSBlockOperation and queues. This question was addressing a different topic, but I include a NSOperation subclass example at the end.

3. If instead of data tasks you used upload/download tasks, you could then use a [NSURLSessionConfiguration backgroundSessionConfiguration] and URLSessionDidFinishEventsForBackgroundURLSession: of your NSURLSessionDelegate would then get called when all of the tasks are done and the app is brought back into the foreground. (A little annoyingly, though, this is only called if your app was not active when the downloads finished: I wish there was a rendition of this delegate method that was called even if the app was in the foreground when the downloads finished.)

   While you asked about data tasks (which cannot be used with background sessions), using background session with upload/download tasks enjoys a significant advantage of background operation. If your process really takes 10 minutes (which seems extraordinary), refactoring this for background session might offer significant advantages.

4. I hate to even mention this, but for the sake a completeness, I should acknowledge that you could theoretically just by maintain an mutable array or dictionary of pending data tasks, and upon the completion of every data task, remove an item from that list, and, if it concludes it is the last task, then manually initiate the completion process.

---

Related Topics