Finish All Asynchronous Requests Before Loading Data

Finish all asynchronous requests before loading data?

For users seeking answer to question in title then use of dispatch_group and GCD outlined here: i.e embedding one group inside the notification method of another dispatch_group is valid. Another way to go at a higher level would be NSOperations and dependencies which would also give further control such as canceling operations.

Outline:

func doStuffonObjectsProcessAndComplete(arrayOfObjectsToProcess: Array) -> Void){

    let firstGroup = dispatch_group_create()

    for object in arrayOfObjectsToProcess {

        dispatch_group_enter(firstGroup)

        doStuffToObject(object, completion:{ (success) in
            if(success){
                // doing stuff success
            }
            else {
                // doing stuff fail
            }
            // regardless, we leave the group letting GCD know we finished this bit of work
            dispatch_group_leave(firstGroup)
        })
    }

    // called once all code blocks entered into group have left
    dispatch_group_notify(firstGroup, dispatch_get_main_queue()) {

        let processGroup = dispatch_group_create()

        for object in arrayOfObjectsToProcess {

            dispatch_group_enter(processGroup)

            processObject(object, completion:{ (success) in
                if(success){
                    // processing stuff success
                }
                else {
                    // processing stuff fail
                }
                // regardless, we leave the group letting GCD know we finished this bit of work
                dispatch_group_leave(processGroup)
            })
        }

        dispatch_group_notify(processGroup, dispatch_get_main_queue()) {
            print("All Done and Processed, so load data now")
        }
    }
}

The remainder of this answer is specific to this codebase.

There seem to be a few problems here:
The getAttendees function takes an event child and returns an objectID and Name which are both Strings? Shouldn't this method return an array of attendees? If not, then what is the objectID that is returned?

Once an array of attendees is returned, then you can process them in a group to get the pictures.

The getAttendeesPictures eventually returns UIImages from Facebook. It's probably best to cache these out to the disk and pass path ref - keeping all these fetched images around is bad for memory, and depending on size and number, may quickly lead to problems.

Some examples:

func getAttendees(child: String, completion: (result: Bool, attendees: Array?) -> Void){

    let newArrayOfAttendees = []()

    // Get event attendees of particular event

    // process attendees and package into an Array (or Dictionary)

    // completion
    completion(true, attendees: newArrayOfAttendees)
}

func getAttendeesPictures(attendees: Array, completion: (result: Bool, attendees: Array)-> Void){

    println("Attendees Count: \(attendees.count)")

    let picturesGroup = dispatch_group_create()

    for attendee in attendees{

       // for each attendee enter group
       dispatch_group_enter(picturesGroup)

       let key = attendee.objectID

       let url = NSURL(string: "https://graph.facebook.com/\(key)/picture?type=large")

        let urlRequest = NSURLRequest(URL: url!)

        //Asynchronous request to display image
        NSURLConnection.sendAsynchronousRequest(urlRequest, queue: NSOperationQueue.mainQueue()) { (response:NSURLResponse!, data:NSData!, error:NSError!) -> Void in
            if error != nil{
                println("Error: \(error)")
            }

            // Display the image
            let image = UIImage(data: data)
            if(image != nil){
               attendee.image = image
            }

            dispatch_group_leave(picturesGroup)
        }
    }

    dispatch_group_notify(picturesGroup, dispatch_get_main_queue()) {
         completion(true, attendees: attendees)
    }
}

func setupEvents(completion: (result: Bool, Event: Event) -> Void){

    // get event info and then for each event...

    getAttendees(child:snapshot.key, completion: { (result, attendeesReturned) in
        if result {
            self.getAttendeesPictures(attendees: attendeesReturned,         completion: { (result, attendees) in

              // do something with completed array and attendees


            }
        }
        else {

        }
    })

}

The above code is just an outline, but hopefully points you in the right direction.

Wait for multiple http requests to finish before running a function in angular

First off the async/await keywords are for use with promises, not Observables. So you can convert an observable to a promise using toPromise() or you can use rxjs ForkJoin. You also need to change the methods to return either an observable or return a promise, you can't do much with Subscription which is what subscribe returns when you call it and that is what you are passing now from getRiskTable.

One other issue is that you are not taking advantage of the type system, do not use any everywhere because these errors could have been caught at transpile time and now you have to guess why it's failing at run time which is more difficult.

import { forkJoin } from 'rxjs';
import { shareReplay } from 'rxjs/operators';

getRiskTable() : Observable {
  const tableObservable = this.riskService.getRiskTable().pipe(shareReplay());
  tableObservable.subscribe((res) => this.riskTable = res, (err) => {this.error = err;});
  return tableObservable;
}

getAllData() {
  let riskTable = this.getRiskTable();
  let risks = this.getAllRisks(); // should also return some observable

  forkJoin(riskTable, risks).subscribe(_ => {
    // all observables have been completed
  });
}

Note that I also added shareReplay which ensures that multiple subscriptions to the same observable do not result in multiple calls to your API end point. One result is shared between multiple subscriptions to the observable.

How to show 'loading' before all async requests done?

There are a number of ways to do this but here is one pattern I tend to use:

const ComponentMakingApiCall = () => {
  const [ isLoading, setIsLoading ] = useState(true)
  useEffect(() => {
    axios.post({
      // whatever request you are making
    }).then(response => {
      setIsLoading(false)
    })
  },[])
  
  if (isLoading){
    return 
Loading...
  } else {
    return (
      

        // render some content
      

    )
  }
}

If you have multiple requests, you could do something like this:

const ComponentMakingApiCall = () => {
  const [ requestALoading, setRequestALoading ] = useState(true)
  const [ requestBLoading, setRequestBLoading ] = useState(true)
  useEffect(() => {
    axios.post({
      // request A
    }).then(response => {
      handleResponse(response)
      setRequestALoading(false)
    })
  },[])
  
  useEffect(() => {
    axios.post({
      // request B
    }).then(response => {
      handleResponse(response)
     setRequestBLoading(false)
    })
    
  },[])

  if ( requestALoading || requestBLoading ) {
    return 
Loading...
  } else {
    return (
      

        // render some content
      

    )
  }
}

How to wait until all async calls are finished

You could switch to promises instead of Observables, thus turning the method into an async one that recurs as long as data has a nextPageToken:

private async loadVideos(
        playListId: string,
        channel: Channel,
        nextPageToken: string,
        stopLoadingOnVideoId: string,
    ) {
        const baseUrl = YoutubeService.VIDEO_URL_SNIPPET_BY_ID + playListId;
        const response = await this.httpService
            .get(nextPageToken ? url + '&pageToken=' + nextPageToken : url).toPromise();
        const { data } = response;
        for (const item of data.items) {
            if (stopLoadingOnVideoId && item.snippet.resourceId.videoId === stopLoadingOnVideoId) {
                continue;
            }
            const partialVideo = await this.prepareVideoEntity(item.snippet, channel);
            const video = await this.videoService.create(partialVideo)
            this.videoIdMap[video.youtubeId] = video.id;
        }
        if (data.nextPageToken) {
            await this.loadVideos(
                playListId,
                channel,
                data.nextPageToken,
                stopLoadingOnVideoId,
            );
        }
    }

In your caller you can then simply await loadVideos(...):

private async initVideoIdMap(...) {
  await this.loadVideos(...);
  // this.videoIdMap should be correctly populated at this point
}

How to show a loading indicator until all async http calls are completed - Angular

You could use a resolver. A resolver ensures data is loaded before the component loads.

Alternatively, if you don't want to use *ngIf you could just use [ngClass]="{hideMe: allAsyncRequestsComplete}" to style the bit you don't want to show until loading is complete. CSS might be:

.hideMe {
  visibility: hidden;
}

And set allAsyncRequestsComplete to true when loading is done.

Wait for multiple async calls to finish before React render

Using Promise.all:

componentDidMount() {
  const fetchData1 = ExternalComponent.fetchData()
  const fetchData2 = AnotherExternalComponent.fetchData()

  Promise.all([ fetchData1, fetchData2 ]).then((responses) => {
      this.setState({
          loading: false,
          data: responses[0]
      });
  });
}

How to wait for an API request to finish before storing data from function callback?

First you need to call completion when all of your data is loaded. In your case you call completion(tracksArray) before any of the getSavedTracks return.

For this part I suggest you to recursively accumulate tracks by going through all pages. There are multiple better tools to do so but I will give a raw example of it:

class TracksModel {
    
    static func fetchAllPages(completion: @escaping ((_ tracks: [Track]?) -> Void)) {
        var offset: Int = 0
        let limit: Int = 50
        var allTracks: [Track] = []
        
        func appendPage() {
            fetchSavedMusicPage(offset: offset, limit: limit) { tracks in
                guard let tracks = tracks else {
                    completion(allTracks) // Most likely an error should be handled here
                    return
                }
                if tracks.count < limit {
                    // This was the last page because we got less than limit (50) tracks
                    completion(allTracks+tracks)
                } else {
                    // Expecting another page to be loaded
                    offset += limit // Next page
                    allTracks += tracks
                    appendPage() // Recursively call for next page
                }
            }
        }

        appendPage() // Load first page
        
    }
    
    private static func fetchSavedMusicPage(offset: Int, limit: Int, completion: @escaping ((_ tracks: [Track]?) -> Void)) {
        APICaller.shared.getUsersSavedTracks(limit: limit, offset: offset) { result in
            switch result {
            case .success(let model):
                completion(model)
            case .failure(let error):
                print(error)
                completion(nil) // Error also needs to call a completion
            }
        }
    }
    
}

I hope comments will clear some things out. But the point being is that I nested an appendPage function which is called recursively until server stops sending data. In the end either an error occurs or the last page returns fewer tracks than provided limit.
Naturally it would be nicer to also forward an error but I did not include it for simplicity.

In any case you can now anywhere TracksModel.fetchAllPages { } and receive all tracks.

When you load and show your data (createSpinnerView) you also need to wait for data to be received before continuing. For instance:

func createSpinnerView() {

    let loadViewController = LoadViewController.instantiateFromAppStoryboard(appStoryboard: .OrganizeScreen)
    add(asChildViewController: loadViewController)

    TracksModel.fetchAllPages { tracks in
        DispatchQueue.main.async {
            self.tracksArray = tracks
            self.remove(asChildViewController: loadViewController)
            self.navigateToFilterScreen(tracksArray: tracks)
        }
    }
    
}

A few components may have been removed but I hope you see the point. The method should be called on main thread already. But you are unsure what thread the API call returned on. So you need to use DispatchQueue.main.async within the completion closure, not outside of it. And also call to navigate within this closure because this is when things are actually complete.

Adding situation for fixed number of requests

For fixed number of requests you can do all your requests in parallel. You already did that in your code.
The biggest problem is that you can not guarantee that responses will come back in same order than your requests started. For instance if you perform two request A and B it can easily happen due to networking or any other reason that B will return before A. So you need to be a bit more sneaky. Look at the following code:

private func loadPage(pageIndex: Int, perPage: Int, completion: @escaping ((_ items: [Any]?, _ error: Error?) -> Void)) {
    // TODO: logic here to return a page from server
    completion(nil, nil)
}

func load(maximumNumberOfItems: Int, perPage: Int, completion: @escaping ((_ items: [Any], _ error: Error?) -> Void)) {
    let pageStartIndicesToRetrieve: [Int] = {
        var startIndex = 0
        var toReturn: [Int] = []
        while startIndex < maximumNumberOfItems {
            toReturn.append(startIndex)
            startIndex += perPage
        }
        return toReturn
    }()
    
    guard pageStartIndicesToRetrieve.isEmpty == false else {
        // This happens if maximumNumberOfItems == 0
        completion([], nil)
        return
    }
    
    enum Response {
        case success(items: [Any])
        case failure(error: Error)
    }
    
    // Doing requests in parallel
    // Note that responses may return in any order time-wise (we can not say that first page will come first, maybe the order will be [2, 1, 5, 3...])
    
    var responses: [Response?] = .init(repeating: nil, count: pageStartIndicesToRetrieve.count) { // Start with all nil
        didSet {
            // Yes, Swift can do this :D How amazing!
            guard responses.contains(where: { $0 == nil }) == false else {
                // Still waiting for others to complete
                return
            }
            
            let aggregatedResponse: (items: [Any], errors: [Error]) = responses.reduce((items: [], errors: [])) { partialResult, response in
                switch response {
                case .failure(let error): return (partialResult.items, partialResult.errors + [error])
                case .success(let items): return (partialResult.items + [items], partialResult.errors)
                case .none: return (partialResult.items, partialResult.errors)
                }
            }
            
            let error: Error? = {
                let errors = aggregatedResponse.errors
                if errors.isEmpty {
                    return nil // No error
                } else {
                    // There was an error.
                    return NSError(domain: "Something more meaningful", code: 500, userInfo: ["all_errors": errors]) // Or whatever you wish. Perhaps just "errors.first!"
                }
            }()
            
            completion(aggregatedResponse.items, error)
        }
    }
    
    pageStartIndicesToRetrieve.enumerated().forEach { requestIndex, startIndex in
        loadPage(pageIndex: requestIndex, perPage: perPage) { items, error in
            responses[requestIndex] = {
                if let error = error {
                    return .failure(error: error)
                } else {
                    return .success(items: items ?? [])
                }
            }()
        }
    }
    
}

The first method is not interesting. It just loads a single page. The second method now collects all the data.

First thing that happens is we calculate all possible requests. We need a start index and per-page. So the pageStartIndicesToRetrieve for case of 145 items using 50 per page will return [0, 50, 100]. (I later found out we only need count 3 in this case but that depends on the API, so let's stick with it). We expect 3 requests starting with item indices [0, 50, 100].

Next we create placeholders for our responses using

var responses: [Response?] = .init(repeating: nil, count: pageStartIndicesToRetrieve.count)

for our example of 145 items and using 50 per page this means it creates an array as [nil, nil, nil]. And when all of the values in this array turn to not-nil then all requests have returned and we can process all of the data. This is done by overriding the setter didSet for a local variable. I hope the content of it speaks for itself.

Now all that is left is to execute all requests at once and fill the array. Everything else should just resolve by itself.

The code is not the easiest and again; there are tools that can make things much easier. But for academical purposes I hope this approach explains what needs to be done to accomplish your task correctly.

JavaScript - Run callback after all fetch requests to finish in multiple for loops - errored or not

Push all of your promises into an array, and then use Promise.allSettled(promises).then((results) => {})

Documentation: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled

Example:

const promises = images.map(async (image) => {
    // do some async work
    return fetch(whatever); // don't .catch this

})

Promise.allSettled(promises).then((results) => {
  // results is a result of errors or successes
})

Wait until swift for loop with asynchronous network requests finishes executing

You can use dispatch groups to fire an asynchronous callback when all your requests finish.

Here's an example using dispatch groups to execute a callback asynchronously when multiple networking requests have all finished.

override func viewDidLoad() {
    super.viewDidLoad()

    let myGroup = DispatchGroup()

    for i in 0 ..< 5 {
        myGroup.enter()

        Alamofire.request("https://httpbin.org/get", parameters: ["foo": "bar"]).responseJSON { response in
            print("Finished request \(i)")
            myGroup.leave()
        }
    }

    myGroup.notify(queue: .main) {
        print("Finished all requests.")
    }
}

Output

Finished request 1
Finished request 0
Finished request 2
Finished request 3
Finished request 4
Finished all requests.

---

Related Topics