Urlresponse Is Not Retrieved After Storing in Cache Using Storecachedresponse

URLresponse is not retrieved after storing in cache using storeCachedResponse

Welcome to the wonderful world of asynchronous caches. NSURLCache is highly asynchronous. Just because you've shoved data into it doesn't mean it is available for retrieval. You have to let the main run loop return before it will be available, and possibly even wait a little while. The failure to return a response immediately after storing it is not at all unusual. Try dispatching it after five seconds or so.

Second, your cache might be a bit on the small size for storing multi-megabyte images. Try bumping that up and see if it helps.

Finally, what do you mean when you say that you "turn off your Internet?" You say that you're getting a timeout. Normally, if you put the device into Airplane mode with all connectivity disabled, it should not sit there for any significant amount of time before failing with an error indicating no connectivity). If that isn't happening, something strange is happening, almost as if waitsForConnectivity is getting set on the session or something. (You aren't making the network requests in the background, are you? If so, try explicitly setting waitsForConnectivity to NO so that they won't wait for a connection to be available.)

Also, for this usage, you may have to strip out the Vary: Accept-Encoding header or provide a consistent user agent string. That header causes the cache to basically be per-browser. This may cause the cache to behave in unexpected ways, and is probably the cause of the weirdness you're seeing.

Note that stripping out the Vary header is a bit of a hack, and probably isn't the most correct way to fix the issue; ideally, you should tweak whatever outgoing header fields you have to tweak so that it works even with that header present. But you'd have to research it and figure out exactly what fields are needed, because I have no idea. :-)

URLCache (CS193P Assignment 6)

You can use URLCache by making a request for the image data with URLSession then using the data and response available in its completion handler, for example:

import UIKit

class GalleryCollectionViewController: UICollectionViewController, UICollectionViewDragDelegate, UICollectionViewDropDelegate, UICollectionViewDelegateFlowLayout {

 // MARK: - Model

 var gallery: Gallery?

 // MARK: - Properties

 private var cache = URLCache.shared
 private var session = URLSession(configuration: .default)

 override func viewDidLoad() {
    super.viewDidLoad()
    cache = URLCache(memoryCapacity: 100, diskCapacity: 100, diskPath: nil) // replace capacities with your own values
 }

 override func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {
    let cell = collectionView.dequeueReusableCell(withReuseIdentifier: "GalleryCell", for: indexPath)
    if let galleryCell = cell as? GalleryCollectionViewCell {
        galleryCell.image = nil
        galleryCell.imageView.isHidden = true
        if let url = gallery?.images[indexPath.item].url {
            let request = URLRequest(url: url.imageURL) // imageURL from Utilities.swift of Stanford iOS course
            if let cachedResponse = cache.cachedResponse(for: request), let image = UIImage(data: cachedResponse.data) {
                galleryCell.image = image
                galleryCell.imageView.isHidden = false
            } else {
                DispatchQueue.global(qos: .userInitiated).async { [weak self, weak galleryCell] in
                    let task = self?.session.dataTask(with: request) { (urlData, urlResponse, urlError) in
                        DispatchQueue.main.async {
                            if urlError != nil { print("Data request failed with error \(urlError!)") }
                            if let data = urlData, let image = UIImage(data: data) {
                                if let response = urlResponse {
                                    self?.cache.storeCachedResponse(CachedURLResponse(response: response, data: data), for: request)
                                }
                                galleryCell?.image = image
                            } else {
                                galleryCell?.image = UIImage(named: "placeholder")
                            }
                            galleryCell?.imageView.isHidden = false
                        }
                    }
                    task?.resume()
                }
            }
        }
    }
    return cell
 }
}

NSURLCache does not clear stored responses in iOS8

NSURLCache is broken on iOS 8.0.x - it never purges the cache at all, so it grows without limit. See http://blog.airsource.co.uk/2014/10/11/nsurlcache-ios8-broken/ for a detailed investigation. Cache purging is fixed in the 8.1 betas - but removeCachedResponseForRequest: is not.

removeCachedResponsesSinceDate: does appear to work on iOS 8.0 - an API that was added for 8.0, but hasn't made it to the docs yet (it is in the API diffs). I am unclear what use it is to anyone - surely what you normally want to do is remove cached responses before a particular date.

removeAllCachedResponses works as well - but that's a real sledgehammer solution.

cachedResponseForRequest method not being accessed

connection:willCacheResponse: is only called in cases when the response will be cached. POST requests are not cacheable in most cases. (More details: Is it possible to cache POST methods in HTTP?)

You should probably look at something like MKNetworkKit which handles a lot of this kind of caching, particularly for REST protocols.

You can also look at Drop-in offline caching for UIWebView. You'd have to modify it significantly, but NSURLProtocol can be used to solve this kind of problem. AFCache is currently working to integrate this approach, and is another toolkit to consider. (Read through the comments in the blog post for more background on the issues.)

iOS URLCache caching when it shouldn't (IMHO)

I've always assumed that without cache headers a response won't be cached :|

That's not how it works, it WILL cache responses even without Cache-Control or Expires headers, if all other criteria met. However, it will use heuristics for determining cached response freshness, as exact expiration time was not provided in response headers. NSURLCache is implemented according to section 13 of RFC 2616 and this behavior is stated there.

For more information you can check the following articles:

A Primer In Http–caching And Its–native Support–by–ios
Image Caching

How to cache using NSURLSession and NSURLCache. Not working

Note that the following SO post helped me solve my problem: Is NSURLCache persistent across launches?

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    // Set app-wide shared cache (first number is megabyte value)
    NSUInteger cacheSizeMemory = 500*1024*1024; // 500 MB
    NSUInteger cacheSizeDisk = 500*1024*1024; // 500 MB
    NSURLCache *sharedCache = [[NSURLCache alloc] initWithMemoryCapacity:cacheSizeMemory diskCapacity:cacheSizeDisk diskPath:@"nsurlcache"];
    [NSURLCache setSharedURLCache:sharedCache];
    sleep(1); // Critically important line, sadly, but it's worth it!
}

In addition to the sleep(1) line, also note the size of my cache; 500MB.

According to docs you need a cache size that is way bigger than what you're trying to cache.

The response size is small enough to reasonably fit within the cache.
(For example, if you provide a disk cache, the response must be no
larger than about 5% of the disk cache size.)

So for example if you want to be able to cache a 10MB image, then a cache size of 10MB or even 20MB will not be enough. You need 200MB.
Honey's comment below is evidence that Apple is following this 5% rule. For an 8Mb he had to set his cache size to minimum 154MB.

Does not setting cache-control automatically enable caching even without conditional request?

doesn't the URLSession need to perform a conditional request to make sure its still valid?

The user-agent should be performing a conditional request, because of the

Etag: 00e21950bf432476c91b811bb685b6af

present. My desktop Chrome certainly does performs the conditional request (and gets back 304 Not Modified).

But it's free not to

But a user-agent is perfectly free to decide on it's own. It's perfectly free to look at:

Last-Modified: Fri, 04 Oct 2013 23:30:08 GMT

and decide that there resource is probably good for the next five minutes¹. And if the network connection is down, its perfectly reasonable and correct to display the cached version instead. In fact, your browser would show you web-sites even while your dial-up 0.00336 Mbps dial-up modem was disconnected.

You wouldn't want your browser to show you nothing, when it knows full well it can show you something. It becomes even more useful when we're talking about poor internet connectivity not because of slow dialup and servers that go down, but of mobile computing, and metered data plans.

¹I say 5 minutes, because in the early web, servers did not give cache hints. So browsers cached things without even being asked. And 5 minutes was a good number. And you used Ctrl+F5 (or was it Shift+F5, or was it Shift+Click, or was it Alt+Click) to force the browser to bypass the cache.

When exactly do things get removed from urlcache's memory and disk?

Short answer:

The memory cache is lost when the app terminates (or, likely, under memory pressure, too). The disk cache persists across multiple invocations of the app, though can be removed if the device runs out of persistent storage and the OS reclaims space from caches and temp folders.

Long answer:

You ask:

Is it correct to say, every time you force quit or your app crashes due to a memory warning/issue your cache will get flushed, though it leaves your disk intact?

Largely. It may be more precise to say simply that all memory related to your app is discarded when the app terminates and that only those items saved to persistent storage can be retrieved when the app restarts.

Any other place where cache can get flushed out of memory?

You lose everything in the memory cache when the app terminates. There are obviously a few other situations in which case items can be removed:

If you manually remove responses from the URLCache.
Older, individual items are removed from the cache as you approach the max capacity of the cache and you try to add new items, forcing older items out.
Network responses often include a cache policy (indicating how long the response can be safely cached), so it's likely that they're removed at that point.
The memory cache may reasonably be purged upon memory pressure (e.g. .UIApplicationDidReceiveMemoryWarning).

When can things stored in disk get removed?

The logic is largely the same as prior point, except that (a) it can survive across invocations of the app); and (b) it's not flushed upon memory pressure, though it can be removed when the device runs low on persistent storage.

URLCache (iOS). storeCachedResponse works asynchronously. How to catch the completion?

Actually I couldn't understand why you calling cached data immediately after caching!? In my opinion you should call cached data before requesting url with session if data is exist return cached data else request from the scratch.

For example :

private let allowedDiskSize = 100 * 1024 * 1024
private lazy var cache: URLCache = {
    return URLCache(memoryCapacity: 0, diskCapacity: allowedDiskSize, diskPath: "gifCache")
}()

typealias DownloadCompletionHandler = (Result<Data,Error>) -> ()

private func createAndRetrieveURLSession() -> URLSession {
    let sessionConfiguration = URLSessionConfiguration.default
    sessionConfiguration.requestCachePolicy = .returnCacheDataElseLoad
    sessionConfiguration.urlCache = cache
    return URLSession(configuration: sessionConfiguration)
}

private func downloadContent(fromUrlString: String, completionHandler: @escaping DownloadCompletionHandler) {

    guard let downloadUrl = URL(string: fromUrlString) else { return }
    let urlRequest = URLRequest(url: downloadUrl)
    // First try to fetching cached data if exist
    if let cachedData = self.cache.cachedResponse(for: urlRequest) {
        print("Cached data in bytes:", cachedData.data)
        completionHandler(.success(cachedData.data))

    } else {
        // No cached data, download content than cache the data
        createAndRetrieveURLSession().dataTask(with: urlRequest) { (data, response, error) in

            if let error = error {
                completionHandler(.failure(error))
            } else {

                let cachedData = CachedURLResponse(response: response!, data: data!)
                self.cache.storeCachedResponse(cachedData, for: urlRequest)

                completionHandler(.success(data!))
            }
        }.resume()
    }
}

And usage:

self.downloadContent(fromUrlString: ANY_URL, completionHandler: { (result) in

            switch result {
            case .success(let yourData):
                // handle data

            case .failure(let error):
                debugPrint(error.localizedDescription)
            }
 })

First time it will fetch data from the web and in second request it will return cached data immediately.

Urlresponse Is Not Retrieved After Storing in Cache Using Storecachedresponse