Upload chunked file using XHR and web workers
I resolved reading file from php://input with this code:
$putdata = fopen("php://input", "r");
$fp = fopen($target, "w");
while ($data = fread($putdata, 16384)) {
fwrite($fp, $data);
}
fclose($fp);
fclose($putdata);
In this way, I don't need to write the HTTP headers of the multipart/form-data FormData in a webworker - in some browsers - is this wrong?
Chrome supports FormData in Web Workers since version 36.0.1935.0 (crbug.com/360546).
It exists because the latest specification of FormData
requires it to be exposed to Worker contexts. Firefox has not implemented this yet, but it is on their radar (bugzil.la/739173).
I think that you're misreading my answer that you've linked. new FormData(<HTMLFormElement>);
is not supported in the sense that the constructor that takes a <form>
and initializes its fields based on the form elements is not supported, because <form>
elements can obviously not be created in a Web worker. But you can create an empty FormData
object and use it as you wish (if the browser implements the latest version of the specification).
If you want to use the FormData
API in all current browsers, then you have to load my polyfill that you referenced in your question. This polyfill returns early if it detects that the FormData
API is already defined, so it won't cause issues in browsers that already support FormData
. Note that this polyfill is inefficient compared to a native FormData
API, because the polyfill has to create the full string upfront (in memory), whereas a native implementation can just hold light data structures for a file, and stream the file from the disk when the File/Blob is uploaded.
For small pieces of data, this is a non-issue, but if you plan on uploading huge files, then you'd better pass the data to the main thread using postMessage
(with transferables if you use typed arrays) and construct the XMLHttpRequest object over there, and send it. Web Workers are mainly useful for offloading CPU-heavy tasks, but XMLHttpRequest is mainly network (which happens on a separate IO thread, at least in Chrome), so there is no benefit of usign Web Workers over the main thread in this regard.
Handling File Uploads When Offline With Service Worker
One way to handle file uploads/deletes and almost everything, is by keeping track of all the changes made during the offline requests. We can create a sync
object with two arrays inside, one for pending files that will need to be uploaded and one for deleted files that will need to be deleted when we'll get back online.
tl;dr
Key phases
Service Worker Installation
Along with static data, we make sure to fetch dynamic data as the main listing of our uploaded files (in the example case
/uploads
GET
returns JSON data with the files).
Service Worker Fetch
Handling the service worker
fetch
event, if the fetch fails, then we have to handle the requests for the files listing, the requests that upload a file to the server and the request that deletes a file from the server. If we don't have any of these requests, then we return a match from the default cache.- Listing
GET
We get the cached object of the listing (in our case/uploads
) and thesync
object. Weconcat
the default listing files with thepending
files and we remove thedeleted
files and we return new response object with a JSON result as the server would have returned it. - Uloading
PUT
We get the cached listing files and thesync
pending
files from the cache. If the file isn't present, then we create a new cache entry for that file and we use the mime type and theblob
from the request to create a newResponse
object that it will be saved to the default cache. - Deleting
DELETE
We check in the cached uploads and if the file is present we delete the entry from both the listing array and the cached file. If the file is pending we just delete the entry from thepending
array, else if it's not already in thedeleted
array, then we add it. We update listing, files and sync object cache at the end.
- Listing
Syncing
When the
online
event gets triggered, we try to synchronize with the server. We read thesync
cache.- If there are pending files, then we get each file
Response
object from cache and we send aPUT
fetch
request back to the server. - If there are deleted files, then we send a
DELETE
fetch
request for each file to the server. - Finally, we reset the
sync
cache object.
- If there are pending files, then we get each file
Code implementation
(Please read the inline comments)
Service Worker Install
const cacheName = 'pwasndbx';
const syncCacheName = 'pwasndbx-sync';
const pendingName = '__pending';
const syncName = '__sync';
const filesToCache = [
'/',
'/uploads',
'/styles.css',
'/main.js',
'/utils.js',
'/favicon.ico',
'/manifest.json',
];
/* Start the service worker and cache all of the app's content */
self.addEventListener('install', function(e) {
console.log('SW:install');
e.waitUntil(Promise.all([
caches.open(cacheName).then(async function(cache) {
let cacheAdds = [];
try {
// Get all the files from the uploads listing
const res = await fetch('/uploads');
const { data = [] } = await res.json();
const files = data.map(f => `/uploads/${f}`);
// Cache all uploads files urls
cacheAdds.push(cache.addAll(files));
} catch(err) {
console.warn('PWA:install:fetch(uploads):err', err);
}
// Also add our static files to the cache
cacheAdds.push(cache.addAll(filesToCache));
return Promise.all(cacheAdds);
}),
// Create the sync cache object
caches.open(syncCacheName).then(cache => cache.put(syncName, jsonResponse({
pending: [], // For storing the penging files that later will be synced
deleted: [] // For storing the files that later will be deleted on sync
}))),
])
);
});
Service Worker Fetch
self.addEventListener('fetch', function(event) {
// Clone request so we can consume data later
const request = event.request.clone();
const { method, url, headers } = event.request;
event.respondWith(
fetch(event.request).catch(async function(err) {
const { headers, method, url } = event.request;
// A custom header that we set to indicate the requests come from our syncing method
// so we won't try to fetch anything from cache, we need syncing to be done on the server
const xSyncing = headers.get('X-Syncing');
if(xSyncing && xSyncing.length) {
return caches.match(event.request);
}
switch(method) {
case 'GET':
// Handle listing data for /uploads and return JSON response
break;
case 'PUT':
// Handle upload to cache and return success response
break;
case 'DELETE':
// Handle delete from cache and return success response
break;
}
// If we meet no specific criteria, then lookup to the cache
return caches.match(event.request);
})
);
});
function jsonResponse(data, status = 200) {
return new Response(data && JSON.stringify(data), {
status,
headers: {'Content-Type': 'application/json'}
});
}
Service Worker Fetch Listing GET
if(url.match(/\/uploads\/?$/)) { // Failed to get the uploads listing
// Get the uploads data from cache
const uploadsRes = await caches.match(event.request);
let { data: files = [] } = await uploadsRes.json();
// Get the sync data from cache
const syncRes = await caches.match(new Request(syncName), { cacheName: syncCacheName });
const sync = await syncRes.json();
// Return the files from uploads + pending files from sync - deleted files from sync
const data = files.concat(sync.pending).filter(f => sync.deleted.indexOf(f) < 0);
// Return a JSON response with the updated data
return jsonResponse({
success: true,
data
});
}
Service Worker Fetch Uloading PUT
// Get our custom headers
const filename = headers.get('X-Filename');
const mimetype = headers.get('X-Mimetype');
if(filename && mimetype) {
// Get the uploads data from cache
const uploadsRes = await caches.match('/uploads', { cacheName });
let { data: files = [] } = await uploadsRes.json();
// Get the sync data from cache
const syncRes = await caches.match(new Request(syncName), { cacheName: syncCacheName });
const sync = await syncRes.json();
// If the file exists in the uploads or in the pendings, then return a 409 Conflict response
if(files.indexOf(filename) >= 0 || sync.pending.indexOf(filename) >= 0) {
return jsonResponse({ success: false }, 409);
}
caches.open(cacheName).then(async (cache) => {
// Write the file to the cache using the response we cloned at the beggining
const data = await request.blob();
cache.put(`/uploads/${filename}`, new Response(data, {
headers: { 'Content-Type': mimetype }
}));
// Write the updated files data to the uploads cache
cache.put('/uploads', jsonResponse({ success: true, data: files }));
});
// Add the file to the sync pending data and update the sync cache object
sync.pending.push(filename);
caches.open(syncCacheName).then(cache => cache.put(new Request(syncName), jsonResponse(sync)));
// Return a success response with fromSw set to tru so we know this response came from service worker
return jsonResponse({ success: true, fromSw: true });
}
Service Worker Fetch Deleting DELETE
// Get our custom headers
const filename = headers.get('X-Filename');
if(filename) {
// Get the uploads data from cache
const uploadsRes = await caches.match('/uploads', { cacheName });
let { data: files = [] } = await uploadsRes.json();
// Get the sync data from cache
const syncRes = await caches.match(new Request(syncName), { cacheName: syncCacheName });
const sync = await syncRes.json();
// Check if the file is already pending or deleted
const pendingIndex = sync.pending.indexOf(filename);
const uploadsIndex = files.indexOf(filename);
if(pendingIndex >= 0) {
// If it's pending, then remove it from pending sync data
sync.pending.splice(pendingIndex, 1);
} else if(sync.deleted.indexOf(filename) < 0) {
// If it's not in pending and not already in sync for deleting,
// then add it for delete when we'll sync with the server
sync.deleted.push(filename);
}
// Update the sync cache
caches.open(syncCacheName).then(cache => cache.put(new Request(syncName), jsonResponse(sync)));
// If the file is in the uplods data
if(uploadsIndex >= 0) {
// Updates the uploads data
files.splice(uploadsIndex, 1);
caches.open(cacheName).then(async (cache) => {
// Remove the file from the cache
cache.delete(`/uploads/${filename}`);
// Update the uploads data cache
cache.put('/uploads', jsonResponse({ success: true, data: files }));
});
}
// Return a JSON success response
return jsonResponse({ success: true });
}
Synching
// Get the sync data from cache
const syncRes = await caches.match(new Request(syncName), { cacheName: syncCacheName });
const sync = await syncRes.json();
// If the are pending files send them to the server
if(sync.pending && sync.pending.length) {
sync.pending.forEach(async (file) => {
const url = `/uploads/${file}`;
const fileRes = await caches.match(url);
const data = await fileRes.blob();
fetch(url, {
method: 'PUT',
headers: {
'X-Filename': file,
'X-Syncing': 'syncing' // Tell SW fetch that we are synching so to ignore this fetch
},
body: data
}).catch(err => console.log('sync:pending:PUT:err', file, err));
});
}
// If the are deleted files send delete request to the server
if(sync.deleted && sync.deleted.length) {
sync.deleted.forEach(async (file) => {
const url = `/uploads/${file}`;
fetch(url, {
method: 'DELETE',
headers: {
'X-Filename': file,
'X-Syncing': 'syncing' // Tell SW fetch that we are synching so to ignore this fetch
}
}).catch(err => console.log('sync:deleted:DELETE:err', file, err));
});
}
// Update and reset the sync cache object
caches.open(syncCacheName).then(cache => cache.put(syncName, jsonResponse({
pending: [],
deleted: []
})));
Example PWA
I have created a PWA example that implements all these, which you can find and test here. I have tested it using Chrome and Firefox and using Firefox Android on a mobile device.
You can find the full source code of the application (including an express
server) in this Github repository: https://github.com/clytras/pwa-sandbox.
jQuery Ajax File Upload
File upload is not possible through AJAX.
You can upload file, without refreshing page by using IFrame
.
You can check further details here.
UPDATE
With XHR2, File upload through AJAX is supported. E.g. through FormData
object, but unfortunately it is not supported by all/old browsers.
FormData
support starts from following desktop browsers versions.
- IE 10+
- Firefox 4.0+
- Chrome 7+
- Safari 5+
- Opera 12+
Related Topics
How to Get an Array of Data from $_Post
How Does PHP Max_Execution_Time Work
How to Avoid Request Entity Too Large 413 Error
PHP - Get Array Value with a Numeric Index
How to Keep a PHP Session Active Even If the Browser Is Closed
Selenium2 Firefox: Use the Default Profile
Zend Framework - Multiplate Navigation Blocks
Possible to View PHP Code of a Website
Rounding Up to the Second Decimal Place
Zend Framework 2 Routing Subdomains to Module
PHP Considers Null Is Equal to Zero
Combining And/Or Eloquent Query in Laravel
Str_Replace() for Multiple Value Replacement
PHP Readfile() Causing Corrupt File Downloads
Check If a String Is All Caps in PHP