* Changes in `FileUload` to use `chunk upload` in some cases

* `chunk-enabled` prop added to enable chunk upload
  * `chunk` prop added to modify chunk upload parameters
* `ChunkUploadHandler` class created to handler chunk upload process
* Example added to the docs
* Chunk documentation added to the docs
master
José Cámara 8 years ago
parent 7d88edc716
commit 872c984eb2

@ -0,0 +1,38 @@
// http://eslint.org/docs/user-guide/configuring
module.exports = {
root: true,
parser: 'babel-eslint',
parserOptions: {
sourceType: 'module'
},
env: {
browser: true,
jest: true
},
// https://github.com/standard/standard/blob/master/docs/RULES-en.md
extends: "standard",
// required to lint *.vue files
plugins: [
'html'
],
// add your custom rules here
'rules': {
// allow paren-less arrow functions
'arrow-parens': 0,
// allow async-await
'generator-star-spacing': 0,
// allow debugger during development
'no-debugger': process.env.NODE_ENV === 'production' ? 2 : 0,
'no-useless-escape': 0,
'comma-dangle': 0,
'space-before-function-paren': 0,
'no-multiple-empty-lines': 0,
'no-multi-spaces': 0,
'padded-blocks': 0,
'prefer-promise-reject-errors': 0,
'operator-linebreak': 0
}
}

@ -15,6 +15,7 @@
- [x] `PUT` method - [x] `PUT` method
- [x] Customize the filter - [x] Customize the filter
- [x] thumbnails - [x] thumbnails
- [x] Chunk upload

@ -131,7 +131,81 @@ new Vue({
</html> </html>
``` ```
### Chunk Upload
This package allows chunk uploads, which means you can upload a file in different parts.
This process is divided in three phases: <strong>start</strong>, <strong>upload</strong>,<strong>finish</strong></p>
#### start
This is the first phase of the process. We'll tell the backend that we are going to upload a file, with certain `size` and `mime_type`.
Use the option `startBody` to add more parameters to the body of this request.
The backend will provide a `session_id` (to identify the upload) and a `end_offset` which is the size of every chunk
#### upload
In this phase we'll upload every chunk until all of them are uploaded. This step allows some failures in the backend, and will retry up to `maxRetries` times.
We'll send the `session_id`, `start_chunk` and `chunk` (the sliced blob - part of file we are uploading). We expect the backend to return `status = 'success'`, we'll retry otherwise.
Use the option `uploadBody` to add more parameters to the body of this request.
#### finish
In this phase we tell the backend that there are no more chunks to upload, so it can wrap everything. We send the `session_id` in this phase.
Use the option `finishBody` to add more parameters to the body of this request.
#### Example
```html
<file-upload
ref="upload"
v-model="files"
post-action="/post.method"
put-action="/put.method"
chunk-enabled
:chunk="{
action: '/upload/chunk',
minSize: 1048576,
maxActive: 3,
maxRetries: 5,
// In this case our backend also needs the user id to operate
startBody: {
user_id: user.id
}
}"
@input-file="inputFile"
@input-filter="inputFilter"
>
Upload file
</file-upload>
```
#### Extending the handler
We are using the class `src/chunk/ChunkUploadHandler` class to implement this protocol. You can extend this class (or even create a different one from scratch) to implement your own way to communicat with the backend.
This class must implement a method called `upload` which **must** return a promise. This promise will be used by the `FileUpload` component to determinate whether the file was uploaded or failed.
Use the `handler` parameter to use a different Handler
```html
:chunk="{
action: '/upload/chunk',
minSize: 1048576,
maxActive: 3,
maxRetries: 5,
handler: MyHandlerClass
}
```
### SSR (Server isomorphism) ### SSR (Server isomorphism)
@ -213,9 +287,9 @@ const nodeExternals = require('webpack-node-externals');
} }
``` ```
* [https://github.com/liady/webpack-node-externals](https://github.com/liady/webpack-node-externals) * [https://github.com/liady/webpack-node-externals](https://github.com/liady/webpack-node-externals)
* [**`vue-hackernews` demo**](https://github.com/lian-yue/vue-hackernews-2.0/) * [**`vue-hackernews` demo**](https://github.com/lian-yue/vue-hackernews-2.0/)
* [**View changes**](https://github.com/lian-yue/vue-hackernews-2.0/commit/bd6c58a30cc6b8ba6c0148e737b3ce9336b99cf8) * [**View changes**](https://github.com/lian-yue/vue-hackernews-2.0/commit/bd6c58a30cc6b8ba6c0148e737b3ce9336b99cf8)
@ -290,9 +364,9 @@ The `name` attribute of the input tag
* **Browser:** `> IE9` * **Browser:** `> IE9`
* **Details:** * **Details:**
`put-action` is not empty Please give priority to` PUT` request `put-action` is not empty Please give priority to` PUT` request
* **Usage:** * **Usage:**
```html ```html
@ -324,7 +398,7 @@ Attach `header` data
### data ### data
`POST request`: Append request `body` `POST request`: Append request `body`
`PUT request`: Append request `query` `PUT request`: Append request `query`
* **Type:** `Object` * **Type:** `Object`
@ -347,9 +421,9 @@ File List
* **Default:** `[]` * **Default:** `[]`
* **Details:** * **Details:**
View **[`File`](#file)** details View **[`File`](#file)** details
> In order to prevent unpredictable errors, can not directly modify the `files`, please use [`add`](#instance-methods-add), [`update`](#instance-methods-update), [`remove`](#instance-methods-remove) method to modify > In order to prevent unpredictable errors, can not directly modify the `files`, please use [`add`](#instance-methods-add), [`update`](#instance-methods-update), [`remove`](#instance-methods-remove) method to modify
* **Usage:** * **Usage:**
@ -365,7 +439,7 @@ File List
### accept ### accept
The `accept` attribute of the input tag, MIME type The `accept` attribute of the input tag, MIME type
* **Type:** `String` * **Type:** `String`
@ -386,14 +460,14 @@ The `accept` attribute of the input tag, MIME type
### multiple ### multiple
The `multiple` attribute of the input tag The `multiple` attribute of the input tag
Whether to allow multiple files to be selected Whether to allow multiple files to be selected
* **Type:** `Boolean` * **Type:** `Boolean`
* **Default:** `false` * **Default:** `false`
* **Details:** * **Details:**
If it is `false` file inside only one file will be automatically deleted If it is `false` file inside only one file will be automatically deleted
@ -406,8 +480,8 @@ Whether to allow multiple files to be selected
### directory ### directory
The `directory` attribute of the input tag The `directory` attribute of the input tag
Whether it is a upload folder Whether it is a upload folder
* **Type:** `Boolean` * **Type:** `Boolean`
@ -426,7 +500,7 @@ Whether it is a upload folder
### extensions ### extensions
Allow upload file extensions Allow upload file extensions
* **Type:** `Array | String | RegExp` * **Type:** `Array | String | RegExp`
@ -498,7 +572,7 @@ List the maximum number of files
### thread ### thread
Also upload the number of files at the same time (number of threads) Also upload the number of files at the same time (number of threads)
* **Type:** `Number` * **Type:** `Number`
@ -512,8 +586,42 @@ Also upload the number of files at the same time (number of threads)
``` ```
### chunk-enabled
Whether chunk uploads is enabled or not
* **Type:** `Boolean`
* **Default:** `false`
* **Usage:**
```html
<file-upload :chunk-enabled="true"></file-upload>
<file-upload chunk-enabled></file-upload>
```
### chunk
All the options to handle chunk uploads
* **Type:** `Object`
* **Default:**
```js
{
headers: {
'Content-Type': 'application/json'
},
action: '',
minSize: 1048576,
maxActive: 3,
maxRetries: 5,
// This is the default Handler implemented in this package
// you can use your own handler if your protocol is different
handler: ChunkUploadDefaultHandler
}
```
### drop ### drop
@ -527,7 +635,7 @@ Drag and drop upload
* **Details:** * **Details:**
If set to `true`, read the parent component as a container If set to `true`, read the parent component as a container
* **Usage:** * **Usage:**
```html ```html
@ -615,7 +723,7 @@ Default for `v-model` binding
### @input-filter ### @input-filter
Add, update, remove pre-filter Add, update, remove pre-filter
* **Arguments:** * **Arguments:**
@ -630,14 +738,14 @@ Add, update, remove pre-filter
If the `oldFile` value is `undefined` 'is added If the `oldFile` value is `undefined` 'is added
If `newFile`, `oldFile` is exist, it is updated If `newFile`, `oldFile` is exist, it is updated
> Synchronization modify `newFile` > Synchronization modify `newFile`
> Asynchronous Please use `update`,` add`, `remove`,` clear` method > Asynchronous Please use `update`,` add`, `remove`,` clear` method
> Asynchronous Please set an error first to prevent being uploaded > Asynchronous Please set an error first to prevent being uploaded
> Synchronization can not use `update`,` add`, `remove`,` clear` methods > Synchronization can not use `update`,` add`, `remove`,` clear` methods
> Asynchronous can not modify `newFile` > Asynchronous can not modify `newFile`
* **Usage:** * **Usage:**
```html ```html
<template> <template>
<ul> <ul>
@ -712,8 +820,8 @@ Add, update, remove after
If `newFile`, `oldFile` is exist, it is updated If `newFile`, `oldFile` is exist, it is updated
>You can use `update`,` add`, `remove`,` clear` methods in the event >You can use `update`,` add`, `remove`,` clear` methods in the event
>You can not modify the `newFile` object in the event >You can not modify the `newFile` object in the event
>You can not modify the `oldFile` object in the event >You can not modify the `oldFile` object in the event
* **Usage:** * **Usage:**
@ -955,7 +1063,7 @@ Add the file selected by `<input type = "file">` to the upload list
### addDataTransfer() ### addDataTransfer()
Add files that are dragged or pasted into the upload list Add files that are dragged or pasted into the upload list
* **Arguments:** * **Arguments:**
@ -1092,7 +1200,7 @@ Empty the file list
* **Details:** * **Details:**
If the attribute does not exist, the object will not be processed internally If the attribute does not exist, the object will not be processed internally
If the attribute does not exist, it is not `File` but `Object` If the attribute does not exist, it is not `File` but `Object`
@ -1110,7 +1218,7 @@ File ID
* **Details:** * **Details:**
>`id` can not be repeated >`id` can not be repeated
>Upload can not modify `id` >Upload can not modify `id`
@ -1129,7 +1237,7 @@ File size
### name ### name
Filename Filename
* **Type:** `String` * **Type:** `String`
@ -1139,7 +1247,7 @@ Filename
* **Details:** * **Details:**
Format: `directory/filename.gif` `filename.gif` Format: `directory/filename.gif` `filename.gif`
@ -1174,8 +1282,8 @@ Activation or abort upload
* **Details:** * **Details:**
`true` = Upload `true` = Upload
`false` = Abort `false` = Abort
@ -1194,7 +1302,7 @@ Upload failed error code
* **Details:** * **Details:**
Built-in Built-in
`size`, `extension`, `timeout`, `abort`, `network`, `server`, `denied` `size`, `extension`, `timeout`, `abort`, `network`, `server`, `denied`

@ -25,6 +25,7 @@ export default {
avatar: 'Upload avatar', avatar: 'Upload avatar',
drag: 'Drag and drop', drag: 'Drag and drop',
multiple: 'Multiple instances', multiple: 'Multiple instances',
chunk: 'Chunk upload',
vuex: 'Vuex', vuex: 'Vuex',
} }
} }

@ -11,6 +11,7 @@ import SimpleExampleComponent from './views/examples/Simple'
import AvatarExampleComponent from './views/examples/Avatar' import AvatarExampleComponent from './views/examples/Avatar'
import DragExampleComponent from './views/examples/Drag' import DragExampleComponent from './views/examples/Drag'
import MultipleExampleComponent from './views/examples/Multiple' import MultipleExampleComponent from './views/examples/Multiple'
import ChunkExampleComponent from './views/examples/Chunk'
import VuexExampleComponent from './views/examples/Vuex' import VuexExampleComponent from './views/examples/Vuex'
@ -43,6 +44,10 @@ let examples = [
path: 'multiple', path: 'multiple',
component: MultipleExampleComponent, component: MultipleExampleComponent,
}, },
{
path: 'chunk',
component: ChunkExampleComponent,
},
{ {
path: 'vuex', path: 'vuex',
component: VuexExampleComponent, component: VuexExampleComponent,

@ -19,6 +19,9 @@
<li class="nav-item"> <li class="nav-item">
<router-link active-class="active" class="nav-link" :to="'/examples/multiple' | toLocale">{{$t('example.multiple')}}</router-link> <router-link active-class="active" class="nav-link" :to="'/examples/multiple' | toLocale">{{$t('example.multiple')}}</router-link>
</li> </li>
<li class="nav-item">
<router-link active-class="active" class="nav-link" :to="'/examples/chunk' | toLocale">{{$t('example.chunk')}}</router-link>
</li>
<li class="nav-item"> <li class="nav-item">
<router-link active-class="active" class="nav-link" :to="'/examples/vuex' | toLocale">{{$t('example.vuex')}}</router-link> <router-link active-class="active" class="nav-link" :to="'/examples/vuex' | toLocale">{{$t('example.vuex')}}</router-link>
</li> </li>

@ -0,0 +1,245 @@
<template>
<div class="example-simple">
<h1 id="example-title" class="example-title">Chunk Upload Example</h1>
<p>When using chunk uploads, the file will be uploaded in different parts (or chunks). All the files with a size higher than the set in the input will be uploaded using this method.</p>
<p>You will be able to see the different parts being uploaded individually. Some parts might fail, and the package is prepared to <em>retry</em> up to a certain amount of times.</p>
<p>You can also pause / resume the upload process.</p>
<div class="upload">
<div class="form-horizontal">
<div class="form-group">
<div class="col-sm-offset-2 col-sm-10">
<div class="checkbox">
<label>
<input v-model="chunkEnabled" type="checkbox"> Use chunk upload
</label>
</div>
</div>
</div>
<div class="form-group">
<label for="inputMinSize" class="col-sm-2 control-label">Min Size</label>
<div class="col-sm-10">
<div class="input-group">
<input id="inputMinSize" v-model="chunkMinSize" type="number" class="form-control">
<span class="input-group-addon">MB</span>
</div>
</div>
</div>
<div class="form-group">
<label for="inputMaxActive" class="col-sm-2 control-label">Max Active Chunks</label>
<div class="col-sm-10">
<input id="inputMaxActive" v-model="chunkMaxActive" type="number" class="form-control">
</div>
</div>
<div class="form-group">
<label for="inputMaxRetries" class="col-sm-2 control-label">Max Chunk Retries</label>
<div class="col-sm-10">
<input id="inputMaxRetries" v-model="chunkMaxRetries" type="number" class="form-control">
</div>
</div>
</div>
<table class="table table-striped table-condensed">
<thead class="thead-dark">
<tr>
<th>Name</th>
<th class="text-right">Size</th>
<th class="text-right">Progress</th>
<th>Status</th>
<th>Pause</th>
<th colspan="3" class="text-center">Chunks</th>
</tr>
<tr>
<th colspan="5"></th>
<th class="text-right">Total</th>
<th class="text-right">Active</th>
<th class="text-right">Completed</th>
</tr>
</thead>
<tbody>
<template v-for="file in files">
<tr :key="file.id + '-info'">
<td>{{ file.name }}</td>
<td class="text-right">{{ file.size | formatSize }}</td>
<td class="text-right">{{ file.progress }}%</td>
<td v-if="file.error">{{ file.error }}</td>
<td v-else-if="file.success">Success</td>
<td v-else-if="file.active">Active</td>
<td v-else> - </td>
<td>
<template v-if="file.chunk">
<button
class="btn btn-sm btn-danger"
v-if="file.active"
@click="file.chunk.pause()"
>
<i class="fa fa-pause"/>
</button>
<button
class="btn btn-sm btn-primary"
v-if="!file.active && file.chunk.hasChunksToUpload"
@click="file.chunk.resume()"
>
<i class="fa fa-play"/>
</button>
</template>
</td>
<template v-if="file.chunk">
<td class="text-right">{{ file.chunk.chunks.length }}</td>
<td class="text-right">{{ file.chunk.chunksUploading.length }}</td>
<td class="text-right">{{ file.chunk.chunksUploaded.length }}</td>
</template>
<template v-else>
<td class="text-right"> - </td>
<td class="text-right"> - </td>
<td class="text-right"> - </td>
</template>
</tr>
<tr :key="file.id + '-loading'">
<td colspan="8">
<div class="chunk-loading" v-if="file.chunk">
<span
v-for="(chunk, index) in file.chunk.chunks"
:key="index"
class="chunk-loading-part"
:class="{'chunk-loading-part__uploaded': chunk.uploaded}"
>
<template v-if="chunk.retries != file.chunk.maxRetries">
{{ file.chunk.maxRetries - chunk.retries }} error(s)
</template>
</span>
</div>
</td>
</tr>
</template>
</tbody>
</table>
<div class="example-btn">
<file-upload
class="btn btn-primary"
post-action="/upload/post"
:chunk-enabled="chunkEnabled"
:chunk="{
action: '/upload/chunk',
minSize: chunkMinSize * 1048576,
maxActive: chunkMaxActive,
maxRetries: chunkMaxRetries
}"
extensions="gif,jpg,jpeg,png,webp"
accept="image/png,image/gif,image/jpeg,image/webp"
:multiple="true"
:size="1024 * 1024 * 10"
v-model="files"
@input-filter="inputFilter"
@input-file="inputFile"
ref="upload">
<i class="fa fa-plus"></i>
Select files
</file-upload>
</div>
</div>
<div class="pt-5">
Source code: <a href="https://github.com/lian-yue/vue-upload-component/blob/master/docs/views/examples/Chunk.vue">/docs/views/examples/Chunk.vue</a>
</div>
</div>
</template>
<style>
.example-simple label.btn {
margin-bottom: 0;
margin-right: 1rem;
}
</style>
<script>
import FileUpload from 'vue-upload-component'
export default {
components: {
FileUpload,
},
data() {
return {
files: [],
chunkEnabled: true,
// 1MB by default
chunkMinSize: 1,
chunkMaxActive: 3,
chunkMaxRetries: 5
}
},
methods: {
inputFilter(newFile, oldFile, prevent) {
if (newFile && !oldFile) {
// Before adding a file
//
// Filter system files or hide files
//
if (/(\/|^)(Thumbs\.db|desktop\.ini|\..+)$/.test(newFile.name)) {
return prevent()
}
// Filter php html js file
// php html js
if (/\.(php5?|html?|jsx?)$/i.test(newFile.name)) {
return prevent()
}
}
},
inputFile(newFile, oldFile) {
if (newFile && !oldFile) {
// add
console.log('add', newFile)
this.$refs.upload.active = true
}
if (newFile && oldFile) {
// update
console.log('update', newFile)
}
if (!newFile && oldFile) {
// remove
console.log('remove', oldFile)
}
}
}
}
</script>
<style scoped>
.chunk-loading {
margin: -12px;
display: flex;
width: calc(100% + 24px);
}
.chunk-loading .chunk-loading-part {
height: 25px;
line-height: 25px;
flex: 1;
background: #ccc;
font-size: 14px;
color: white;
text-align: center;
}
.chunk-loading .chunk-loading-part.chunk-loading-part__uploaded {
background: #28A745;
}
</style>

12108
package-lock.json generated

File diff suppressed because it is too large Load Diff

@ -53,8 +53,13 @@
"cross-env": "^1.0.6", "cross-env": "^1.0.6",
"css-loader": "^0.28.7", "css-loader": "^0.28.7",
"eslint": "^4.8.0", "eslint": "^4.8.0",
"eslint-config-standard": "^11.0.0-beta.0",
"eslint-loader": "^1.9.0", "eslint-loader": "^1.9.0",
"eslint-plugin-html": "^3.2.2", "eslint-plugin-html": "^3.2.2",
"eslint-plugin-import": "^2.8.0",
"eslint-plugin-node": "^5.2.1",
"eslint-plugin-promise": "^3.6.0",
"eslint-plugin-standard": "^3.0.1",
"eslint-plugin-vue": "^3.13.0", "eslint-plugin-vue": "^3.13.0",
"raw-loader": "^0.5.1", "raw-loader": "^0.5.1",
"rollup": "^0.50.0", "rollup": "^0.50.0",
@ -67,6 +72,7 @@
"vue-loader": "^13.0.5", "vue-loader": "^13.0.5",
"vue-template-compiler": "^2.4.4", "vue-template-compiler": "^2.4.4",
"webpack": "^3.6.0", "webpack": "^3.6.0",
"webpack-body-parser": "^1.11.110",
"webpack-dev-server": "^2.9.1", "webpack-dev-server": "^2.9.1",
"webpack-merge": "^4.1.0" "webpack-merge": "^4.1.0"
} }

@ -33,7 +33,21 @@
} }
</style> </style>
<script> <script>
import ChunkUploadDefaultHandler from './chunk/ChunkUploadHandler'
import InputFile from './InputFile.vue' import InputFile from './InputFile.vue'
const CHUNK_DEFAULT_OPTIONS = {
headers: {
'Content-Type': 'application/json'
},
action: '',
minSize: 1048576,
maxActive: 3,
maxRetries: 5,
handler: ChunkUploadDefaultHandler
}
export default { export default {
components: { components: {
InputFile, InputFile,
@ -123,6 +137,20 @@ export default {
type: Number, type: Number,
default: 1, default: 1,
}, },
// Chunk upload enabled
chunkEnabled: {
type: Boolean,
default: false
},
// Chunk upload properties
chunk: {
type: Object,
default: () => {
return CHUNK_DEFAULT_OPTIONS
}
}
}, },
data() { data() {
@ -216,6 +244,9 @@ export default {
return true return true
}, },
chunkOptions () {
return Object.assign(CHUNK_DEFAULT_OPTIONS, this.chunk)
},
className() { className() {
return [ return [
@ -731,14 +762,41 @@ export default {
return Promise.reject('size') return Promise.reject('size')
} }
if (this.features.html5) {
if (this.shouldUseChunkUpload(file)) {
return this.uploadChunk(file)
}
if (file.putAction) {
return this.uploadPut(file)
}
if (this.features.html5 && file.putAction) {
return this.uploadPut(file)
} else if (this.features.html5) {
return this.uploadHtml5(file) return this.uploadHtml5(file)
} else {
return this.uploadHtml4(file)
} }
return this.uploadHtml4(file)
},
/**
* Whether this file should be uploaded using chunk upload or not
*
* @param Object file
*/
shouldUseChunkUpload (file) {
return this.chunkEnabled &&
!!this.chunkOptions.handler &&
file.size > this.chunkOptions.minSize
},
/**
* Upload a file using Chunk method
*
* @param File file
*/
uploadChunk (file) {
const HandlerClass = this.chunkOptions.handler
file.chunk = new HandlerClass(file, this.chunkOptions)
return file.chunk.upload()
}, },
uploadPut(file) { uploadPut(file) {

@ -0,0 +1,329 @@
import { default as request, createRequest, sendRequest } from '../utils/request'
export default class ChunkUploadHandler {
/**
* Constructor
*
* @param {File} file
* @param {Object} options
*/
constructor (file, options) {
this.file = file
this.options = options
}
/**
* Gets the max retries from options
*/
get maxRetries () {
return parseInt(this.options.maxRetries)
}
/**
* Gets the max number of active chunks being uploaded at once from options
*/
get maxActiveChunks () {
return parseInt(this.options.maxActive)
}
/**
* Gets the file type
*/
get fileType () {
return this.file.type
}
/**
* Gets the file size
*/
get fileSize () {
return this.file.size
}
/**
* Gets action (url) to upload the file
*/
get action () {
return this.options.action || null
}
/**
* Gets the body to be merged when sending the request in start phase
*/
get startBody () {
return this.options.startBody || {}
}
/**
* Gets the body to be merged when sending the request in upload phase
*/
get uploadBody () {
return this.options.uploadBody || {}
}
/**
* Gets the body to be merged when sending the request in finish phase
*/
get finishBody () {
return this.options.finishBody || {}
}
/**
* Gets the headers of the requests from options
*/
get headers () {
return this.options.headers || {}
}
/**
* Whether it's ready to upload files or not
*/
get readyToUpload () {
return !!this.chunks
}
/**
* Gets the progress of the chunk upload
* - Gets all the completed chunks
* - Gets the progress of all the chunks that are being uploaded
*/
get progress () {
const completedProgress = (this.chunksUploaded.length / this.chunks.length) * 100
const uploadingProgress = this.chunksUploading.reduce((progress, chunk) => {
return progress + ((chunk.progress | 0) / this.chunks.length)
}, 0)
return Math.min(completedProgress + uploadingProgress, 100)
}
/**
* Gets all the chunks that are pending to be uploaded
*/
get chunksToUpload () {
return this.chunks.filter(chunk => {
return !chunk.active && !chunk.uploaded
})
}
/**
* Whether there are chunks to upload or not
*/
get hasChunksToUpload () {
return this.chunksToUpload.length > 0
}
/**
* Gets all the chunks that are uploading
*/
get chunksUploading () {
return this.chunks.filter(chunk => {
return !!chunk.xhr && !!chunk.active
})
}
/**
* Gets all the chunks that have finished uploading
*/
get chunksUploaded () {
return this.chunks.filter(chunk => {
return !!chunk.uploaded
})
}
/**
* Creates all the chunks in the initial state
*/
createChunks () {
this.chunks = []
let start = 0
let end = this.chunkSize
while (start < this.fileSize) {
this.chunks.push({
blob: this.file.file.slice(start, end),
startOffset: start,
active: false,
retries: this.maxRetries
})
start = end
end = start + this.chunkSize
}
}
/**
* Updates the progress of the file with the handler's progress
*/
updateFileProgress () {
this.file.progress = this.progress
}
/**
* Paues the upload process
* - Stops all active requests
* - Sets the file not active
*/
pause () {
this.file.active = false
this.chunksUploading.forEach(chunk => {
chunk.xhr.abort()
chunk.active = false
})
}
/**
* Resumes the file upload
* - Sets the file active
* - Starts the following chunks
*/
resume () {
this.file.active = true
this.startChunking()
}
/**
* Starts the file upload
*
* @returns Promise
* - resolve The file was uploaded
* - reject The file upload failed
*/
upload () {
this.promise = new Promise((resolve, reject) => {
this.resolve = resolve
this.reject = reject
})
this.start()
return this.promise
}
/**
* Start phase
* Sends a request to the backend to initialise the chunks
*/
start () {
request({
method: 'POST',
headers: this.headers,
url: this.action,
body: Object.assign(this.startBody, {
phase: 'start',
mime_type: this.fileType,
size: this.fileSize
})
}).then(res => {
if (res.status !== 'success') {
return this.reject(res.message)
}
this.sessionId = res.data.session_id
this.chunkSize = res.data.end_offset
this.createChunks()
this.startChunking()
}).catch(error => this.reject(error))
}
/**
* Starts to upload chunks
*/
startChunking () {
for (let i = 0; i < this.maxActiveChunks; i++) {
this.uploadNextChunk()
}
}
/**
* Uploads the next chunk
* - Won't do anything if the process is paused
* - Will start finish phase if there are no more chunks to upload
*/
uploadNextChunk () {
if (this.file.active) {
if (this.hasChunksToUpload) {
return this.uploadChunk(this.chunksToUpload[0])
}
if (this.chunksUploading.length === 0) {
return this.finish()
}
}
}
/**
* Uploads a chunk
* - Sends the chunk to the backend
* - Sets the chunk as uploaded if everything went well
* - Decreases the number of retries if anything went wrong
* - Fails if there are no more retries
*
* @param {Object} chunk
*/
uploadChunk (chunk) {
chunk.progress = 0
chunk.active = true
this.updateFileProgress()
chunk.xhr = createRequest({
method: 'POST',
headers: this.headers,
url: this.action
})
chunk.xhr.upload.addEventListener('progress', function(evt) {
if (evt.lengthComputable) {
chunk.progress = Math.round(evt.loaded / evt.total * 100)
}
}, false)
sendRequest(chunk.xhr, Object.assign(this.uploadBody, {
phase: 'upload',
session_id: this.sessionId,
start_offset: chunk.startOffset,
chunk: chunk.blob
})).then(res => {
chunk.active = false
if (res.status === 'success') {
chunk.uploaded = true
} else {
if (chunk.retries-- <= 0) {
this.pause()
return this.reject('File upload failed')
}
}
this.uploadNextChunk()
}).catch(() => {
chunk.active = false
if (chunk.retries-- <= 0) {
this.pause()
return this.reject('File upload failed')
}
this.uploadNextChunk()
})
}
/**
* Finish phase
* Sends a request to the backend to finish the process
*/
finish () {
this.updateFileProgress()
request({
method: 'POST',
headers: this.headers,
url: this.action,
body: Object.assign(this.finishBody, {
phase: 'finish',
session_id: this.sessionId
})
}).then(res => {
if (res.status !== 'success') {
return this.reject(res.message)
}
this.resolve(res)
}).catch(error => this.reject(error))
}
}

@ -0,0 +1,50 @@
/**
* Creates a XHR request
*
* @param {Object} options
*/
export const createRequest = (options) => {
const xhr = new XMLHttpRequest()
xhr.responseType = 'json'
xhr.open(options.method || 'GET', options.url)
if (options.headers) {
Object.keys(options.headers).forEach(key => {
xhr.setRequestHeader(key, options.headers[key])
})
}
return xhr
}
/**
* Sends a XHR request with certain body
*
* @param {XMLHttpRequest} xhr
* @param {Object} body
*/
export const sendRequest = (xhr, body) => {
return new Promise((resolve, reject) => {
xhr.onload = () => {
if (xhr.status >= 200 && xhr.status < 300) {
resolve(xhr.response)
} else {
reject(xhr.statusText)
}
}
xhr.onerror = () => reject(xhr.statusText)
xhr.send(JSON.stringify(body))
})
}
/**
* Creates and sends XHR request
*
* @param {Object} options
*
* @returns Promise
*/
export default function (options) {
const xhr = createRequest(options)
return sendRequest(xhr, options.body)
}

@ -4,10 +4,67 @@ const webpack = require('webpack')
const packageInfo = require('./package') const packageInfo = require('./package')
const bodyParser = require('webpack-body-parser')
process.env.NODE_ENV = process.env.NODE_ENV || 'production' process.env.NODE_ENV = process.env.NODE_ENV || 'production'
const isDev = process.env.NODE_ENV === 'development' const isDev = process.env.NODE_ENV === 'development'
const CHUNK_SIZE = 1048576
const ChunkActiveUploads = {}
const chunkUploadStart = (req, res) => {
const uuid = Math.floor((1 + Math.random()) * 0x10000)
.toString(16)
.substring(1)
ChunkActiveUploads[uuid] = {}
return res.json({
status: 'success',
data: {
session_id: uuid,
start_offset: 0,
end_offset: CHUNK_SIZE
}
})
}
const chunkUploadPart = (req, res) => {
setTimeout(() => {
const rand = Math.random()
if (rand <= 0.25) {
res.status(500)
res.json({ status: 'error', error: 'server' })
} else {
res.send({ status: 'success' })
}
}, 100 + parseInt(Math.random() * 2000, 10))
}
const chunkUploadFinish = (req, res) => {
setTimeout(() => {
const rand = Math.random()
if (rand <= 0.25) {
res.status(500)
res.json({ status: 'error', error: 'server' })
} else {
res.send({ status: 'success' })
}
}, 100 + parseInt(Math.random() * 2000, 10))
}
const chunkUpload = (req, res) => {
switch (req.body.phase) {
case 'start':
return chunkUploadStart(req, res)
case 'upload':
return chunkUploadPart(req, res)
case 'finish':
return chunkUploadFinish(req, res)
}
}
function baseConfig() { function baseConfig() {
let config = { let config = {
@ -185,6 +242,10 @@ module.exports = merge(baseConfig(), {
let del = function (req, res) { let del = function (req, res) {
res.json({ success: true }) res.json({ success: true })
} }
// Chunk upload
app.post('/upload/chunk', bodyParser.json(), chunkUpload)
app.post('/upload/post', put) app.post('/upload/post', put)
app.put('/upload/put', put) app.put('/upload/put', put)
app.post('/upload/delete', del) app.post('/upload/delete', del)

Loading…
Cancel
Save