Post

Handling File Uploads with Hapi.js

Draft updated on Invalid Date
    Default avatar

    By Jecelyn Yeen

    Handling File Uploads with Hapi.js

    This tutorial is out of date and no longer maintained.

    Introduction

    File uploading is a common feature that almost every website needs. We will go through step by step on how to handle single and multiple file(s) uploads with Hapi, save it to the database (LokiJS), and retrieve the saved file for viewing.

    The complete source code is available here: https://github.com/chybie/file-upload-hapi.

    We will be using TypeScript throughout this tutorial.

    Install Required Dependencies

    I am using Yarn for package management. However, you can use npm if you like.

    Dependencies

    Run this command to install required dependencies

    1. // run this for yarn
    2. yarn add hapi boom lokijs uuid del
    3. // or using npm
    4. npm install hapi boom lokijs uuid del --save
    • hapi: We will develop our API using HapiJs
    • boom: A plugin for Hapi, HTTP-friendly error objects
    • loki: LokiJS, a fast, in-memory document-oriented datastore for node.js, browser and cordova
    • uuid: Generate unique id
    • del: Delete files and folders

    Development Dependencies

    Since we are using TypeScript, we need to install typings files in order to have an auto-complete function (IntelliSense) during development.

    1. // run this for yarn
    2. yarn add typescript @types/hapi @types/boom @types/lokijs @types/uuid @types/del --dev
    3. // or using npm
    4. npm install typescript @types/hapi @types/boom @types/lokijs @types/uuid @types/del --save-dev

    Setup

    A couple of setup steps to go before we start.

    TypeScript Configuration

    Add a typescript configuration file. To know more about TypSscript configuration, visit https://www.typescriptlang.org/docs/handbook/tsconfig-json.html.

    // tsconfig.json
    
    {
        "compilerOptions": {
            "module": "commonjs",
            "moduleResolution": "node",
            "target": "es6",
            "noImplicitAny": false,
            "sourceMap": true,
            "outDir": "dist"
        }
    }
    
    1. The compiled JavaScript code will be output to the dist folder.
    2. Since Node.js 7.5+ supports ES6 / 2015, we will set the target as es6.

    Start Script

    Add the following scripts.

    // package.json
    
    {
      ...
      "scripts": {
        "prestart": "tsc",
        "start": "node dist/index.js"
      }
      ...
    }
    

    Later on, we can run yarn start or npm start to start our application.

    1. When we run yarn start, it will trigger prestart script first. The command tsc will read the tsconfig.json file and compile all typescript files to javascript in dist folder.
    2. Then, we will run the compiled index file dist/index.js.

    Starting Hapi Server

    Let’s start creating our Hapi server.

    // index.ts
    
    import * as Hapi from 'hapi';
    import * as Boom from 'boom';
    import * as path from 'path'
    import * as fs from 'fs';
    import * as Loki from 'lokijs';
    
    // setup
    const DB_NAME = 'db.json';
    const COLLECTION_NAME = 'images';
    const UPLOAD_PATH = 'uploads';
    const fileOptions = { dest: `${UPLOAD_PATH}/` };
    const db = new Loki(`${UPLOAD_PATH}/${DB_NAME}`, { persistenceMethod: 'fs' });
    
    // create folder for upload if not exist
    if (!fs.existsSync(UPLOAD_PATH)) fs.mkdirSync(UPLOAD_PATH);
    
    // app
    const app = new Hapi.Server();
    app.connection({
        port: 3001, host: 'localhost',
        routes: { cors: true }
    });
    
    // start our app
    app.start((err) => {
    
        if (err) {
            throw err;
        }
        console.log(`Server running at: ${app.info.uri}`);
    });
    
    

    The code is pretty expressive itself. We set the connection port to 3001, allow Cross-Origin Resource Sharing (CORS), and start the server.

    Upload a Single File

    Let’s create our first route. We will create a route to allow users to upload their profile avatar.

    Route

    // index.ts
    ...
    import {
      loadCollection, uploader
    } from './utils';
    ...
    
    app.route({
      method: 'POST',
      path: '/profile',
      config: {
        payload: {
          output: 'stream',
          allow: 'multipart/form-data' // important
        }
      },
      handler: async function (request, reply) {
        try {
          const data = request.payload;
          const file = data['avatar']; // accept a field call avatar
    
          // save the file
          const fileDetails = await uploader(file, fileOptions);
    
          // save data to database
          const col = await loadCollection(COLLECTION_NAME, db);
          const result = col.insert(fileDetails);
          db.saveDatabase();
    
          // return result
          reply({ id: result.$loki, fileName: result.filename, originalName: result.originalname });
    
        } catch (err) {
          // error handling
          reply(Boom.badRequest(err.message, err));
        }
      }
    });
    
    1. This is an HTTP POST function.
    2. We configure the payload to allow multipart/form-data and receive the data as a stream.
    3. We will read the field avatar for file upload.
    4. We will call uploader function (we will create it soon) to save the input file.
    5. Then, we will load the LokiJS images table / collection (we will create loadCollection next) and create a new record.
    6. Save the database.
    7. Return result.

    Load LokiJS Collection

    A generic function to retrieve a LokiJS collection if exists, or create a new one if it doesn’t.

    // utils.ts
    
    import * as del from 'del';
    import * as Loki from 'lokijs';
    import * as fs from 'fs';
    import * as uuid from 'uuid;
    
    const loadCollection = function (colName, db: Loki): Promise<LokiCollection<any>> {
        return new Promise(resolve => {
            db.loadDatabase({}, () => {
                const _collection = db.getCollection(colName) || db.addCollection(colName);
                resolve(_collection);
            })
        });
    }
    
    export { loadCollection }
    
    

    Uploader Function

    Our uploader will handle single file upload and multiple file upload (will create later).

    // utils.ts
    ...
    
    const uploader = function (file: any, options: FileUploaderOption) {
        if (!file) throw new Error('no file(s)');
    
        return _fileHandler(file, options);
    }
    
    const _fileHandler = function (file: any, options: FileUploaderOption) {
        if (!file) throw new Error('no file');
    
        const orignalname = file.hapi.filename;
        const filename = uuid.v1();
        const path = `${options.dest}${filename}`;
        const fileStream = fs.createWriteStream(path);
    
        return new Promise((resolve, reject) => {
            file.on('error', function (err) {
                reject(err);
            });
    
            file.pipe(fileStream);
    
            file.on('end', function (err) {
                const fileDetails: FileDetails = {
                    fieldname: file.hapi.name,
                    originalname: file.hapi.filename,
                    filename,
                    mimetype: file.hapi.headers['content-type'],
                    destination: `${options.dest}`,
                    path,
                    size: fs.statSync(path).size,
                }
    
                resolve(fileDetails);
            })
        })
    }
    
    ...
    
    export { loadCollection, uploader }
    
    1. We will read the uploaded file name.
    2. We will generate a random UUID as the new file name to avoid name conflicts.
    3. We will then stream and write the file to the defined folder. It’s uploads folder for our case.

    Run Our Application

    You may run the application with yarn start. I try to call the locahost:3001/profile API with (Postman)[https://www.getpostman.com/apps], a GUI application for API testing.

    When I upload a file, you can see that a new file is created in uploads folder, and the database file db.json is created as well.

    When I issue a call without passing in avatar, an error will be returned.

    Upload single file

    Filter File Type

    We can handle file upload successfully now. Next, we need to limit the file type to image only. To do this, let’s create a filter function that will test the file extensions, then modify our _fileHandler to accept an optional filter option.

    // utils.ts
    
    ...
    
    const imageFilter = function (fileName: string) {
        // accept image only
        if (!fileName.match(/\.(jpg|jpeg|png|gif)$/)) {
            return false;
        }
    
        return true;
    };
    
    const _fileHandler = function (file: any, options: FileUploaderOption) {
        if (!file) throw new Error('no file');
    
        // apply filter if exists
        if (options.fileFilter && !options.fileFilter(file.hapi.filename)) {
            throw new Error('type not allowed');
        }
    
        ...
    }
    
    ...
    
    export { imageFilter, loadCollection, uploader }
    

    Apply the Image Filter

    We need to tell the uploader to apply our image filter function. Add it in fileOptions variable.

    // index.ts
    import {
        imageFilter, loadCollection, uploader
    } from './utils';
    
    ..
    // setup
    ...
    
    const fileOptions: FileUploaderOption = { dest: `${UPLOAD_PATH}/`, fileFilter: imageFilter };
    
    ...
    

    Restart the application, try to upload a non-image file and you should get an error.

    Upload Multiple Files

    Let’s proceed to handle multiple files upload now. We will create a new route to allow user to upload their photos.

    Route

    ...
    
    app.route({
        method: 'POST',
        path: '/photos/upload',
        config: {
            payload: {
                output: 'stream',
                allow: 'multipart/form-data'
            }
        },
        handler: async function (request, reply) {
            try {
                const data = request.payload;
                const files = data['photos'];
    
                const filesDetails = await uploader(files, fileOptions);
                const col = await loadCollection(COLLECTION_NAME, db);
                const result = [].concat(col.insert(filesDetails));
    
                db.saveDatabase();
                reply(result.map(x => ({ id: x.$loki, fileName: x.filename, originalName: x.originalname })));
            } catch (err) {
                reply(Boom.badRequest(err.message, err));
            }
        }
    });
    
    ...
    

    The code is similar to single file upload, except we accept a field photos instead of avatar, accept an array of files as input, and reply the result as an array.

    Modify Uploader Function

    We need to modify our uploader function to handle multiple files upload.

    // utils.ts
    ...
    
    const uploader = function (file: any, options: FileUploaderOption) {
        if (!file) throw new Error('no file(s)');
    
        // update this line to accept single or multiple files
        return Array.isArray(file) ? _filesHandler(file, options) : _fileHandler(file, options);
    }
    
    const _filesHandler = function (files: any[], options: FileUploaderOption) {
        if (!files || !Array.isArray(files)) throw new Error('no files');
    
        const promises = files.map(x => _fileHandler(x, options));
        return Promise.all(promises);
    }
    
    ...
    

    Retrieve List of Images

    Next, create a route to retrieve all images.

    // index.ts
    ...
    
    app.route({
        method: 'GET',
        path: '/images',
        handler: async function (request, reply) {
            try {
                const col = await loadCollection(COLLECTION_NAME, db)
                reply(col.data);
            } catch (err) {
                reply(Boom.badRequest(err.message, err));
            }
        }
    });
    
    ...
    

    The code is super easy to understand.

    Retrieve Image by Id

    Next, create a route to retrieve an image by id.

    // index.ts
    ...
    
    app.route({
        method: 'GET',
        path: '/images/{id}',
        handler: async function (request, reply) {
            try {
                const col = await loadCollection(COLLECTION_NAME, db)
                const result = col.get(request.params['id']);
    
                if (!result) {
                    reply(Boom.notFound());
                    return;
                };
    
                reply(fs.createReadStream(path.join(UPLOAD_PATH, result.filename)))
                    .header('Content-Type', result.mimetype); // important
            } catch (err) {
                reply(Boom.badRequest(err.message, err));
            }
        }
    });
    
    ...
    
    1. We will return 404 if the image does not exist in the database.
    2. We will stream the file as output, set the content-type correctly so our client or browser knows how to handle it.

    Run the Application

    Now restart the application, upload a couple of images, and retrieve it by id. You should see the image is return as an image instead of a JSON object.

    Get image by id

    Clear All Data When Restart

    Sometimes, you might want to clear all the images and database collection during development. Here’s a helper function to do so.

    // utils.ts
    
    ....
    
    const cleanFolder = function (folderPath) {
        // delete files inside folder but not the folder itself
        del.sync([`${folderPath}/**`, `!${folderPath}`]);
    };
    
    ...
    
    export { imageFilter, loadCollection, cleanFolder, uploader }
    
    
    // index.ts
    
    // setup
    ...
    
    // optional: clean all data before start
    cleanFolder(UPLOAD_PATH);
    if (!fs.existsSync(UPLOAD_PATH)) fs.mkdirSync(UPLOAD_PATH);
    
    ...
    

    Conclusion

    Handling file(s) uploads with Hapi is not as hard as you thought.

    The complete source code is available here: https://github.com/chybie/file-upload-hapi.

    That’s it. Happy coding.

    Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

    Learn more about us


    About the authors
    Default avatar
    Jecelyn Yeen

    author

    Still looking for an answer?

    Ask a questionSearch for more help

    Was this helpful?
     
    Leave a comment
    

    This textbox defaults to using Markdown to format your answer.

    You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

    Try DigitalOcean for free

    Click below to sign up and get $200 of credit to try our products over 60 days!

    Sign up

    Join the Tech Talk
    Success! Thank you! Please check your email for further details.

    Please complete your information!

    Get our biweekly newsletter

    Sign up for Infrastructure as a Newsletter.

    Hollie's Hub for Good

    Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

    Become a contributor

    Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

    Welcome to the developer cloud

    DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

    Learn more
    DigitalOcean Cloud Control Panel