Node Express Endpoint to Upload and Process CSV File

It’s very common in Node/Express backend development to come across a requirement of parsing and processing CSV file. Perhaps storing the content in database after processing.

This post will do just that. We’ll upload a CSV file, store it temporarily in server’s directory to fetch all rows in JSON form, push all rows in an array, and remove the temporary file.

Let’s jump straight into it. Here are the steps:

Create index.js file for simple express server. Include npm packages fs http, multer, fast-csv and express (install with npm install by including the packages in dependency as shown in the end or install them by running npm install --save multer express fast-csv command. fs and http comes with node, so no need of installation). Also define Router.

const http = require('http');
const fs = require('fs');

const express = require('express');
const multer = require('multer');
const csv = require('fast-csv');

const Router = express.Router;

Define multer upload directory ‘tmp/csv/’ as destination for CSV file to be uploaded temporarily. Also add definitions for express server app.

const upload = multer({ dest: 'tmp/csv/' });
const app = express();
const router = new Router();
const server = http.createServer(app);
const port = 9000

Define upload-csv endpoint with app.use and add post route. Include upload.single('file') as second argument

router.post('/', upload.single('file'), function (req, res) {
});

app.use('/upload-csv', router);

Add code to start server at the end:

function startServer() {
  server.listen(port, function () {
    console.log('Express server listening on ', port);
  });
}

setImmediate(startServer);

Now on hitting /upload-csv endpoint with a CSV file, req.file will contain the uploaded file information. We’re only interested in req.file.path, passing which to csv’s fromPath method opens the file and returns one row at a time on data event in JSON form.

As we’re interested in processing data only and not the file itself, we push each row in an array so that temporary file can be removed later.

  const fileRows = [];
  csv.fromPath(req.file.path)
    .on("data", function (data) {
      fileRows.push(data); // push each row
    })

Another event end is fired when all rows are returned in data event callback. Since we have stored them in array we can safely remove the temporary file. We use fs.unlinkSync.

  const fileRows = [];
  csv.fromPath(req.file.path)
    .on("data", function (data) {
      fileRows.push(data); // push each row
    })
    .on("end", function () {
      console.log(fileRows) //contains array of arrays. Each inner array represents row of the csv file, with each element of it a column
      fs.unlinkSync(req.file.path);   // remove temp file
      //process "fileRows" and respond
    })

And that’s it! For validation of uploaded CSV data check Validate Uploaded CSV Data In Node Express.

Putting server code and package.json for npm packages all together:

index.js

'use strict';

const http = require('http');
const fs = require('fs');

const express = require('express');
const multer = require('multer');
const csv = require('fast-csv');

const Router = express.Router;
const upload = multer({ dest: 'tmp/csv/' });
const app = express();
const router = new Router();
const server = http.createServer(app);
const port = 9000

router.post('/', upload.single('file'), function (req, res) {
  const fileRows = [];

  // open uploaded file
  csv.fromPath(req.file.path)
    .on("data", function (data) {
      fileRows.push(data); // push each row
    })
    .on("end", function () {
      console.log(fileRows)
      fs.unlinkSync(req.file.path);   // remove temp file
      //process "fileRows" and respond
    })
});

app.use('/upload-csv', router);

// Start server
function startServer() {
  server.listen(port, function () {
    console.log('Express server listening on ', port);
  });
}

setImmediate(startServer);

package.json

{
  "name": "csv-upload",
  "main": "index.js",
  "dependencies": {
    "express": "^4.16.3",
    "fast-csv": "^2.4.1",
    "multer": "^1.3.0"
  }
}

Also check Export Mongodb Collection Data In CSV From Node Express Mongoose Application