I recently ran into an issue with compression() not working for a local dev server that we are running with Node and express.
We had been using the dev server for ages and it wasn't until we had a project which was trying to load a much larger data.json file that we noticed something was wrong. We were trying to load a 10Mb JSON file via the endpoint api/project/:project and noticed that it was not being compressed.
Here is a simplified version of our main index.js file that we use to serve our API and other endpoints using Express.
const http = require('http');
const express = require('express');
const compression = require('compression');
const apiRouter = require.main.require('./test/server/router.api.js');
const app = express();
app.use(compression());
app.use(cors({ origin: '*', methods: '*' }));
app.use(express.urlencoded({ extended: true }));
app.use('/api', apiRouter);
http.createServer({ }, app).listen(3000, () => {
console.log('secure server running on 3000');
});
In a nutshell, we are passing all the requests that start with /api to the router.api.js file where we parse the projectId from the URL and then serve up the relevant data.json file.
router.get('/project/:projectId', (request, response) => {
const { projectId } = request.params;
const data = fs.readFileSync(`${__dirname}/project/${projectId}/data.json`);
response.send(data);
});
I tried all kinds of silly things and couldn’t find any direct explanation for what might be going on and after reading lots of comments attached to articles or stack-overflow answers I managed to figure out that the solution is actually very simple.
All I needed to do was make sure that the Content-Type was correctly set for the response. The 10Mb file was now compressed down to about 400kb.
Thinking about this now, this of course makes sense, how could the compression() algorithm know how to compress the data if it doesn’t know the kind of data that it is trying to compress?
It was a silly mistake and we had missed it because everything was working. The only thing that was not actually working correctly on this route was the compression and most of our JSON files were much smaller (less than 250kb).
router.get('/project/:projectId', (request, response) => {
const { projectId } = request.params;
const data = fs.readFileSync(`${__dirname}/project/${projectId}/data.json`);
response.setHeader('Content-Type', 'application/json');
response.send(data);
});
