Exercise: Photo Contest Uploads
Finish Snapshot Saturday's API, accepting one photo per entry with multer's size and type limits, answering 202 while a small job queue makes each thumbnail after the response, and streaming every entry as a CSV download with pipeline, with Supertest and node --test checking each part.
Freelancer · Node.js · Web Development · Updated
Snapshot Saturday, the town's photo contest, opens for entries next week. You'll finish its API: entrants send a photo with their name and its title, the API checks the photo's size and type and saves it under a name it makes, answers at once with 202 while a background job makes the thumbnail, and the judges download every entry as a CSV. Tests check every part as you go.
Get the exercise code
Open this lesson's GitHub repository, github.com/ask-codey/exercise-node-photo-contest-uploads, also linked from the Exercise box beside the lesson. Clone it into your nodejs-course folder. In VS Code's terminal:
cd ~/nodejs-course
git clone https://github.com/ask-codey/exercise-node-photo-contest-uploads.git
Git makes a folder, exercise-node-photo-contest-uploads, with the exercise code inside. Open that folder in VS Code (File > Open Folder), open its terminal with Terminal > New Terminal, and install the packages:
npm install
Get the Exercise Code with Git walks through cloning step by step, if you'd like a reminder.
The folder holds:
app.js: the Express app,export const app = express(), for you to finish, withsendProblem(res, status, detail)from Error Handling and Consistent Error Responses ready to send every error.queue.js: the job queue,addJob(work)andwhenIdle(), for you to finish.entries.js: the contest's three entries so far, kept in memory, withnextId()andresetEntries().csv.js:csvHeaderandtoCsvLine(entry), which turn entries into lines of CSV text, commas and quotes taken care of.thumbnails.js:makeThumbnail(photo), a stand-in for an image library that waits 100 milliseconds and answers with the thumbnail's name.folders.js:folders.uploads, the folder photos are saved in. The tests point it at a temporary folder of their own.server.js: makes theuploadsfolder and starts the API athttp://localhost:3100.fixtures/: three small files to upload,sunset.jpg,lighthouse.pngandentry-rules.txt.contest.http: the requests to try with REST Client, uploads included.app.test.jsandqueue.test.js: the tests. You don't need to change them.
What to do
app.js answers every request with a 404 that says it isn't built yet, and both functions in queue.js throw an error that says the same. The comment at the top of app.js lists every request. Every error answer has the course's one error shape, { status, title, detail }, with the exact detail shown:
// GET /entries.csv 200, every entry as CSV, streamed with pipeline (not built up in one string first):
// Content-Type text/csv; charset=utf-8, Content-Disposition attachment; filename=entries.csv
// the body is csvHeader, then toCsvLine(entry) for each entry in entries, in order
// GET /entries/2 200, entry 2 (every entry's id works the same way)
// GET /entries/99 404, { "status": 404, "title": "Not Found", "detail": "There's no entry 99." }
//
// POST /entries a multipart/form-data form: the text fields name and title, and one photo in the field photo.
// 202, the new entry: { id: nextId(), name, title, photo, thumbnail: null },
// also added to entries, with the header Location: /entries/<its id>.
// photo is the name the photo was saved under, in the folder folders.uploads:
// a new random UUID from node:crypto, then .jpg for image/jpeg or .png for image/png,
// never the file name the client sent.
// Then, after the answer, a job on the queue (addJob in queue.js) sets the entry's
// thumbnail to await makeThumbnail(photo).
// 400, { "status": 400, "title": "Bad Request", "detail": "Only JPEG and PNG photos can enter." }
// for any other type of file
// 413, { "status": 413, "title": "Payload Too Large", "detail": "A photo can be at most 5 MB." }
// for a photo over 5 MB (5 * 1024 * 1024 bytes; exactly 5 MB is fine)
// 400, { "status": 400, "title": "Bad Request", "detail": "Attach one photo in the photo field." }
// with no file, or a file in any other field
// A refused upload leaves no file behind in the upload folder.
And the comment at the top of queue.js describes the queue:
// addJob(work) work is an async function. addJob puts it at the back of the queue and returns at once,
// without waiting for the job to finish.
// Jobs run one at a time, in the order they were added: a job starts only when the one before
// it has finished.
// A job that throws doesn't stop the queue: log it with
// console.error(`A background job failed: ${err.message}`)
// and go on to the next job.
// whenIdle() Returns a promise that resolves once every job added so far has finished (or failed).
// When no job is waiting or running, it resolves at once.
- Build the queue in
queue.js:addJobfirst, thenwhenIdle. Runnpm testand watchqueue.test.jsturn green. - Answer
GET /entries/:idwith the entry, or the 404. - Stream
GET /entries.csvwithpipeline, as a download namedentries.csv. - Accept a photo with multer: saved in
folders.uploadsunder a name the server makes, with the size limit and the allowed types, answering 202 with theLocationheader. - Queue the thumbnail after the answer, with
addJob. - Turn every refusal into its answer in an error-handling middleware, with
sendProblem: 413 for a photo that's too big, 400 for the wrong type or the wrong field.
Check your work
Run the tests from the exercise folder:
npm test
No server needs to be running: the tests hand each request straight to app, and they save uploads in a temporary folder they make and remove themselves. At first all 19 tests fail. That's expected: each part you build turns its tests green. A failing test's message says what came back, for example POST /entries should answer 202, but it answered 404: {"message":"Not built yet."}
To try the API yourself, start it with npm start (or npm run dev, which restarts it each time you save), open contest.http, and click Send Request above each request. Your uploads land in the uploads folder, which .gitignore keeps out of Git. Stop the server with Ctrl+C.
Hints
- Read the folder at the moment of each upload.
destinationcan be a function,(req, file, cb) => cb(null, folders.uploads), so it sees the folder the tests choose. File Uploads with Limits showsdiskStoragewith afilenamefunction in the same style. - The extension comes from your own list. An object such as
{ "image/jpeg": ".jpg", "image/png": ".png" }answers both questions: is this type allowed, and which extension does it get? - multer's errors have a
code. A file over the limit isLIMIT_FILE_SIZE, and a file in a field other thanphotoisLIMIT_UNEXPECTED_FILE. File Uploads with Limits gives your ownfileFiltererror astatusof 400 for the error-handling middleware to read. - The CSV is a list of lines.
Readable.from([csvHeader, ...entries.map(toCsvLine)])is a readable stream, and the response is the writable one. Streams: Big Files a Piece at a Time sends a download withres.attachmentandpipeline. - The job changes the entry itself. Inside the job,
entry.thumbnail = await makeThumbnail(entry.photo)updates the same object that's inentries, soGET /entries/4sees it. Answer withres.status(202).location(...)as Work That Runs After the Response does. whenIdlekeeps promises waiting. Keep theresolvefunction of each promise it hands out in a list,new Promise((resolve) => waiters.push(resolve)), and call them all whenrunNextfinds nothing left to do.
When you're done
Every test is green, and with npm start running, contest.http's requests get these answers (each request, then its answer's status line, the headers that matter, and the body). Your photo names are new UUIDs, so yours differ:
GET /entries.csv
HTTP/1.1 200 OK
Content-Type: text/csv; charset=utf-8
Content-Disposition: attachment; filename=entries.csv
id,name,title,photo,thumbnail
1,Ava,Harbor at dawn,3f2b7c1e-5d4a-4c8e-9b1f-2a6d8e0c4b71.jpg,thumb-3f2b7c1e-5d4a-4c8e-9b1f-2a6d8e0c4b71.jpg
2,Mateo,"Fog, over the bridge",8c1d4e2a-7b3f-4a9c-8e5d-1f0b6c2a9d34.png,thumb-8c1d4e2a-7b3f-4a9c-8e5d-1f0b6c2a9d34.png
3,Grace,"The ""lucky"" oak",b7e9a0d3-2c6f-4e1b-a8d4-5f3c9e7b0a12.jpg,thumb-b7e9a0d3-2c6f-4e1b-a8d4-5f3c9e7b0a12.jpg
POST /entries, Noor's sunset.jpg
HTTP/1.1 202 Accepted
Location: /entries/4
{"id":4,"name":"Noor","title":"Lighthouse in the rain","photo":"8e1b461a-6daa-408c-a58d-f5f7123a663c.jpg","thumbnail":null}
GET /entries/4, a moment later
HTTP/1.1 200 OK
{"id":4,"name":"Noor","title":"Lighthouse in the rain","photo":"8e1b461a-6daa-408c-a58d-f5f7123a663c.jpg","thumbnail":"thumb-8e1b461a-6daa-408c-a58d-f5f7123a663c.jpg"}
POST /entries, entry-rules.txt
HTTP/1.1 400 Bad Request
{"status":400,"title":"Bad Request","detail":"Only JPEG and PNG photos can enter."}
POST /entries, a 6 MB photo
HTTP/1.1 413 Payload Too Large
{"status":413,"title":"Payload Too Large","detail":"A photo can be at most 5 MB."}
The CSV puts Mateo's title in double quotes because of its comma, and writes each " in Grace's title twice: that's how a CSV keeps one value in one column.
Now look back at your work. If the server restarted between Noor's 202 and her thumbnail, what would GET /entries/4 show, and what would a job service such as BullMQ do differently?
Submit your exercise
The exercise is how you finish this lesson. When your work is done, put it on GitHub in a repo of your own, then submit that repo's address here. Hand In Your Work with Git walks through each step.
On GitHub, click +, then New repository. Name it (for example
exercise-node-photo-contest-uploads), choose Public, and click Create repository. Leave out the README: your folder already has one.In VS Code's terminal, in your exercise folder, send your work to your new repo. Point your copy at your own repo, then commit and push:
git remote set-url origin https://github.com/your-name/exercise-node-photo-contest-uploads.git git add . git commit -m "My solution" git push -u origin mainYour test uploads stay behind:
.gitignorekeeps theuploadsfolder out ofgit add ..Paste your repo's address, such as
https://github.com/your-name/exercise-node-photo-contest-uploads, into Submit your exercise on this page. The site checks that the repo is public and holds your commits, and then the lesson is complete.
Changed something after you submitted? Push it, and submit again: your latest submission is the one that counts.
So what
Uploads with limits, a quick 202 with the slow part queued, and a download that streams: that's the shape of most real file features, from profile pictures to monthly reports, and you've now built each piece and tested it.