RSS

Tag Archives: #ux

Best Practices for Easy Bulk Data Configuration

A warehouse scene featuring a forklift driver transporting a wooden crate labeled 'DATA'. A woman in a blue jumpsuit stands with a clipboard, checking items off a list. The background shows shelves of boxes and a rules sign on the wall.

Our customer-facing portal and some of our internal tools support bulk configuration via file upload. Over time, our experience has removed a lot of friction from the process making it a great enabler of efficient workflows. A few rules guide our work.

Meet Your Users Where They Are

Of the myriad file formats that can be used for structured data (XML, JSON, CSV, XLSX), some are well-suited for scripts and others are more human-friendly. In business, the universal editor for tabular data is Microsoft Excel. Libraries for C#, Python, and Java (among many other languages) make reading (and usually writing) Excel spreadsheets straightforward.

CSV is an important second option, especially in Linux-heavy environments. The files can be edited in the user’s text editor of choice and tools like CSV Kit allow scripting to manipulate the data. Of course CSV lacks formatting and calculations, but bulk configuration rarely has a need for those.

Rule 1: Support native Excel (.xlsx) first, and offer CSV as a lightweight alternative.

Tell Them What You Expect

It’s a truism that users don’t read documentation. You can write pages describing your upload requirements, but showing is better than telling. Right next to every upload button, provide a one-click download for a sample template with headers your system expects.

Rule 2: Provide a downloadable template at the point of upload.

Ignore What You Don’t Care About

Postel’s Law tells us to be liberal in what we accept. If an uploaded file has an extra column you don’t recognize, is that an error? Not really, you can just ignore it. Don’t assume your system is the only destination for the file. Similarly, ignore case when handling column headers; Username, username, and USERNAME should all map to the same field.

Rule 3: Be permissive with extra data and case variation in headers.

Enforce What You Do Care About

Validating each field in each row is standard practice. But bulk data processing must also have batch-level validation. Before you commit any data from an uploaded file, validate the entire batch, including across rows and against existing data. It’s easy to make sure every value in the batch is unique where necessary. But every value in that column must also be distinct from every value in the database you’re updating. An important edge case arises when swapping two values: the first row uses a value already stored in the database but when the whole batch is processed another row updates the database so the conflict no longer exists. Make sure your upload can handle swapping unique values.

Rule 4: Validate the entire payload against the target state before executing any writes.

Give Specific Feedback

If the user uploads a file of 100 rows, “Update failed” is not useful feedback. “Username ‘asdf$1234’ on row 4 is invalid; usernames cannot include $ or @” tells the user what they did wrong, where, and how to fix it.

Rule 5: Point directly to the row, column, and business logic error.

Give Exhaustive Feedback

It’s incredibly inefficient to stop at the first row with invalid data and make the user iterate until all the problems are fixed. Validation should process all the rows and report all the problems it finds. In the user interface, make it easy to copy that list of problems to another window for reference as the user works through addressing them.

Rule 6: Report every error in the file at once.

Data Rows Start at 2

Programmers may start counting at zero but normal humans do not. Bulk uploads have a header row and data starts at row 2. Provide feedback map internal indices to something natural to the user as seen in their editor.

Rule 7: Translate internal 0-indexed arrays to match the spreadsheet UI.

Frictionless Uploads Build Trust

File uploads are often treated as an unglamorous utility, but for power users, they are a primary interface to your system. Every uninformative error message, rigid schema demand, or broken row index adds unnecessary friction to their workday. By treating bulk upload as a core product feature (one that is forgiving on input and explicit on feedback), you turn a potentially frustrating task into a reliable, efficient workflow.

 
Leave a comment

Posted by on September 22, 2026 in Software techniques

 

Tags: , ,

Storage is Essentially Free!

Twice in the last few days, I ran into low, artificial limits on how much text a website would allow me to enter into a field.

Once upon a time, this was reasonable. Disk was expensive. RAM was expensive. Constraints were real, and tradeoffs were unavoidable.

That time is long past.

Out of curiosity, I asked Gemini to help me research the history of nonvolatile storage costs. This is what I learned:

  • In 1956, an IBM 350 provided about 3.75 MB of nonvolatile storage for $9,200/MB.
  • Today — during a price spike caused by a NAND shortage — a 2 TB SSD costs around $250.

That’s a 99.99999864% reduction in price per megabyte, without adjusting the 1956 price for nearly 70 years of inflation. For all practical purposes, storage is free.

And yet, both places I hit limits this week were meant to hold directions.

In one case, I was updating instructions to cover a scenario that had been missed in the previous version. I couldn’t add the necessary text without hitting the limit. I trimmed the existing content and ended up with something serviceable, but fragile. The very next update will run into the same wall.

This is absurd.

Text (especially text meant to instruct) should be effectively infinite. Modern disks can handle it. Modern memory can handle it. The constraint is not technical; it’s conceptual.

(And don’t get me started on amateurs who think eight characters is enough for a first name, so I end up with mail addressed to “Christop!” See Falsehoods Programmers Believe About Names.)

 
Leave a comment

Posted by on February 10, 2026 in Software techniques

 

Tags: , , ,