CSV is simple to transport and surprisingly easy to misinterpret. A product code can lose a leading zero, a date can be read in the wrong order and an empty cell can mean either leave unchanged or delete the value. A reliable import makes those decisions explicit before it changes records.
Define the file contract
Publish the expected headers, encoding and field rules with a small example file. State whether identifiers are strings, which date format is accepted and how blanks are handled. Reject missing required columns with a specific message. Do not let the order of spreadsheet columns decide which database fields receive their values.
Use a CSV parser rather than splitting text on commas. Python's CSV module documentation explains dialects, quoting and newline handling; equivalent parser libraries exist in other stacks. Parsing tells you what a cell contains. It does not tell you whether that value is valid for your business operation.
Show a row-level preview
Validate the parsed records before starting writes. Present the number of proposed creates, updates, unchanged rows and rejected rows. Let the user download or inspect an error report that identifies the source row and the correction required. For a catalogue update, show both the old value and the proposed value for sensitive fields such as availability.
Decide how duplicate identifiers within one file behave. Failing the affected rows is often clearer than silently keeping the last occurrence. A preview should also detect conflicts with existing records. If another user edits a record after the preview, the commit needs a version check or a clearly defined reconciliation rule.
Make the commit recoverable
Choose whether the import is atomic or permits partial success, and explain the choice before confirmation. Store an import identifier and a record of applied changes. A retry should resume or recognise the completed job rather than create the same records again. Avoid promising an undo action unless the system can restore the previous values safely.
For a data engineering project, the preview report and reconciliation process are part of the import feature. The same principle appears in our webhook retry guide: repeated delivery must not become a repeated business action.
