Intermediate · Practise · 30 min · Estimated time
Validate a CSV with Python before automating tasks
Run a local example to detect duplicate identifiers and incomplete rows without changing the original file or installing packages.
By Eric Muriel
Before you start
You will build: A reproducible report: two valid rows, one duplicate and one incomplete row.
- Python 3 with the standard csv and pathlib modules; a text editor and terminal.
- Know how to open a folder in a terminal. No API or paid account required.
Example executed and checked locally with Python 3.9.6, including empty values, duplicates and incorrect headers.
01Prepare the working folder
Create an empty folder, download both tutorial files and save them there. Open your terminal in that folder. Check the version below. On Windows you can use py instead of python3 throughout.
python3 --versionCheck the result
The terminal shows Python 3.x and the files are named check_csv.py and tasks.csv.
If it does not work
If Python is missing, install a version supported by your system from python.org. Remove any extra .txt extension your editor added to the script.
02Inspect the data before running
Open tasks.csv as text rather than as a spreadsheet. The first line names the columns. T-1 appears twice and the last row has no ID. The code uses the csv library to handle quoting and delimiters.
id,task
T-1,Review draft
T-1,Review draft
T-2,Check source
,Missing identifierCheck the result
Four data rows plus a header. Keep the original unchanged for this first check.
If it does not work
If you see semicolons instead of commas, download the example again. The script requires exactly the columns id,task.
03Run and compare the report
Run the command from the same folder. The script reads the file and prints a summary; it does not write or send data. Each valid ID counts once, repeats count separately, and rows missing required values are invalid.
python3 check_csv.py tasks.csvCheck the result
{'valid': 2, 'duplicates': 1, 'invalid': 1}
If it does not work
FileNotFoundError indicates a wrong folder or filename. Expected columns indicates a different header. Open the file as text to check it.
04Make one controlled change
Copy tasks.csv to tasks-test.csv. In the copy, replace the last row with T-3,Check date. Run the script using the copy’s filename. Making just one change helps explain the different result.
python3 check_csv.py tasks-test.csvCheck the result
{'valid': 3, 'duplicates': 1, 'invalid': 0}
If it does not work
If the result does not change, check that you saved the copy and passed its filename. Resolve duplicate records before importing them.
Sources, verification and limitations
The exercise checks basic structure, not task meaning. A repeated ID with different text counts as a duplicate requiring review; it is not merged or deleted.
How this content is prepared