About File Splitter

Split large files into chunks by size or line count. Part of the DevTools Surf developer suite. Browse more tools in the Data / SQL collection.

Use Cases

• Split large CSV files into processable chunks for import into databases
• Divide large log files for parallel processing across multiple machines
• Create file chunks for upload to services with per-file size limits
• Prepare text corpus splits for machine learning training, validation, and test sets

Tips

• Choose between size-based splitting (e.g., 10MB chunks) and line-count splitting (e.g., 1000 rows) — line-count is better for CSV files where SQL import tools have row limits
• The overlap mode copies the last N lines of each chunk to the start of the next — useful for sliding window processing or context-aware NLP tasks
• Enable the naming convention option to add sequential suffixes (file.001, file.002) rather than timestamps — easier for automated reassembly

Fun Facts

• The Unix 'split' command, first included in AT&T Unix in the 1970s, is still widely used for splitting large files. The option to split by byte count (-b) was added for binary file splitting — enabling files to be distributed across multiple floppy disks.
• Email attachments historically had a 10MB size limit enforced by many mail servers — driving the practice of manually splitting files for email. The attachment limit has grown to 25MB (Gmail), but cloud storage links have largely replaced large email attachments for files above 5MB.
• HDF5 (Hierarchical Data Format 5), used by NASA and scientific computing communities for large datasets, supports files larger than 2^64 bytes theoretically — avoiding the need to split very large scientific data files. HDF5 files can contain multiple datasets with different chunk sizes for partial access.

FAQ

How do I reassemble split files?

Use the file-merger tool to concatenate the parts in order. On Linux/Mac, 'cat file.001 file.002 file.003 > file_complete' works for binary and text parts. Verify the reassembled file hash matches the original if you saved it.

Should I split by size or by line count?

Split by size for binary files and when downstream tools have file size limits. Split by line count for text/CSV when you need each chunk to have exactly N rows (e.g., database batch inserts, API rate limits per call). Byte splits may split mid-line in CSV files.

What happens to the header row when splitting CSV files?

The tool's CSV mode automatically copies the header row to the first row of every chunk — so each split file is a valid, independently usable CSV. Plain text mode does not add headers; each chunk starts with the next line after where the previous chunk ended.