Posted By

eme_dlr on 09/14/10


Tagged

csv split


Versions (?)

Who likes this?

1 person have marked this snippet as a favorite

jeremydouglass


Split large CSV file in several files


 / Published in: Bash
 

  1. #!/bin/bash
  2.  
  3. # check if an input filename was passed as a command
  4. # line argument:
  5. if [ ! $# == 1 ]; then
  6. echo "Please specify the name of a file to split!"
  7. exit
  8. fi
  9.  
  10. # create a directory to store the output:
  11. mkdir output
  12.  
  13. # create a temporary file containing the header without
  14. # the content:
  15. head -n 1 $1 > header.csv
  16.  
  17. # create a temporary file containing the content without
  18. # the header:
  19. tail +2 $1 > content.csv
  20.  
  21. # split the content file into multiple files of 5 lines each:
  22. split -l 1 content.csv output/data_
  23.  
  24. # loop through the new split files, adding the header
  25. # and a '.csv' extension:
  26. for f in output/*; do cat header.csv $f > $f.csv; rm $f; done;
  27.  
  28. # remove the temporary files:
  29. rm header.csv
  30. rm content.csv

Report this snippet  

You need to login to post a comment.