Posted By

tm on 01/18/10


Tagged

Bash recursive copy directories


Versions (?)

Recursively copy a directory (using cp, tar or rsync)


 / Published in: Bash
 

  1. # How does one make a recursive, identical copy of /source/ into /target/?
  2.  
  3. # I suppose you want to do that for archiving or duplicating something and want to preserve
  4. # "everything". That includes permissions, ownership, filetypes, timestamps etc.
  5. # Additionally I assume, that you create /target in advance. This is not required for any
  6. # of the methods, but I would have to make a case-distinction for every method depending on
  7. # whether the /target-directory exists or not (but see Note 3 at the very end). Check note
  8. # 2 for a remark about sparse files.
  9.  
  10. # There are multiple ways to do it. On a GNU system the simplest way is:
  11.  
  12. ############
  13. # METHOD 1: Using GNU cp
  14. cp -a /source/. /target/
  15.  
  16. # Two remarks:
  17. # (1) If you used "/source" (or "/source/") as first argument, you would end up with /target
  18. # /source/... This is usually no problem though: simply cd into target, then mv everything
  19. # from ./source to . (either using mv twice, once with source/* and once with source/.* or
  20. # using it only once with "shopt -s dotglob" in front). Copying one level of directories to
  21. # many (like "source" in the example above) is much less annoying than copying one level to
  22. # few (and e.g. ending up with all stuff under "source" besides /target instead of inside
  23. # it).
  24. # So if in doubt, use a syntax that copies "one level too many".
  25. # However, it could be a problem if either "*" or ".*" expand to "very much entries" and
  26. # you get an error along the lines of "argument list too long". In that case you'd need to
  27. # work around the issus with a loop:
  28.  
  29. # NOTE: not a method to copy but how to clean up after copying "one level too much"
  30. cd /target; shopt -s dotglob; for f in source/*; do mv "$f" .; done; rmdir source.
  31. # Best thing of course is, to do it right in the first place as I showed above.
  32.  
  33. # (2) The -a-option to cp is a GNUism. But GNU tools are very widespread, so it's the first
  34. # method I show and the preferred one if it's available because of its simplicity. Note
  35. # that the often used "cp -R" does not preserve attributes: "cp -R is a broken -a" - even
  36. # with -p and especially if people suggest "-r" (lower case).
  37. # YOU ALMOST NEVER WANT -R or -r!
  38.  
  39. # The second method is (pretty) universally available (and my personal favourite):
  40.  
  41. ############
  42. # METHOD 2: tar (available everywhere)
  43. cd /source; tar cf - . | (cd /target && tar xf -)
  44.  
  45. # I can't think of a system where that would not work (though it's not POSIX, since POSIX
  46. # does not know tar...). Some notes as well:
  47. # (1) the cd commands could be replaced by -C option to tar, which again is a GNUism.
  48. # (2) the "&&" in the extraction-subshell makes sure you don't clobber your filesystem if
  49. # /target is not there.
  50.  
  51. # The third (and last) method I show uses rsync. The advantage here is that it can be
  52. # "restarted from where it left" if interrupted (while cp or tar wouuld have to start from
  53. # the beginning). So it may be the preferred method for large amounts of data. The
  54. # disadvantage is, that rsync is far less readily available than tar or even GNU cp.
  55.  
  56. ############
  57. # METHOD 3: rsync (can be "re-started", mind the -H option!)
  58. rsync -aH /source/ /target
  59.  
  60. # (many pepole like to add a "-v" (--verbose) to the options, to make rsync show, what it
  61. # does). The rsync "a" option is similar to the cp "a" option. They both stand for
  62. # "archive" which is, what we want to do: preseve everything. For rsync however there is
  63. # one notable exception: hard links. That's why we throw in the additional "H". Also note
  64. # the trailing "/" on "/source/": It tells rsync to copy the contents of "/source", not
  65. # "/source" itself. Omitting the trailing slash we'd end up with "/target/source/..." - not
  66. # a desaster, as I noted above.
  67.  
  68. Some final notes:
  69.  
  70. # (1) there sure are other ways. I can think of cpio and pax (which nobody knows but is
  71. # POSIX...)
  72. # (2) If sparse files are an issue check the -S parameter for rsync/(GNU)tar or --sparse
  73. # for (GNU) cp
  74. # (3) to copy "everything but some subdirectories" you need the respective exclude-options
  75. # for tar/rsync. cp cannot do that, although by using bash extended globbing, trivial
  76. # use-cases can be covered [Eample.: shopt -s exglob; echo /(!tmp)]
  77. # (4) ok, I didn't want to...but: If /target does not exists (or is a file instead of a
  78. # directory) the "cp" method does not change. It errors if /target exists but is not a
  79. # directory and creates the directory if no /target exists. The same goes for rsync.
  80. # The tar method requires a container in advance to "untar" into.

Report this snippet  

You need to login to post a comment.