Old Emmanuel Oga's Weblog (new one is at www.emmanueloga.com)

Linux: moving around files with white spaces on the path name

Posted in ruby by emmanueloga on agosto 11, 2009

I have a couple of big hard drives on my desktop PC. Still, from time to time I need to do some maintenance to get rid of all the junk I throw at them. Lately, I’ve been moving files around a lot. And I was doing it all wrong! I set the laziness aside and thought about ways of improving my «work flow».

Whitespace, go away!

The first issue I had is that bash gets confused when the filenames include white space.


[emmanuel@going_merry ~/files]
$ ll
total 4
-rw-r--r-- 1 emmanuel users 2818 2009-08-11 11:38 This is my afn.py

[emmanuel@going_merry ~/files]
$ mv This is my afn.py afn.py
mv: target `afn.py’ is not a directory

[emmanuel@going_merry ~/files]
$ mv This\ is\ my\ afn.py afn.py

Notice the slashes I had to add on the second mv call. This might seem pretty obvious. Bash usually puts those slashes automatically when using tab completion. But the thing is, what I really need in my case is to move a whole set of files returned by other utility, like find, to another place:


[emmanuel@going_merry ~/files]
$ ll
total 8
drwxr-xr-x 2 emmanuel users 4096 2009-08-11 11:41 dir1
drwxr-xr-x 2 emmanuel users 4096 2009-08-11 11:41 dir2

[emmanuel@going_merry ~/files]
$ find dir1/
dir1/
dir1/My afn 1.py
dir1/My afn 2.py

[emmanuel@going_merry ~/files]
$ find dir1/ | xargs -I {} mv {} dir2
mv: cannot stat `dir1/My afn 1.py’: No such file or directory
mv: cannot stat `dir1/My afn 2.py’: No such file or directory

The problem here is in the find output. The list of  path names returned by find may have unescaped spaces on path names. I needed to escape those spaces before using those path names with other utilities. I looked around for a unix utility to help me in this regard, but not easily finding anything good around, I just wrote a little ruby script for dealing with this: «escapepath».


[emmanuel@going_merry ~/files]
$ which escapepath
/usr/bin/escapepath


[emmanuel@going_merry ~/files]
$ cat `which escapepath`

#!/usr/bin/ruby
require 'shellwords'

while not STDIN.eof?
  puts(STDIN.gets.chomp.shellescape)
end

gist

Now I can do things like:


[emmanuel@going_merry ~/files]
$ find dir1/ -type f | escapepath | xargs -I {} mv {} dir2/


If you are typing too much, you are not aliasing enough

An anti-pattern I was using was… typing a lot. I kept running again and again things like:

[emmanuel@going_merry ~/files]
$ ack -a -f | grep thing_i_want_to_find -i | escapepath | xargs -I {} mv {} /some/directory

…to find several files at once using a case insensitive regular expression and move all of them to a certain directory. The solution: bash aliases:


[emmanuel@going_merry ~/files]
$ cat ~/.bashrc

# ...
alias xmv='escapepath | xargs -I {} mv {}'
alias ifind='ack -a -f | grep -i'
# ...

Note: ack is a cool alternative to find+grep which also has some nice file finding features. Check it out.

Know I can do things like:


[emmanuel@going_merry ~/files]
$ ifind thing_i_want_to_rm | xargs rm
[emmanuel@going_merry ~/files]
$ ifind thing_i_want_to_move | xmv /some/directory

Ah! much more easy on my poor fingers…. 🙂

9 respuestas

Subscribe to comments with RSS.

  1. Andy Lester said, on agosto 11, 2009 at 5:04 pm

    You shouldn’t need to use grep -i in ifind? ack supports -i with -f. Also, ack works just fine as a filter, so you can do «ack -f | ack -i»

    • emmanueloga said, on agosto 11, 2009 at 5:58 pm

      Well I’m using ack to generate a list of files here, since I like the convenience of ack skipping .svn directories and stuff. Ack does have a regexp filter for filenames (-g) but I could not find any way of making it behave in case insensitive way for file names. You are right about it being able to be used as filter, thats a possibility too: I think I just enjoy mixing and matching unix tools 🙂

      • Andy Lester said, on agosto 11, 2009 at 8:39 pm

        Wow, you’re right, -i doesn’t work with -g. I thought I had that working.

  2. Ralphie said, on septiembre 20, 2009 at 1:09 am

    Emman {gracias}

    For pointing out a practical use for xargs command! I used it to do something like this:

    ls -c |grep [Bb]odyb |xargs -I {} mv {} /xas/weightlifting

    and it worked in nicely.

    thanks again,
    Ralphie

  3. Suda said, on septiembre 20, 2009 at 3:22 am

    Why not use -exec, I think that handles the spaces fine – as in

    find find dir1/ -type f -exec mv {} dir2/ \;

    For example, if I have x and y as two directories and x has a file named «this is a new file» – the following works.

    >find y -type f -exec cat {} \;
    This is test.

    >find y -type f
    y/this is a new file

    >find y -type f -exec mv {} x \;

    >find y -type f

    >find x -type f
    x/this is a new file

    >find x -type f -exec cat {} \;
    This is test.

  4. Ralphie said, on septiembre 23, 2009 at 12:13 am

    Yes i guess exec would do the job but in my case I had done some homework for example:
    1. I new where the directories for my 14gigs of data were at.
    2. 14 gigs of data had a keyword bodybuilding that I used for pattern matching.
    3. I wanted my command(s) to me short sweet and simple.

    That was the reason I went in this direction, but had I needed to move more data that was scattered all over the place then your method should solve that problem nicely.

    Keep in mind I’m very green with linux in general, but I like it much more then the other alternatives out there.

    I’m always learning something new from it – not just feeling inept 😛

    How long have you been using linux/unix os?

    And what are your thoughts on it?

    • emmanueloga said, on septiembre 24, 2009 at 9:40 am

      I’ve been using linux for a while now, don’t remember exactly how long. I love the command line, but some times it can become tricky. Sometimes is hard to keep all those command line switches on the top of my mind, but lucky for me, there are a ton of documentation both in the form of man pages, and also on the web (wikis, forums, blogs, etc…). It is definitely easier to use linux today than it was, say, 10 years ago 🙂

  5. Pádraig Brady said, on noviembre 26, 2009 at 9:50 pm

    To mv files with dodgy characters the standard idiom is:
    find dir1/ -type f -print0 | xargs -r0 mv –target=dir2/

    Also have a look at findrepo as an alternative to ack.
    It’s much simpler and also much faster:
    http://www.pixelbeat.org/scripts/findrepo

    • emmanueloga said, on noviembre 27, 2009 at 2:32 am

      Hello Pádraig, thanks for the comment. Funny, I was browsing your excellent web site (loads of cool stuff) just today and then I get one comment from you, guess I leaved a trail somewhere :-).

      Greets!


Deja un comentario