Sunday, 8 July 2012

Android NDK & Windows symbolic links

There are projects out there that have symbolic links in them.  Because they're stored in GIT, you get these files and directories in a corrupted form.  They just don't work, so you write a script to convert them to Windows symbolic links.


git ls-files -s | awk '/120000/{print $4}' | while read FILEPATH
    if [ ! -e $FILEPATH ]; then
        echo "Please check that GIT core.symlink for this project is 'false'" 1>&2
        exit 1

    if [ ! -L $FILEPATH ] && [ -f $FILEPATH ]; then
        wc -l $FILEPATH | while read LINES DISCARD
            if [ "$LINES" = "0" ]; then
                FILEDIR=`dirname $FILEPATH`
                FILENAME=`basename $FILEPATH`
                LINKPATH=`cat $FILEPATH`
                pushd . > /dev/null
                cd $FILEDIR
                if [ -e $FILENAME ]; then
                    if [ -e $LINKPATH ]; then
                        echo "Repairing symlink: $FILEPATH"
                        if [ -f $FILEPATH ]; then
                            rm $FILENAME
                            cmd /c "mklink ${FILENAME//\//\\} ${LINKPATH//\//\\}"
                            rm $FILENAME
                            cmd /c "mklink /d ${FILENAME//\//\\} ${LINKPATH//\//\\}"
                        echo "$FILEPATH: symlink path '$LINKPATH' invalid" 1>&2
                    echo "$FILEPATH: problem finding this file"
                popd > /dev/null
                # ERROR: Expected a file with one line for the link path
                echo "$FILEPATH: bad link file: expected 0 lines, got $LINES" > /dev/null # 1>&2
        echo "$FILEPATH: already a symlink" > /dev/null # 1>&2

You run it within MinGW Shell in the top level directory of the project, but MinGW Shell is run as Administrator so that it has permissions to create Windows symbolic links.  And it appears to work.  You can see the linked files or directories looking like shortcuts in explorer, and you can open them.  But then you try and compile something with the Android NDK.

Compile thumb  : sdl-1.2 <= SDL_androidinput.c
/d/Code/android-ndk-r8/toolchains/arm-linux-androideabi-4.4.3/prebuilt/windows/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/local/armeabi/objs-debug/sdl-1.2/src/video/android/ -fpic -ffunction-sections -funwind-tables -fstack-protector -D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__  -Wno-psabi -march=armv5te -mtune=xscale -msoft-float -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../jni/sdl-1.2/include -Ijni/../jni/sdl-1.2 -DANDROID -O3 -DSDL_JAVA_PACKAGE_PATH=net_sourceforge_gemrb -DSDL_CURDIR_PATH=\"net.sourceforge.gemrb\" -DSDL_TRACKBALL_KEYUP_DELAY=1 -DSDL_VIDEO_RENDER_RESIZE_KEEP_ASPECT=1 -DSDL_VIDEO_RENDER_RESIZE=1 -DSDL_ANDROID_KEYCODE_0=LCTRL -DSDL_ANDROID_KEYCODE_1=c -DSDL_ANDROID_KEYCODE_2=NO_REMAP -DSDL_ANDROID_KEYCODE_3=NO_REMAP -DSDL_ANDROID_KEYCODE_4=e -DSDL_ANDROID_SCREENKB_KEYCODE_0=LCTRL -DSDL_ANDROID_SCREENKB_KEYCODE_1=c -DSDL_ANDROID_SCREENKB_KEYCODE_2=NO_REMAP -DSDL_ANDROID_SCREENKB_KEYCODE_3=NO_REMAP -DSDL_ANDROID_SCREENKB_KEYCODE_4=e -Wa,--noexecstack -O0 -g -O2 -DNDEBUG -g -I/d/Code/android-ndk-r8/platforms/android-8/arch-arm/usr/include -c  jni/../jni/sdl-1.2/src/video/android/SDL_androidinput.c -o ./obj/local/armeabi/objs-debug/sdl-1.2/src/video/android/SDL_androidinput.o && ./obj/ ./obj/local/armeabi/objs-debug/sdl-1.2/src/video/android/SDL_androidinput.o.d
jni/../jni/sdl-1.2/src/video/android/SDL_androidinput.c:41:30: error: SDL_androidvideo.h: No such file or directory

Observing the file access in Process Monitor, access flags it as a directory in some instances. Even though file access appears to work outside the NDK case.  MinGW shell almost believes it works when you use the file command on it.
$ file project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
project/jni/sdl-1.2/src/video/android/SDL_androidinput.h: ASCII C program text, with CRLF line terminators
If you try and copy the file that it links to over top of it, that seems to think they're the same.
$ cp -f project/jni/sdl-1.3/src/video/android/SDL_androidinput.h project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
cp: `project/jni/sdl-1.3/src/video/android/SDL_androidinput.h' and `project/jni/sdl-1.2/src/video/android/SDL_androidinput.h' are the same file
Something shows as broken if you try and delete it.
$ rm -f project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
rm: cannot remove `project/jni/sdl-1.2/src/video/android/SDL_androidinput.h': Not owner
A bit of googling suggests trying the rmdir command.
$ rmdir project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
$ file project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
project/jni/sdl-1.2/src/video/android/SDL_androidinput.h: ERROR: cannot open `project/jni/sdl-1.2/src/video/android/SDL_androidinput.h' (No such file or directory)
It looks like the best approach is to copy files rather than link to them. Linking to directories might be stable, but just because it seems to work for now, doesn't mean that problems might be present in certain use cases. As file linking had the above NDK compilation use case, for instance.


  1. Yup, linking stuff in a repo is evil. When we were using sourcesafe, it was rife with all kinds of cross links. We thought we were missing a feature when we moved to Perforce which doesn't support links, but it turned out to be a feature.

  2. I don't remember that and generally liked working with sourcesafe, probably because I had nothing to do with administrating it. What I do remember was a Galician guy having files I needed locked all the time. It's those kind of memories that make me appreciate what Mercurial and GIT have to offer.

    Mercurial unfortunately supports symlinks, and has a potentially problematic workaround. Thankfully, there are no symlinks in the Python repository.