Author Topic: IDEA: Utility to handle files/folders with illegal characters (Read 20387 times)

def · « **on:** May 07, 2015, 01:57 PM »

A Windows utility to rename and/or delete files/folders with illegal characters in their names (e.g. question marks etc.). On the internet, I've mostly found clumsy workarounds such as setting up a Linux VM, trying chkdsk /f, or even trying 8.3 names (if they are not disabled).

The only dedicated tool I have found in a quick web search is DelinvFile (have not tried it myself) which costs $31.95 (USD) and which seemingly offers to at least delete such files/folders (not sure about renaming).

Interesting challenge?

MilesAhead · « **Reply #1 on:** May 07, 2015, 02:53 PM »

Have you tried Unlocker

I'm just wondering how the files get created with illegal names in the first place. It seems strange.

def · « **Reply #2 on:** May 07, 2015, 03:08 PM »

Have you tried Unlocker
-MilesAhead (May 07, 2015, 02:53 PM)

Just tried it. Doesn't work ("no locking handle found").

I'm just wondering how the files get created with illegal names in the first place. It seems strange.
-MilesAhead (May 07, 2015, 02:53 PM)

In my case it's a file and a folder on a USB drive, from a time when I experimented with Linux. At this time I only use Windows, thus I recognize how tedious such a file can be...

4wd · « **Reply #3 on:** May 08, 2015, 01:27 AM »

Possibly Powershell?

Remove Invalid Characters from File Names

Though the problem remains of how to get the file from the filesystem before fixing it.

MilesAhead · « **Reply #4 on:** May 08, 2015, 05:43 AM »

Possibly Powershell?

Remove Invalid Characters from File Names

Though the problem remains of how to get the file from the filesystem before fixing it.
-4wd (May 08, 2015, 01:27 AM)

There are Linux win32 utilities that have bash shell with ls and rm commands. I haven't used it in some time. I'm not sure which is the latest or most friendly to Windows newer than XP. Once the bash shell is running just use rm command to remove files and/or folders

http://www.cyberciti...folder-and-contents/

Ath · « **Reply #5 on:** May 08, 2015, 07:05 AM »

There are Linux win32 utilities that have bash shell with ls and rm commands.
-MilesAhead (May 08, 2015, 05:43 AM)

Cygwin ?

def · « **Reply #6 on:** May 08, 2015, 07:13 AM »

There are Linux win32 utilities that have bash shell with ls and rm commands. I haven't used it in some time. I'm not sure which is the latest or most friendly to Windows newer than XP. Once the bash shell is running just use rm command to remove files and/or folders

http://www.cyberciti...folder-and-contents/
-MilesAhead (May 08, 2015, 05:43 AM)

I downloaded Cygwin but it couldn't delete the file in question:

[Select]

How can I fix_repair a corrupted PDF file? - Super User.html
How can I fix_repair a corrupted PDF file? - Super User-Dateien

$ rm "How can I fix_repair a corrupted PDF file? - Super User.html"
rm: cannot remove ‘How can I fix_repair a corrupted PDF file? - Super User.html’: No such file or directory

MilesAhead · « **Reply #7 on:** May 08, 2015, 07:19 AM »

There are Linux win32 utilities that have bash shell with ls and rm commands. I haven't used it in some time. I'm not sure which is the latest or most friendly to Windows newer than XP. Once the bash shell is running just use rm command to remove files and/or folders

http://www.cyberciti...folder-and-contents/
-MilesAhead (May 08, 2015, 05:43 AM)
I downloaded Cygwin but it couldn't delete the file in question:

[Select]
How can I fix_repair a corrupted PDF file? - Super User.html
How can I fix_repair a corrupted PDF file? - Super User-Dateien

$ rm "How can I fix_repair a corrupted PDF file? - Super User.html"
rm: cannot remove ‘How can I fix_repair a corrupted PDF file? - Super User.html’: No such file or directory
-def (May 08, 2015, 07:13 AM)

I would try ls using a pattern so that only the file(s) you wish to delete match. Then use the same pattern with rm. For example
ls -l How*

def · « **Reply #8 on:** May 08, 2015, 07:44 AM »

I would try ls using a pattern so that only the file(s) you wish to delete match. Then use the same pattern with rm. For example
ls -l How*
-MilesAhead (May 08, 2015, 07:19 AM)

[Select]

$ ls -l
ls: cannot access How can I fix_repair a corrupted PDF file? - Super User-Dateien: No such file or directory
ls: cannot access How can I fix_repair a corrupted PDF file? - Super User.html: No such file or directory
total 19940148
----rwx---+ 1 def None 2304492 Dec 24 2007 9783827241993bsp.pdf
(several other files...)
-?????????? ? ? ? ? ? How can I fix_repair a corrupted PDF file? - Super User.html
d?????????? ? ? ? ? ? How can I fix_repair a corrupted PDF file? - Super User-Dateien
(several other files...)

$ ls -l How*
ls: cannot access How can I fix_repair a corrupted PDF file? - Super User.html: No such file or directory
ls: cannot access How can I fix_repair a corrupted PDF file? - Super User-Dateien: No such file or directory

$ rm -r How*
rm: cannot remove ‘How can I fix_repair a corrupted PDF file? - Super User.html’: No such file or directory
rm: cannot remove ‘How can I fix_repair a corrupted PDF file? - Super User-Dateien’: No such file or directory

Somehow even Cygwin chokes on this file name...

4wd · « **Reply #9 on:** May 08, 2015, 08:37 AM »

Try running chkdsk.exe /f on the volume, it can sometimes fix these problems by removing the (illegally named) file.

If the file gets deleted, it might be reincarnated (with a new name) by looking at any filecheck.* that was created.

MilesAhead · « **Reply #10 on:** May 08, 2015, 09:37 AM »

See if this answer is any help
http://stackoverflow...indows-filename-java

Edit:

The filename looks like it is created from a URL. Somewhere on AHK forum I got this function

[Select]

PathCreateFromURL( URL )
{
  if (! URL)
  return ""
VarSetCapacity( fPath, Sz := 2084, 0 )
DllCall( "shlwapi\PathCreateFromUrl" ( A_IsUnicode ? "W" : "A" )
   , Str,URL, Str,fPath, UIntP,Sz, UInt,0 )
return fPath
}

If you pass in a string with the filename it should spit out an acceptable Windows filename or an empty string on error.

Edit: for example

[Select]

URL := "file://c|MyFolder/yaddyadda.txt"
fname := PathCreateFromURL(URL)
MsgBox % fname

MilesAhead · « **Reply #11 on:** May 08, 2015, 10:05 AM »

You could go at it from the other direction. Copy all files with Windows legal names you want to keep from the thumb drive to some other storage. Format the thumb drive erasing all data.

def · « **Reply #12 on:** May 11, 2015, 11:52 AM »

Try running chkdsk.exe /f on the volume, it can sometimes fix these problems by removing the (illegally named) file.

If the file gets deleted, it might be reincarnated (with a new name) by looking at any filecheck.* that was created.
-4wd (May 08, 2015, 08:37 AM)

I tried this, alas the drive (partition) is truecrypted, and chkdsk refuses working on it as another process (probably TrueCrypt) is accessing it, thus it insists on releasing the partition and subsequently chkdsk-ing it (who knows what that will do that to the partition?). Which is all the more surprising considering that according to the TrueCrypt documentation, using chkdsk should be possible on TrueCrypt volumes...

See if this answer is any help
http://stackoverflow...indows-filename-java

Edit:

The filename looks like it is created from a URL. Somewhere on AHK forum I got this function

[Select]
PathCreateFromURL( URL )
{
if (! URL)
return ""
VarSetCapacity( fPath, Sz := 2084, 0 )
DllCall( "shlwapi\PathCreateFromUrl" ( A_IsUnicode ? "W" : "A" )
, Str,URL, Str,fPath, UIntP,Sz, UInt,0 )
return fPath
}

If you pass in a string with the filename it should spit out an acceptable Windows filename or an empty string on error.

Edit: for example

[Select]
URL := "file://c|MyFolder/yaddyadda.txt"
fname := PathCreateFromURL(URL)
MsgBox % fname
-MilesAhead (May 08, 2015, 09:37 AM)

Well, these might help in the future. Interestingly, NTFS-3G (this Linux thingy that writes files to NTFS drives) also seems to have an option windows_names that should be default anyway, IMHO. Which it currently isn't, at least not in the Linux distro that I tried.

You could go at it from the other direction. Copy all files with Windows legal names you want to keep from the thumb drive to some other storage. Format the thumb drive erasing all data.
-MilesAhead (May 08, 2015, 10:05 AM)

Yea, that's clearly possible.
However, I think I have more files and drives with similar problems. Maybe the easiest would be to reinstall Linux...!

Anyway, thanks for your efforts.

MilesAhead · « **Reply #13 on:** May 11, 2015, 01:57 PM »

However, I think I have more files and drives with similar problems. Maybe the easiest would be to reinstall Linux...
-def (May 11, 2015, 11:52 AM)

Or use a Linux Live that has NTFS support. You can mount the drive with the files that need to be renamed, as part of the live directory tree. I would also search for a "one liner" in case someone has run into this before and posted the solution. Some file pattern that would detect these types of filenames as input etc..

With all the machine dual booting Windows and Linux over the years there must be something canned out there somewhere.

Edit: Looks like someone has attempted to use find Linux command to fix it up. I don't know how successful it would be:

http://stackoverflow...ows-style-file-names

AbteriX · « **Reply #14 on:** May 12, 2015, 12:24 AM »

Or use a Linux Live that has NTFS support.
-MilesAhead (May 11, 2015, 01:57 PM)

Perhaps try mount -o check=r

check=value
Three different levels of pickyness can be chosen:

r[elaxed]
Upper and lower case are accepted and equivalent, long name
parts are truncated (e.g. verylongname.foobar becomes very-
long.foo), leading and embedded spaces are accepted in each
name part (name and extension).

n[ormal]
Like "relaxed", but many special characters (*, ?, <, spaces,
etc.) are rejected. This is the default.

s[trict]
Like "normal", but names may not contain long parts and special
characters that are sometimes used on Linux, but are not
accepted by MS-DOS are rejected. (+, =, spaces, etc.)

The above is a quote out of the `mount` man page; I'm not familiar with CIFS but is it possible
the default option setting for this mount option is causing the problem?
Maybe mounting with check=relaxed option would get around it.

Sasha

EDIT - PS, this option is for FAT, by the way; I forgot to mention

From http://www.linuxques...uestion-mark-735617/

x16wda · « **Reply #15 on:** June 19, 2015, 05:09 AM »

Would be interested in knowing if SetACL Studio will work for this. It has a 30 day trial that ought to give you enough time to try it.

def · « **Reply #16 on:** June 19, 2015, 05:27 PM »

Would be interested in knowing if SetACL Studio will work for this. It has a 30 day trial that ought to give you enough time to try it.
-x16wda (June 19, 2015, 05:09 AM)

Probably not, as it's not a permission/security problem, although I have not tested this particular tool. Generally, when tools try to delete my test file they complain that there (supposedly) is no such file. Thanks for your suggestion, though.

However, I do have tested a bunch of deletion/renaming tools suggested on the internet. None worked. $:-\$

Most problems with "illegal" characters are related to exceeding the Windows file/path name length limits or funny symbols from Unicode file names (and missing fonts). As stated above, the chkdsk trick doesn't work for me. I have even tried to delete the whole containing folder, but no luck.

The only option seems to be the special tool DelinvFile, but I think $31.95 (USD) is a bit much...

Anyway, in the meantime I have reinstalled Linux (not just because of this problem, btw), so I will remove the files in question bit by bit.

Concluding I would like to say thanks to you all for your kind help. Feel free to suggest more solutions, maybe they can help other users with the same problem, although I personally will use Linux to remove all problematic files left. Thanks again.

4wd · « **Reply #17 on:** June 19, 2015, 08:08 PM »

However, I do have tested a bunch of deletion/renaming tools suggested on the internet. None worked. $:-\$
-def (June 19, 2015, 05:27 PM)

Reminds of when I had the same problem about 20 years ago, I had to write a program to rename/delete files that Windows wouldn't touch.

However that was when Windows 3.1 and FAT32 were still the norm.

Shades · « **Reply #18 on:** June 21, 2015, 12:42 PM »

If you still have a 32-bit Windows OS, then I would suggest to download and use PortableLinux. Because then you can run Windows and Linux (Ubuntu) both at the same time. Whatever file gives you problems in on one OS, might not be problematic in the other as Windows and Linux have a different set of illegal characters that can be used with file names.

Too bad PortableLinux only works with 32-bit versions of Windows...the software (co-linux) that allows Linux to run besides Windows is 32-bit only and doesn't look like it is going to be fixed any time soon. Having said that, I personally have tried and tested PortableLinux successfully with Windows 2000, Windows XP, Windows7 and Windows 8.

Curt · « **Reply #19 on:** June 22, 2015, 09:41 AM »

I don't understand why "def" not yet have tested SetACL Studio https://helgeklein.com/setacl-studio/ - his "IDEA" is clearly not that easy to bring alive.

--------------

how to delete this empty file?
Try this - download SetACL Studio (which has a 30 day free trial) and navigate to the folder and see exactly what's there. This is useful software -
-x16wda (May 23, 2015, 09:01 AM)
- "useful software"; you can say that again! It didn't ask any questions, but deleted the empty file just like it would have deleted any other file. BRAVO! Thank you, x16wda,
(but of course, normal price is €90 - so it should be able to do a lot more than those freeware apps)
-Curt (May 23, 2015, 11:40 AM)

Try SetACL Studio https://helgeklein.com/setacl-studio/

def · « **Reply #20 on:** June 22, 2015, 03:13 PM »

I don't understand why "def" not yet have tested SetACL Studio https://helgeklein.com/setacl-studio/ - his "IDEA" is clearly not that easy to bring alive.

--------------

how to delete this empty file?
Try this - download SetACL Studio (which has a 30 day free trial) and navigate to the folder and see exactly what's there. This is useful software -
-x16wda (May 23, 2015, 09:01 AM)
- "useful software"; you can say that again! It didn't ask any questions, but deleted the empty file just like it would have deleted any other file. BRAVO! Thank you, x16wda,
(but of course, normal price is €90 - so it should be able to do a lot more than those freeware apps)
-Curt (May 23, 2015, 11:40 AM)

Try SetACL Studio https://helgeklein.com/setacl-studio/

-Curt (June 22, 2015, 09:41 AM)

Yea, now I'm feeling guilty.

There are several reasons why I didn't test it (which are my own faults - not saying that SetACL Studio was bad in any way).

Firstly, I underestimated the work that would be required for programming this by a huge margin (hence I suggested it as a Coding Snack which was clearly a mistake). Secondly, I have tried to find freeware instead of payware for a task that I'd probably only need once in my life. Thirdly, after trying out several freeware tools that failed on this task, I was sure that any tool that didn't explicitly advertise "can handle file names with illegal characters" is not worth a try. Fourthly, SetACL Studio gave the impression (to me, at least) that it has only something to do with security permissions and not file names. Fifthly, SetACL Studio is payware - now, I'm not generally opposed to paying for software, but if I'd pay for a lot of features that I don't need (or that I don't even understand, for that matter), then I'm being reluctant. And sixthly, the problem wasn't that urgent to begin with, i.e. if no easy free/cheap solution comes about, reinstalling Linux is good enough for me.

Sorry for my ambiguous stance on this and, again, thanks for all your input. It's nice to know that according to Curt, SetACL Studio works for such files.