hmmmmmmm guess ahk has some restrictions about using a hotkey inside a function. Just quickly looking through some user defined functions I see people using a line label like you do for gosub.. maybe that's why. It would be nice to put this hack in a separate script in a very usable way.
So far I cleaned it up just a bit to use a variable for the Open Folder Dialog so that the same variable can be used by the hotkey to see if that window is active. After return from
FileSelectFile() the SelectedFolder variable should either contain the folder path or blank. Note that ErrorLevel should be non-0 on success because the hotkey code sends {Esc} to close the dialog.
;set File Open Dialog to Prompt users how to select a folder
FolderPrompt = Alt Right Mouse Click to select Folder
FileSelectFile,DummyVar,2,%A_MyDocuments%\*.*,%FolderPrompt%,All Files(*.*)
If Not SelectedFolder
MsgBox, 0, , No Folder Selected
Else
MsgBox, 0, , Folder Selected is %SelectedFolder%
ExitApp
!RButton::
IfWinActive,%FolderPrompt%
{
Clipboard =
Send ^c
ClipWait,2
SelectedFolder = %Clipboard%
Send {Esc}
}
Return