Articles

  • PowerShell and external commands done right

    Windows PowerShell is a massive step up from the VBScript horror used to manage Windows systems (I have no idea how people wrote websites with it without going mental). One of the things that annoyed me to no end though was how there seemed to be black magic involved when trying to make PowerShell execute external commands, i.e. not PowerShell cmdlets.

    It is actually quite straight-forward once you wrap your head around it - it's just that we try to do things the way we did in VBScript or in OO languages, and PowerShell doesn't like that.

    Background

    I'm currently writing a script to automate creating and deleting volume shadow copies, creating a ShadowProtect image in between.

    This includes normal looking commands like,

    H:\backup\scripts\vshadow.exe -p -script="H:\backup\scripts\vss.cmd" E: M: P:

    As well as funny looking ShadowProtect commands,

    H:\backup\scripts\sbrun.exe  -mdn ( sbvol -f  \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy43 \\?\E: : sbcrypt -50 : sbfile -wd H:\backup\backups )

    The wrong way to do it

    If you ask Google, you'll probably get responses telling you to do this:

    $exe = "H:\backup\scripts\vshadow.exe"
    $arguments = '-p -script="H:\backup\scripts\vss.cmd" E: M: P:'
    $proc = [Diagnostics.Process]::Start($exe, $arguments)
    $proc.WaitForExit()

    This works, but it isn't the right way to going about mainly because it isn't the PowerShell way. You're actually calling the class in the .NET Framework (which PowerShell is based on) that application developers use to launch external applications. Not only is this more code, but also makes it a bit more complicated if you want to process the standard output/error, and also results in a new command window popping up even if it is just another command line application. This method has gained a bit more legitimacy in some use cases though, and in PowerShell v2, is now accessible using the Start-Process cmdlet.

    The next most popular way, but also somewhat error prone and hence the most frustrating, is this:

    $exe = "H:\backup\scripts\vshadow.exe"
    $arguments = "-p -script=`"H:\backup\scripts\vss.cmd`" E: M: P:"
    &$exe $arguments

    The ampersand (the 'and' sign) here tells PowerShell to execute that command, instead of treating it as a cmdlet or a string. The backticks (the funny looking single-quotes) are there to escape the following character, similar to the \" in C-based languages, or double-double-quotes ("") in VB. Otherwise the " character will end the string and the parser will cry when it can't understand what you're trying to say after that. (You can alternatively use single-quotes instead in this case, as I have in the previous example.)

    The reason why this doesn't work is because PowerShell is a shell first and foremost. What PowerShell is actually doing is executing the specified executable, but then passes all your parameters within quotes (or if it makes more sense, as a single parameter), as you can see in this alternate, more concise version:

    & "H:\backup\scripts\vshadow.exe" "-p -script=`"H:\backup\scripts\vss.cmd`" E: M: P:"

    You'll probably spend hours pulling your hair out wondering why things aren't working even when the arguments seem to be passed ok (to make things worse, some command line apps work fine with it). You'll also likely get cryptic error messages like,

    Invalid parameter: "-p -script=`"H:\backup\scripts\vss.cmd`" E: M: P:"

    And you'll be like wtf, why the hell is that invalid!?!?!

    Enter echoargs.exe

    Echoargs is a simple tool that spits out the arguments it receives. It is part of the PowerShell Community Extensions download, but if you can't be bothered downloading and installing that, here it is on its own.

    If you replaced the executable in the above command with echoargs.exe, you'll be able to see what's happening.

    & "H:\backup\scripts\echoargs.exe" "-p -script=`"H:\backup\scripts\vss.cmd`" E: M: P:"

    Execute that and you'll get the following output,

    Arg 0 is <-p -script=H:\backup\scripts\vss.cmd E: M: P:>

    See how all the parameters are being passed in one string? That is not what you want as most command line apps will not know what you intended.

    How to do it the PowerShell way

    Remember that PowerShell is a shell first and foremost. To run the above external command, just do the following:

    $exe = "H:\backup\scripts\vshadow.exe"
    &$exe -p -script=H:\backup\scripts\vss.cmd E: M: P:

    Notice that I'm not putting all the arguments into a single string, I'm just writing them as they are. If you run this with echoargs.exe, you'll get the following:

    Arg 0 is <-p>
    Arg 1 is <-script=H:\backup\scripts\vss.cmd>
    Arg 2 is <E:>
    Arg 3 is <M:>
    Arg 4 is <P:>

    That is what the command line application expects. Notice that each parameter is considered a different argument, as opposed to a single string for all parameters.

    Using PowerShell v3?

    If you're using PowerShell v3 (which shipped with Windows 8 and Windows Server 2012 and is also available for Windows 7/2008 as a separate download), there is a new language feature that simplifies a lot of this. Instead of having to stuff around with escaping and quoting parameters to dodge the PowerShell parser, you can now use the --% operator which tells PowerShell to stop parsing from that point onward until the end of the line. Everything from that operator onwards is parsed by the Windows Command Processor (cmd.exe) parser used by the program (e.g. MS C/C++ runtime) and all those rules apply instead. This means that you can't reference any PowerShell variables after that operator (any references will be past literally, i.e. $dir will be passed to the external command as $dir), but it also means you can reference environment variables using the cmd.exe syntax, e.g. %USERPROFILE% (I have no idea what expands them; is cmd.exe still involved somehow?). 

    For example, the following command (which will work only in cmd.exe and not PowerShell in its current form),

    H:\backup\scripts\sbrun.exe -mdn ( sbvol -f  \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy43 \\?\E: : sbcrypt -50 : sbfile -wd H:\backup\backups )
    

    can be written as,

    &"H:\backup\scripts\sbrun.exe" --% -mdn ( sbvol -f  \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy43 \\?\E: : sbcrypt -50 : sbfile -wd H:\backup\backups )

    for execution within PowerShell - there is no need to bother with escaping the brackets.

    Remember that this only exists in PowerShell v3 so if your scripts target older versions of PowerShell, you cannot use this. Also, if you need to reference PowerShell variables, you can't use this trick either. Read on.

    What about parameters with spaces in them?

    Now, you might be asking, how do I send parameters that contain spaces? Normally we would quote the part that has spaces, e.g.

    &$exe -p -script="H:\backup\scripts temp\vss.cmd" E: M: P:

    But not in Powershell. That will simply confuse it. Instead, just place the entire parameter in quotes, e.g.

    &$exe -p "-script=H:\backup\scripts temp\vss.cmd" E: M: P:

    Or parameters where the quote characters need to passed on?

    If it is necessary for the quotes to be passed on to the external command (it very rarely is), you will need to double-escape the quotes inside the string, once for PowerShell using the backtick character (`), and again for the Windows Command Processor parser using the backslash character (\). For example,

    &$exe -p "-script=\`"H:\backup\scripts temp\vss.cmd\`"" E: M: P:

    When you execute an external command, Powershell grabs the command and the arguments (after the strings have been processed by Powershell and the Powershell escape characters removed), then passes it as a single string to the Windows Command Processor (or possibly straight to the Windows Shell/ Win32 API) program for execution. The Windows Command Processor program, depending on the parser used (e.g. MS C/C++ runtime), has a separate set of rules for escaping things, therefore it is necessary to escape again to prevent it from interpreting the quotes. Most (but annoyingly, not all) use the MS C/C++ runtime parser, and from what I can gather, it splits up the string into arguments by splitting at each space, unless the space is inside quotes. Because the inner quotes were not escaped using the Windows Command Processor parser escape character (the backslash), the command processor parser interpreted them as if the quoted parts contained "-script=" and "", therefore the space between 'scripts' and 'temp' isn't actually within any quotes and hence split.

    You can see this happening by playing with echoargs.exe (which uses the MS C/C++ runtime parser) inside the Command Prompt (not the PowerShell prompt).

    The order of the escape characters is important - it must be the backslash character first, then the backtick character. Otherwise, because PowerShell processes the command first, the backtick will escape the backslash instead of the quote as intended.

    If the program does not use the MS C/C++ runtime parser to parse command line arguments, then how it is parsed is entirely dependent on how the program implemented it. The following is a quick PowerShell script that shows you what the raw command line is being passed to the program as well as how one of the alternate methods of parsing it works (CommandLineToArgvW - I believe this is not what the MS C/C++ runtime uses).

    $Kernel32Definition = @'
    [DllImport("kernel32")]
    public static extern IntPtr GetCommandLineW();
    [DllImport("kernel32")]
    public static extern IntPtr LocalFree(IntPtr hMem);
    '@
    
    $Kernel32 = Add-Type -MemberDefinition $Kernel32Definition -Name 'Kernel32' -Namespace 'Win32' -PassThru
    
    $Shell32Definition = @'
    [DllImport("shell32.dll", SetLastError = true)]
    public static extern IntPtr CommandLineToArgvW(
        [MarshalAs(UnmanagedType.LPWStr)] string lpCmdLine,
        out int pNumArgs);
    '@
    
    $Shell32 = Add-Type -MemberDefinition $Shell32Definition -Name 'Shell32' -Namespace 'Win32' -PassThru
    
    $RawCommandLine = [System.Runtime.InteropServices.Marshal]::PtrToStringUni($Kernel32::GetCommandLineW())
    Write-Host "The raw command line is (excluding the angle brackets)\:`n<$RawCommandLine>`n"
    
    $ParsedArgCount = 0
    $ParsedArgsPtr = $Shell32::CommandLineToArgvW($RawCommandLine, [ref] $ParsedArgCount)
    
    try
    {
        $ParsedArgs = @( );
    
        0..$ParsedArgCount | ForEach-Object {
            $ParsedArgs += [System.Runtime.InteropServices.Marshal]::PtrToStringUni(
                [System.Runtime.InteropServices.Marshal]::ReadIntPtr($ParsedArgsPtr, $_ * [IntPtr]::Size)
            )
        }
    }
    finally
    {
        $Kernel32::LocalFree($ParsedArgsPtr) | Out-Null
    }
    
    Write-Host "The command line as parsed by CommandLineToArgvW (not MSVCRT) is:"
    # -lt to skip the last item, which is a NULL ptr
    for ($i = 0; $i -lt $ParsedArgCount; $i += 1) {
        Write-Host "argv[$i] <$($ParsedArgs[$i])>"
    }

    Save the above script to a file, e.g. GetCommandLine.ps1, and execute it like so -

    PS C:\Users\User\Desktop> powershell .\GetCommandLine.ps1 a b"c d"e f
    The raw command line is (excluding the angle brackets)\:
    <"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe"  .\GetCommandLine.ps1 a "bc de" f>
    
    The command line as parsed by CommandLineToArgvW (not MSVCRT) is:
    argv[0] <C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe>
    argv[1] <.\GetCommandLine.ps1>
    argv[2] <a>
    argv[3] <bc de>
    argv[4] <f>

    It is important to execute it in a separate instance of PowerShell (hence the 'powershell' in the command line). Otherwise, it will simply show you the arguments of the current instance of PowerShell when it was launched. Use the output of the raw command line to see what PowerShell is passing to the program you're running, i.e. after any parsing and manipulating that PowerShell does. In this example, notice how the position of the quote characters have changed between the original command line and the raw command line as printed by the script - this is PowerShell manipulating the strings (I'm not quite sure what it is doing though; looks like string concatenation for adjacent strings, but not sure why the quote character has moved).

    For more on the bizarre and inconsistent world of Windows command argument parsing, see http://www.daviddeley.com/autohotkey/parameters/parameters.htm#WINARGV and https://gist.github.com/dolmen/6030690/raw/5dde469149420f12acd6f5a6120c3a90474e4088/ref.md. Hat tip to JFL in their comment for the additional information.

    And parameters with dynamic/calculated values?

    Remember the variable expansion rules in PowerShell. Enclose strings inside double-quotes, and variables inside will be expanded, e.g.

    $scriptsTempPath = "H:\backup\scripts temp"
    &$exe -p "-script=$scriptsTempPath\vss.cmd" E: M: P:

    Because variable expansion only works if strings are enclosed inside double-quotes, double-quotes are required, regardless if whether or not there are spaces in the parameter. You can have as many variables as you want inside each parameter.

    Or a variable containing a single parameter?

    $scriptsParameter = "-script=H:\backup\scripts temp\vss.cmd"
    &$exe -p $scriptsParameter E: M: P:

    No double-quotes are required here because the variable is surrounded by whitespace, so PowerShell will automatically expand the variable into a parameter. Using double-quotes won't break anything though; it is just redundant.

    But what if I want to build the arguments to pass in my script?

    You need to know a PowerShell secret. If you specify an array of values, it will automatically expand them into separate parameters. For example,

    $drivesToBackup = @( ) # new empty array
    $drivesToBackup += "E:" # always backup E drive
    
    # only backup C drive on the first of each month
    if ((Get-Date -Format dd) -eq 1) {
        $drivesToBackup += "C:"
    }
    
    &$exe -p "-script=H:\backup\scripts\vss.cmd" $drivesToBackup

    If today was the first of the month, and if you run echoargs.exe you'll get the following output:

    Arg 0 is <-p>
    Arg 1 is <-script=H:\backup\scripts\vss.cmd>
    Arg 2 is <E:>
    Arg 3 is <C:>

    All of the above tricks work fine with command line apps that use the forward-slash (/) to denote the start of a parameter too (instead of a dash/hyphen), e.g.

    &$exe /p "/script=H:\backup\scripts\vss.cmd" E: M: 

    But it still doesn't work!?!!!

    Sometimes you run into command line apps that use non-standard notation (not that there ever was much of a defined standard). Something like this for example (this is a command line from scripting ShadowProtect),

    &"H:\backup\scripts\sbrun.exe"  -mdn ( sbvol -f  \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy43 \\?\E: : sbcrypt -50 : sbfile -wd H:\backup\backups )

    If we run this using the tricks above, or even with echoargs.exe, you'll get PowerShell errors. Here's why - the parentheses in PowerShell denote code that should be executed and the result inserted in place of the parentheses. So in the above code, PowerShell is trying to find a cmdlet named sbvol, or an executable named sbvol in PATH. It fails because no such command exists by default. In a way, it is like using backticks in UNIX shells.

    To stop PowerShell from interpreting the parentheses and just pass them on instead, simple enclose them in quotes, e.g.

    &"H:\backup\scripts\sbrun.exe"  -mdn "(" sbvol -f  \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy43 \\?\E: : sbcrypt -50 : sbfile -wd H:\backup\backups ")"

    Using the curly brackets, or braces, will also trip up Powershell. If you need to pass the brace characters { } to an external command, they will need to be enclosed in quotes, otherwise you'll get cryptic parameters passed to your external command app, e.g.

    PS C:\Users\Sam> .\echoargs.exe { hello }
    Arg 0 is <-encodedCommand>
    Arg 1 is <IABoAGUAbABsAG8AIAA=>
    Arg 2 is <-inputFormat>
    Arg 3 is <xml>
    Arg 4 is <-outputFormat>
    Arg 5 is <text>
    
    PS C:\Users\Sam> .\echoargs.exe "{" hello "}"
    Arg 0 is <{>
    Arg 1 is <hello>
    Arg 2 is <}>

    What's actually happening is that PowerShell considers the contents of the braces to be a script block, which are often used with cmdlets such as Where-Object or ForEach-Object.

    The square brackets [ ] also have special meaning in PowerShell (globbing), but generally won't be interpreted as anything special when you're executing external commands; only certain cmdlets trigger the globbing behaviour, e.g. Get-ChildItem. So using them without enclosing them in quotes is fine.

    Also remember the character that triggers PowerShell's variable expansion, the dollar sign ($). It should be escaped using a backtick if it is to be passed to the external executable. PowerShell is actually quite specific when it comes to parsing the $ sign, but it is often safer to escape just in case.  If in doubt, try using single-quotes instead (variable expansion does not happen with single-quoted strings).

    Other bits of useful info

    To refer to the current directory, use the dot, e.g.

    &".\echoargs.exe"

    Note that the current directory may not necessarily be the directory the script is running from - it is dependent on the 'working directory' when executing the script, and also if you do any 'cd' or 'Set-Location' commands.

    To get the script directory, include the following line within the script file, in the script scope (i.e. not within a function or some other script block). Source.

    $scriptDirectory = Split-Path ($MyInvocation.MyCommand.Path) -Parent

    Lastly, if you want to send the output of the command line app to the screen, and you're running that inside a function, pipe the command to Out-Host to force it to the screen, e.g.

    &$exe -p "-script=H:\backup\scripts temp\vss.cmd" E: M: P: | Out-Host

    And if you want PowerShell to wait until that external process has finished before proceeding (but you don't want the output going anywhere), use Out-Null, e.g.

    &$exe -p "-script=H:\backup\scripts temp\vss.cmd" E: M: P: | Out-Null

    If you did want the output you can either pipe it to Out-Host instead to show it on the screen or if you want it in a variable, you can pipe it to the Tee-Object cmdlet first, like this —

    &$exe -p "-script=H:\backup\scripts temp\vss.cmd" E: M: P: | Tee-Object -Variable scriptOutput | Out-Null

    The output can then be accessed using the scriptOutput variable, e.g.

    echo $scriptOutput

    When the output of a command is piped to another cmdlet, PowerShell has to stop and wait for the initial command and the cmdlets the output has been piped into to complete before continuing.

     

    Changelog

    1.

    Added a section about using curly brackets in external command parameters.

    23rd June 2010 5:40 PM
     
    2.

    Updated post with the information in comment #3. Thanks @rcr for pointing this out in comment #2.

    28th January 2012 4:04 PM
     
    3.

    Added a new 'Using PowerShell v3?' section to explain the new --% operator.

    2nd September 2012 10:38 AM
     
    4.

    Added further examples on causing PowerShell to stop and wait for a command to complete before continuing.

    16th December 2012 4:20 PM
     
    5.

    Added details and fixed some issues pointed out by JFL in the comments.

    22nd December 2013 3:23 PM
     

Comments

1.

Wow, what a fantastic post. Extremely well-written and easy to read, yet incredibly informative. What a perfect balance.


Thanks!

Posted by Justus Thane, 16th April 2011 7:46 AM
 
2.

Excellent article thanks.

I'm having a problem with one of your examples though. I'm trying to pass an argument that contains spaces and requires quotes. 

Trying your example for this:

&$exe -p "-script=`"H:\backup\scripts temp\vss.cmd`"" E: M: P:

results in the script argument being split at the space:

Arg 0 is <-p>
Arg 1 is <-script=H:\backup\scripts>
Arg 2 is <temp\vss.cmd>
Arg 3 is <E:>
Arg 4 is <M:>
Arg 5 is <P:>

Any idea how to get around this?

Posted by rcr, 20th June 2011 12:27 PM
 
3.

@rcr - That's a curly one. I'm assuming you want this output from echoargs -

Arg 0 is <-p>
Arg 1 is <-script="H:\backup\scripts temp\vss.cmd">
Arg 2 is <E:>
Arg 3 is <M:>
Arg 4 is <P:>

To do this, you need to double-escape the quotes inside the string, once for PowerShell using the backtick character (`), and again for the Windows command processor using the backslash character (\). The following is the amended PowerShell command -

&$exe -p "-script=\`"H:\backup\scripts temp\vss.cmd\`"" E: M: P:

Escaping it a second time for the command processor is necessary, because otherwise it will attempt to interpret the quotes. From what I can gather, the command processor splits up the string into arguments by splitting at each space, unless the space is inside quotes. Because the inner quotes were not escaped, the command processor interpreted them as if the quoted parts contained "-script=" and "", therefore the space between 'scripts' and 'temp' isn't actually within any quotes and hence split.

You can see this happening by playing with echoargs.exe inside the Command Prompt (not the PowerShell prompt).

The order of the escape characters is important - it must be the backslash character first, then the backtick character. Otherwise, because PowerShell processes the command first, the backtick will escape the backslash instead of the quote as intended.

Posted by [edgylogic] sam, 20th June 2011 8:17 PM
 
4.

This has to be one of the greatest articles written for PowerShell.. you have explained everything in great detail.

I happen to use your examples and they work fine untill I discovered something new that I've been cracking my head for a over 2 days.. Consider the following

&$exe "/D:"""D:\My Backup\Documents"""" /T:9 "/O:"""C:\test\testscan1.xml"""" "/T:"""asdf""""

The above works fine.. but the folder paths are dynamic and they keep changing, so I store the location in a string variable.

$D = "/D:`"D:\My Backup\Documents`""

&$exe $D /T:9 "/O:"""C:\test\testscan1.xml"""" "/T:"""asdf""""

the above just doesnt work, I've tried playing around with echoargs.exe and moving the quotes around.. but it still fails. Is there a way to pass a variable in a parameter??


regards,

Jesse

Posted by Jesse, 18th January 2012 11:44 PM
 
5.

@Jesse -

Comment number 3 answers your question. I'll update the post to point to that comment as well, because the line as it is in the post doesn't work.

PS C:\Users\Sam> $exe = "./echoargs.exe"
PS C:\Users\Sam> &$exe "/D:\`"D:\My Backup\Documents\`"" /T:0 "/O:\`"C:\test\testscan1.xml\`"" "/T:\`"asdf\`""
Arg 0 is </D:"D:\My Backup\Documents">
Arg 1 is </T:0>
Arg 2 is </O:"C:\test\testscan1.xml">
Arg 3 is </T:"asdf">

PS C:\Users\Sam> $backupPath = "D:\My Backup\Documents"
PS C:\Users\Sam> &$exe "/D:\`"$backupPath\`"" /T:0 "/O:\`"C:\test\testscan1.xml\`"" "/T:\`"asdf\`""
Arg 0 is </D:"D:\My Backup\Documents">
Arg 1 is </T:0>
Arg 2 is </O:"C:\test\testscan1.xml">
Arg 3 is </T:"asdf">

Posted by [edgylogic] sam, 28th January 2012 3:51 PM
 
6.

how does one call another powershell script with an array of arguments?

my test script:

 

 

param ( [string] $string1, [string] $string2 )

 

if ( $string1 -or $string2 ) {

    write-host "string1 = ", $string1

    write-host "string2 = ", $string2

    exit

} else {

    $test = @( '-string1', "test" )

    & $myinvocation.mycommand.path $test

}

 

 

> .\test.ps1 hello world

string1 =  hello

string2 =  world

 

so far, so good.  but then:

 

 

> .\test.ps1

string1 =  -string1 test

string2 =

 

oops!  if i change the invocation line to call the powershell executable like so:

& 'C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe' -file $myinvocation.mycommand.path $test

then it works as expected, but the return is always a string, i can't pass objects back.

 

any ideas?

Posted by john, 5th April 2012 1:23 AM
 
7.

This got me further than any other help, but, the problematic case of having to send arguements with spaces like "-Optfile="c:\Program Files\Tivoli\TSM\baclient\dsm-srv02.opt" and others that need to include a path possibly with spaces.  None of the cases above worked and I was getting pretty tired of echoargs vs. powershell vs. the Tivoli dsmadmc executable!  Using the \`" simply wouldn't work with this executable.

What finally worked (which I got from http://technet.microsoft.com/en-us/library/ee692752.aspx) was:

 

  $oSFO = New-Object -ComObject Scripting.FileSystemObject     # This exposes the 'GetFile' method, which itself exposes the 'ShortPath' 

  $strDSM_Optfile = "C:\Program Files\Tivoli\TSM\baclient\dsm-srv01.opt" # passed-into the script as a parameter

  $strDSM_Optfile = ($oSFO.GetFile($strDSM_Optfile)).ShortPath  # $strDSM_Optfile contains just the filespec not the option

 

PS > $strDSM_Optfile

C:\PROGRA~1\Tivoli\TSM\baclient\DSM-SR~2.OPT

Finally!  A value that was accepted by the executable!

 

  

Posted by Stef, 19th April 2012 1:55 AM
 
8.

@john - what you're after is called 'argument unpacking' or 'parameter unpacking'. Unfortunately, I don't know of a way to do that in PowerShell. The trick in the article works because PowerShell needs to force the array into strings in order to pass it to a non-PowerShell command (which only allows strings as arguments). As you've noticed, when working with PowerShell commands, PowerShell simply passes the array as an array because PowerShell commands understand arrays.

UPDATE 02/09/2012: @john, I've just discovered a way to do this. You need to use a somewhat obscure PowerShell feature called splatting. Check out this article for more information. Essentially, you need to build your command arguments as a hashtable, e.g.

$args = @{ string1 = 'hello'; string2 = 'world' }
and pass it to the other PowerShell script using the @ operator, e.g.
./test.ps1 @args

@Stef - Out of curiosity, does entering the command as you expect to type it (e.g. dsmadmc -Optfile="c:\Program Files\Tivoli\TSM\baclient\dsm-srv02.opt") work in the Windows Command Prompt, i.e. not PowerShell? Maybe that command (which I'm not familiar with at all :) doesn't understand quoted parameters and just splits each argument at a space. Either way, glad you managed to get it working in the end; should be helpful to anyone else stuck with odd commands like this.

Posted by [edgylogic] sam, 22nd April 2012 12:53 PM
 
9.

What a fabulous post.

Greatest thanks,

Posted by Chengwei, 14th May 2012 9:18 AM
 
10.

Guess I need two posts for all of this...

My problem is that I am reading in the arguments from a parameter file using import-csv, parsing the data and setting PowerShell variables. Thus, I have one entry as such (each is one line in the file):

ROOT,C:\Project\Development\Data Conversion\SQLSCRIPTS\DB_DataConversion\Current\
and

DtCommandLine,/FILE "%ROOT%DB_Extract_Master\DB_Extract_Master.dtsx" /CONFIGFILE "%ROOT%DB_Extract_Master\DB_Extract_Master.dtsConfig" /CONFIGFILE "%ROOT%DB_Extract_Master\EtlConnectionManager.dtsConfig" /CONNECTION "DB_EXTRACT";"\"Data Source=LOCALHOST;Initial Catalog=DB_EXTRACT;Provider=SQLNCLI10.1;Integrated Security=SSPI;\"" /SUM /CHECKPOINTING OFF /REPORTING EWD

The pertinent code in my PS script is as follows:

$dtPath = (get-item 'HKLM:\SOFTWARE\Microsoft\Microsoft SQL Server\100\DTS\Setup').GetValue('SQLPath')

$CSV = import-csv .\Params.csv

$TMP   = $CSV | where { $_.Key -eq "ROOT" } | Select Value
$ROOT   = $TMP.value

$TMP   = $CSV | where { $_.Key -eq "dtExecutable" } | Select Value
$dtExecutable  = $TMP.value

$TMP   = $CSV | where { $_.Key -eq "dtCommandLine" } | Select Value
$dtCommandLine  = $TMP.value

$dtPackagePath  = $ROOT + "DB_Extract_Master\"
$dtCommandLine = $dtCommandLine -replace "%ROOT%", $ROOT

$dtExecutable = '"' + $dtPath + $dtExecutable + '"'
$dtExecutable += " $dtCommandLine"

$PackageLog = $ROOT + "\Package.log"

write-host "Executing: $dtExecutable"
.\echoargs.exe $dtCommandLine

Posted by Doug, 27th July 2012 1:32 AM
 
11.

The line write-host "Executing: $dtExecutable" displays:

Executing: "C:\Program Files\Microsoft SQL Server\100\DTS\Binn\DTExec.exe" /FILE "C:\Crossroads\Development\Data Conversion\SQLSCRIPTS\NC_DataConversion\Current\NC_Extract_Master\NC_Extract_Master.dtsx" /CONFIGFILE "C:\Crossroads\Development\Data Conversion\SQLSCRIPTS\NC_DataConversion\Current\NC_Extract_Master\NC_Extract_Master.dtsConfig" /CONFIGFILE "C:\Crossroads\Development\Data Conversion\SQLSCRIPTS\NC_DataConversion\Current\NC_Extract_Master\NcEtlConnectionManager.dtsConfig" /CONNECTION "NC_EXTRACT";"\"Data Source=LOCALHOST;Initial Catalog=NC_EXTRACT;Provider=SQLNCLI10.1;Integrated Security=SSPI;\"" /SUM /CHECKPOINTING OFF /REPORTING EWD

and this will run if I copy all but Executing: and paste to the command line. However, EchoArgs displays:

Arg 0 is </FILE C:Project\Development\Data>
Arg 1 is <Conversion\SQLSCRIPTS\DB_DataConversion\Current\DB_Extract_Master\DB_Extract_Master.dtsx /CONFIGFILE C:\Project\Development\Data>
Arg 2 is <Conversion\SQLSCRIPTS\DB_DataConversion\Current\DB_Extract_Master\DB_Extract_Master.dtsConfig /CONFIGFILE C:\Project\Development\Data>
Arg 3 is <Conversion\SQLSCRIPTS\DB_DataConversion\Current\DB_Extract_Master\EtlConnectionManager.dtsConfig /CONNECTION DB_EXTRACT;"Data>
Arg 4 is <Source=LOCALHOST;Initial>
Arg 5 is <Catalog=DB_EXTRACT;Provider=SQLNCLI10.1;Integrated>
Arg 6 is <Security=SSPI;" /SUM /CHECKPOINTING OFF /REPORTING EWD >

which obviously is not what I want. I've tried using &, invoke-expression, start-process and other suggestions, to no avail. How would I store the string in a cvs file that is being processed and the data set to variables in PowerShell so that this will be properly interpreted?
 

Posted by Doug, 27th July 2012 1:32 AM
 
12.

@Doug - I'm stumped on this one as well. The only way I can think of is to do it as shown in the first code block under the 'The wrong way to do it' section. That triggers the built in .NET class that probably passes the string to a Win32 API function that parses the argument string into separate arguments and starts the process. Depending on if you need to do anything with the console output, that example there may be all that you need.

You could try splitting the string on the space, but then you need some kind of magic to handle quoted strings with spaces in them.

PowerShell itself has a way of parsing it (as proven when you paste that command string into the shell), but I'm inclined to think that's the whole PowerShell language parser (i.e. the bit of code that parses all of the PowerShell syntax) and not a specific parser for command line strings that you can reuse.

Posted by [edgylogic] sam, 31st July 2012 10:24 PM
 
13.

Not sure if there is a "right" or "wrong" way in powershell.  In many cases you cannot add a "-wait" parameter to pause a script.  However by calling a calss in the .net Framework you can add line 4::

   4 $proc.WaitForExit()

This approach is critical if you need for Powershell to complete a processes before continuing the script

Posted by Rodney Jones, 17th October 2012 4:37 AM
 
14.

Fantastic article, thank-you!  I have been banging my head against the wall for so long trying to figure all of this out and you pretty much covered every problem area I had.

Posted by Rodney, 8th November 2012 9:43 AM
 
15.

@Rodney Jones, you should be able to use the trick listed in the Other bits of useful info section, i.e. piping the output of the command to Out-Null. If you wanted the output, you can pipe the command to Tee-Object first. I've added an example to the article with that.

Posted by [edgylogic] sam, 16th December 2012 4:04 PM
 
16.

I do appreciate this information as documentation isn't as well explained!  My issue is when I call a Powershell script from VBA and pass a variable (a string containing a file's path selected by the user) as a parameter that contains a space.  If the path doesn't contain a space, Powershell finds the file and does its job.  Should I preprocess the string in VBA, so that theres an " ` " tick mark before the space and then pass it to Powershell? Or, would it be easier to process it in Powershell itself?

Posted by Lawrence Knowlton, 22nd June 2013 7:43 AM
 
17.

I figured it out!:

I needed to add single quotes to the beginning and end of the strings containing spaces, before passing them as parameter variables to the shell call to Powershell in VBA.

' BEGIN VBA CODE:

FileToOpen_PS = "'" & <path string with space> & "'"

FileToSave_PS = "'" & <path string with space> & "'"

A_Placeholder_Variable = Shell("powershell.exe -ExecutionPolicy Unrestricted C:\Users\larry\Desktop\Convert_To_Tab_Delimited.ps1 " & _

FileToOpen_PS & " " & FileToSave_PS, 1)

' END VBA CODE

# BEGIN PS CODE:

param([string]$FileToOpen, [string]$FileToSave)

gc $FileToOpen | % { $_ -replace '  +',"`t" } | set-content $FileToSave

# END PS CODE

I've spent hours trying to figure this out, but stopped and thought about what Powershell wanted to see.  I'd gotten side tracked by searching for the answer to this on various forums and none of the situations seemed to fit.  Between ampersands and multiple quotes, my head was spinning, but I learned a lot.  Now to further process the file in VBA!

Posted by Lawrence Knowlton, 22nd June 2013 9:01 PM
 
18.

This article is great... but it still does not get to the bottom of it. I've wasted hours and hours of work trying to fix PowerShell scripts that invoked external programs... And there are some that resist everything.

One thing is definitely incorrect, starting with: "Because the inner quotes were not escaped using the Windows Command Processor escape character (the backslash)"

cmd.exe does not have any notion of lists of arguments to pass to applications, nor does it do any escaping using the backslash character. All it does is to parse the tokens in the command lines, to find the program name, input/output redirections (< >), pipes (|), and command sequencing operators (& && || ( )). The escape character for all these operators is the ^ character. Once it has identified all these tokens, it removes them from the command line, and passes everything that's left (including all quotes), as a single argument line, to the program it starts. That token parsing is influenced by quotes: Every quote encountered flips the parser in and out of an "inside string" mode. When inside a string, the < > | & ( ) characters do not have any special meaning. There is definitely no escaping of quotes with backslashes, and unfortunately not with the ^ character either. Note that this is a simplified description. For a more detailed (and still incomplete) discussion of the (extremely complex) cmd.exe parsing rules, see http://stackoverflow.com/questions/4094699/how-does-the-windows-command-interpreter-cmd-exe-parse-scripts/4095133#4095133

What you observe with the argstest.exe program is the escaping done internally within the C/C++/C# startup code. Every C/C++/C# console program has a startup routine that processes the Windows command line, and tries to emulate the Unix environment that C-family programs expect. This startup routine parses the Windows command line, and generates the argc/argv[] array for the main() routine, the way a Unix shell would have done. It is it that uses quotes to identify the beginning and end of "complex" arguments, then enters them as a single entry in argv[] without the enclosing quotes. And it is it that uses the \ character to escape quotes that are to be retained in argv[] entries.

So why bother discussing all this if \ escaping is done eventually anyway? The distinction is important when you run something other than a console-based C/C++/C# program. And unfortunately, even though it's a console based program, cmd.exe itself, invoked either directly or indirectly by running a batch file, parses arguments differently for its internal commands. Here's for example an ArgsTest.bat script that displays its own arguments like ArgsTest.exe. And when you run ArgsTest.bat, you'll see that the arguments it displays are NOT the same as those ArgsTest.exe displays for the very same command line. Likewise similar programs written in other programming languages would probably display yet another set of different values. Each language will have its own startup code, doing its own parsing of the Windows command line with its own rules. In a distant past I was doing assembly language programming, and command line parsing was a very personal choice everybody did in his own way.

@echo off
:# Display all command line arguments.
:# This is not as easy as it seems, as the last argument may contain mismatched " quotes.
:# Such mismatched quotes break the syntax of many commands.
:#
:# Enable the syntax extensions, to get the "if defined" and "set /a" constructs.
:# But disable delayed expansion, to avoid issues with '!' characters.
setlocal enableextensions disableDelayedExpansion
:# Process command line arguments.
set N=1
goto get_args
:next_arg
shift
:get_args
set ARG=%1
:# If there's no ARG, we're done.
:# Note: Testing it with "if .%1.==.. ()" breaks in case of mismatched " quotes.
if not defined ARG goto :eof
echo ARG%N% = %1
set /a N=N+1
goto next_arg

Actually cmd.exe internal commands themselves seem to have been written in assembly language by different people! They don't all have the same rules for splitting their own arguments! (Some use commas or semi-colons as separators, others don't.) Worse still, the cmd.exe tokenizer itself is self-inconsistent, as the token parsing rules used for command lines typed at the command prompt are slightly different from those used for parsing batch file lines. The most obvious difference is the treatment of % signs in for loops variables. But there are other more subtle ones.

Some Windows system programs are problematic too. Even though they presumably were written in C or C++, they seem to bypass the standard startup routine, and do parsing in their own way. One of them (I can't remember which one now) drove me crazy, because it wanted arguments formatted like name="value string", but failed when receiving "name=value string". I think I fixed the problem by running cmd.exe /c "command line". This works because cmd.exe has a special rule that after /c, if there are special characters like &<>()@^|", it removes the first and last quote, before parsing the line (with all the remaining quotes in between). See cmd /? for details.

More recently I've had serious problems with some Windows GUI programs too. Contrary to console-based programs, they begin with a WinMain() routine, and they too often do their own command line parsing. Some programmers have been very "creative", and have invented syntaxes that are almost impossible to use from powerShell.

All that to say that the only sensible solution guarantied to work in ALL cases, in pure PowerShell, without invoking an intermediate instance of cmd.exe, is PowerShell v3's --% argument. I don't like it very much, because in my opinion it does not look good and is not very readable. Instead they should have created a simple command with just TWO arguments, like: exec $program $argline. This would have matched exactly Windows internal way of invoking programs, and would have given easy control on exactly what argument line ends up on the other side.

Finally, I still have a few remaining XP systems that can't support PowerShell v3. The problem is slowly fading away along with these XP systems. Still, for the sake of the art, I'd be interested in finding a method that works well for them! Even though a pure PowerShell v2 solution that works in all cases is impossible, I'm convinced it IS possible to find something that works in most cases. If anybody makes progress, please tell us :-)

Posted by JFL, 20th December 2013 1:41 AM
 
19.

@JFL thanks for that very detailed comment! I've combined your multiple comments into one (I'll assume you had issues doing that; I'll have a look) and added your information into the original article. I also added a PowerShell script that spits out the command line that Windows passes to a program so you can see what PowerShell did to it without also having to deal with what MS C/C++ runtime does in echoargs.exe.

I must admit, I was never very familiar with cmd.exe but looking at the SO question you linked to just made my head hurt, all for the sake of backwards compatibility!

I suspect the --% operator was modelled on the -- operator for bash, which does effectively the same thing. Having a 'exec $program $argline' option makes sense if you know how Windows works underneath, but I think just introduces confusion otherwise. I think it is possible to write a PowerShell cmdlet that can do it though.

Posted by [edgylogic] sam, 22nd December 2013 4:27 PM
 
20.

Thank you. Solved my problem with passing a {GUID} with the braces to vssadmin. I was trying the back tick and it wasn't working. Who would have thought just double quoting each brace was the the answer? I have a feeling that this is already trodden ground.

Posted by jw, 27th March 2014 9:48 PM
 
21.

I also found this to be a useful function:

function get-args {Param([string]$command) [management.automation.psparser]::Tokenize($command,[ref]$null)}

... then call with your (either single or double quoted) command line:

get-args '.\vssadmin.exe delete shadows /sh
adow={36c9e016-bd30-421d-bb77-b50ad6b007f4}'

... and it produces output similar to:

Content     : .\vssadmin.exe
Type        : Command

Content     : delete
Type        : CommandArgument

Content     : shadows
Type        : CommandArgument

Content     : /shadow=´
Type        : CommandArgument

Content     : {
Type        : GroupStart

Content     : 36c9e016-bd30-421d-bb77-b50ad6b007f4´
Type        : Command

Content     : }
Type        : GroupEnd

... I've condensed the output by removing the Start, Length, StartLine, StartColumn, EndLine, EndColumn properties from the listing.

Posted by jw, 27th March 2014 9:56 PM
 
22.

Thanks. This helped me...a lot!

Posted by Rock, 16th April 2014 12:03 PM
 
23.

Thanks for this post, helped me out of a bind. Was informative and well structured, look forward to reading more like it.

Posted by Steve, 15th October 2014 8:33 PM
 
24.

Hi, maybe a bit off topic, but this is the closest set of advice I've found on how to run Winzip's commandline command wzunzip.  My problem is that when running this command, it displays progress 'dots'.  Cute by themselves, but in a script those show up as red as though there is an error and also make the console screen very difficult to review the rest of the results / progress output in the script.  For example, the progress might show elapsed time during processing for each step, so the expected output is three write-host rows with timestamps.  When running wnunzip as one of the commands, then these dots show up.  I can't figure out how to redirect / suppress those.

Now, you might wonder, why bother at all?  I'm using Powershell v4 and there are scripts that use the unzip feature from windows.  The file is huge, the winzip version takes seconds, the windows version many minutes to unzip the same file to the same location using the same computer.

Posted by Brad, 22nd April 2015 6:42 PM
 
25.

Brad, those red dots you see are being printed by WinZip into a stream called standard error. Traditionally, it is used for error messages during processing, hence the red colour in PowerShell.

To get rid of them, just redirect standard error to $null, by enclosing the command in brackets and adding a redirection expression on the end, e.g.

(ipconfig.exe) 2> $null

The 2> bit tells PowerShell to redirect standard error (without the 2, it would just redirect standard out, the white text in the console) and $null tells it to just throw it away.

Posted by [edgylogic] sam, 20th May 2015 6:52 AM
 
26.

That is one of the most comprehensive and readable and useful tutorials I have ever read. Thanks

Posted by RobG, 31st July 2015 2:48 PM
 
27.

This saved me a whole bunch of time, thanks a lot

Posted by Ryan, 2nd September 2015 3:06 PM
 
28.

Проще всего поддается обустройству комната с двумя окнами, расположенными вдоль одной стены длинного коридора. Такое помещение можно без труда превратить в два, в каждом из которых будет находиться одно окно. Сделать перегородку вы сможете как стационарную, так и раздвижную, что также зависит от личного решения. В качестве межкомнатной перегородки прекрасно подойдут нижнеопорные алюминиевые двери купе от обычного шкафа купе. Пол прихожей самое быстрозагрязняющееся место в квартире, и его приходится убирать и мыть чаще, чем остальную площадь. И чтобы покрытие не потеряло вид, его стоит подобрать правильно. Хорошо подойдет линолеум повышенной прочности, паркетная доска не очень хороший вариант, потому что уборка в прихожей должна быть влажной. Можно рассмотреть, как вариант, напольную плитку, практично, долговечно и не страдает от влаги. Спасибо. мама Прихожая – особенная комната. Переступая порог любого дома, сначала попадаешь именно в нее. Дизайн ее интерьера влияет на первое впечатление о жилом помещении вообще и, конечно же, о хозяевах в частности. Поэтому очень важно оформить ее соответствующим образом, учитывая размер, стилистику и назначение комнаты. Наиболее удачные варианты дизайна и фото прихожих в маленьких квартирах будут предложены в статье.маленькая прихожая пантограф это Совет: Сделайте гостевую зону просторной спальни акцентом всего помещения: расположите в ней дизайнерские кресла, к примеру, на изящных ножках и с яркой растительной обивкой.шкаф купе, двери купе

Posted by AmeliaPare, 4th June 2016 6:25 PM
 
29.

Fantastic post! I was googling 2 days to find out what I learnt here after 5 mins of reading. RESPECT!

Posted by chrisseroka, 16th July 2016 5:53 AM
 
30.

I have a semi-related stumper.  Pipelines.  Specifically, I'm trying to execute a windows port of CPIO, which _demands_ a redirected input.  And just for bonus difficulty, it requires the input to be of a list of filenames, separated by UNIX LFs.  So I've created a file list using dos2unix called cpiolistunix.

In DOS, this works:  (took me days to get to that point, alas)

cpio --create <cpiolistunix --file=testarc

I managed to get this working as well:

cat cpiolistunix | cpio --create --file=testarc

 

However, for the life of me I can't get it to work in powershell.  Even if I do get-content of the unix file and pipeline it to the app, nothing.  Any ideas? MANY thanks!

Posted by mbourgon, 6th August 2016 6:51 AM
 
31.

For Doug.

"%ROOT%"

could be replaced with:

"$ENV:ROOT"

Posted by B-Art, 13th September 2016 6:00 PM
 
32.

Is it possible to show standard output synchronously in real time. When I do it it works but the output only appears when the exe completes .. which is frustrating for a long running exe program.

Posted by Sanjay, 8th February 2017 9:09 AM
 

Leave a comment