The curious case of shell commands, or how "this bug is required by POSIX" (2021)

by wonger_on 6/10/2025, 12:56 PMwith 125 comments

by panzion 6/10/2025, 3:22 PM

Yeah, system() should definitely be deprecated and you should never use it if you write any new program. At least there is exec*() and posix_spawn() under POSIX. Under Windows there is no such thing and every program might parse the command line string differently. You can't naively write a generic posix_spawn() like interface for Windows, see this related Rust CVE: https://blog.rust-lang.org/2024/04/09/cve-2024-24576/ Why is it a CVE in Rust, but not in any other programming language? Did other language handle it better? Dunno, I just know that Rust has a big fat warning about this in their documentation (https://doc.rust-lang.org/std/process/struct.Command.html#me...), but e.g. Java doesn't (https://docs.oracle.com/javase/8/docs/api/java/lang/ProcessB...).

by o11con 6/10/2025, 2:44 PM

This is woefully misguided. Half the time passing it to the shell is explicitly a feature, e.g. `popen("gzip > foo.gz")`. If you have user input you should always sanitize it regardless of API.

But `ssh` does deserve all the shame. It's a pity the real problems are hard to find in an article full of nonsense.

Note also that if you're using a deficient shell that supports neither `printf %q` nor `${var@Q}` it's still easy to do quoting in `sed`. GNU `./configure` scripts do this internally, including special-casing to only quote the right side of `--arg=value`.

by chuboton 6/10/2025, 3:52 PM

The article mentioned printf '%q ', but it is a bit hard to find. Here is a handy way to remember it.

First, define this function:

    quote-argv() { printf '%q ' "$@"; }
    # (uses subtle vectorization of printf over args)
Now this works correctly:

    ssh example.com "$(quote-argv ls 'file with spaces')"
    ls: cannot access 'file with spaces': No such file or directory
In contrast to:

    $ ssh example.com ls 'file with spaces'
    ls: cannot access 'file': No such file or directory
    ls: cannot access 'with': No such file or directory
    ls: cannot access 'spaces': No such file or directory
And yes the "hidden argv join" of ssh is VERY bad, and it is repeated in shell's eval builtin.

They should both only take a SINGLE arg.

It is basically a self-own because spaces are an OPERATOR in shell! (the operator that separates words)

When you concatenate operators and variables, then you are mixing code and data, which is a security problem.

---

As for the exec workaround, I think this is also deficiency of shell. Oils will probably grow an 'invoke' builtin which generalizes 'command' and 'builtin', which are non-orthogonal.

'command true' means "external or builtin" (disabling shell function lookup), but there should be something that means "external only".

by endiangroupon 6/10/2025, 4:18 PM

AD: Huh! I just wrote a utility cmd [1] this weekend to deal with restricting ssh keys to executing only commands that match a rule set via `ForceCommand` in `sshd_config` or `Command=""` in `authorized_keys`. I'm curious to see how susceptible it is to the aforementioned issues, it does delegate to `<shell> -c '<cmd>'` under the hood [2], but there are checks to ensure only a single command option argument `--` is passed (to mitigate metacharacter expansions) [3].

Note this tool is only intended to be another layer in security.

[1] https://github.com/endiangroup/cmdjail [2] https://github.com/endiangroup/cmdjail/blob/main/main.go#L30... [3] https://github.com/endiangroup/cmdjail/blob/main/config.go#L...

by Wicheron 6/10/2025, 3:09 PM

For SSH specifically (ssh user@host "command with args") I've written this workaround pseudoshell that makes it easy to pass your argument vector to execve unmolested.

https://crates.io/crates/arghsh

by blueflowon 6/10/2025, 3:58 PM

Its not the manual pages that are ambiguous on this, its the author who used the word "command" but seemingly had a mental model of it as if it was an argument vector. A command and an argument vector are different things....

by a_t48on 6/10/2025, 4:06 PM

I've definitely gone down the rabbit hole of trying/being forced to fix issues like this. It starts off as just someone taking a shortcut of doing a little shell scripting in a python program or whatever. Generally the best tool I've found for fixing this is python's shlex.quote - https://docs.python.org/3/library/shlex.html but YMMV (multiple levels may be needed). The real best solution is not to shell out from your program when possible. :)

by hackernudeson 6/10/2025, 3:12 PM

I think this is a topic that every Linux user eventually stumbles into. It is indeed quite frustrating.

I found the article hard to follow, but maybe because I was already familiar with the problem and was just skimming. Skip to "Some experiments..." for the actual useful examples.

I disagree with the conclusion, though. I think there should just be more obvious ways to escape the input so one can keep their sanity with nested 'sh -c' invocation. Maybe '${var@Q}' and print '%q' are enough (can't believe I didn't know those existed!)

by 0xbadcafebeeon 6/12/2025, 2:33 PM

What the author is talking about here is the growing pains of learning how 60-year old *NIX systems work, and shells/shell scripting. Once you learn how it works, it all works fine. But it's not easy to learn. If we had a standard education track for this stuff, it would be easier. (but there would still be people who avoid the education, rush into things, then bump their shins into the coffee table, and blame the coffee table)

by pabs3on 6/11/2025, 1:31 PM

Related problems: command-line options that allow code execution[1], and commands that execute arbitrary code from the current directory.

1. https://web.archive.org/web/20201111203646if_/https://www.de...

by 1vuio0pswjnm7on 6/10/2025, 6:19 PM

"bash, obviously for scripting;"

It's possible to use bash for both interactive use and scripting. For example, this author claims to use bash as his scripting shell.

But Debian and the popular Debian-derived distributions do not use bash for scripts beginning with "#!/bin/sh", i.e., "shell scripts".

The interactive shell may be bash, but the scripting shell, /bin/sh, is not bash.

https://www.man7.org/linux/man-pages/man1/dash.1.html

https://wiki.ubuntu.com/DashAsBinSh

https://wiki.archlinux.org/title/Dash

https://www.oreilly.com/library/view/shell-scripting-expert/... ^1

https://www.baeldung.com/linux/dash-vs-bash-performance

https://en.wikipedia.org/wiki/Almquist_shell

https://lwn.net/Articles/343924/

https://scriptingosx.com/2020/06/about-bash-zsh-sh-and-dash-... ^2

I use an Almquist shell, not bash, for both interactive use and scripting. I often write scripts interactively. I use the same scripts on Linux and BSD. I restored tabcomplete and the fc builtin to dash so it feels more like the shell from which it was derived: NetBSD sh.

1. "This makes it smaller, lighter and faster than bash."

2. "... this is strong indicator that Apple eventually wants to use dash as the interpreter for sh scripts."

by zahlmanon 6/10/2025, 6:19 PM

> No, let's just try it out (I've put both the Python and the plain sh -c invocations):

  > python2 -c 'import os; os.system("-x")'
  > sh -c -x
  sh: -c: option requires an argument
I can't reproduce this in Python (including my local 2.7 build), only using sh directly. Going through Python, `sh` correctly tells me that the `-x` command isn't found.

But now I'm wondering: how is one supposed to use `which` (or `type`, or `file`) for a program named `-x`, supposing one had it?

by ptxon 6/12/2025, 2:26 PM

> BTW, this is not something Linux specific. Unfortunately it is a trait inherited from the UNIX ancestry by almost all operating systems, including all BSD variants [...] Hmm... Strange... There is nothing to quote from this manual about warnings, issues or sanitization...

This is not a problem on FreeBSD, if the problem is (as the article seems to say) that the documentation fails to warn about the requirement to properly encode arguments passed to the shell.

Here's the FreeBSD man page [1] for system(3):

  SECURITY CONSIDERATIONS
     The system() function is easily misused in a manner that enables a
     malicious user to run arbitrary command, because all meta-characters
     supported by sh(1) would be honored.  User supplied parameters should
     always be carefully santized before they appear in string.
[1] https://man.freebsd.org/cgi/man.cgi?query=system&sektion=3&m...

by teo_zeroon 6/13/2025, 5:31 AM

In POSIX world you aren't encouraged to use a specific command for each goal you might have. Instead you get a set of basic tools and combine them to achieve the desired result. Without system() you wouldn't have pipes, and you would need one command for every task you might want to run.

Point in case I encountered this week: I was editing a list and wanted to remove duplicates without changing the order of the lines. There's no ready-made program to do that, but this sequence of piped command served the purpose:

  cat -n | sort -uk 2 | sort | cut -f 2-
Fortunately my text editor supports system().

by pabs3on 6/11/2025, 1:40 PM

My 2014 blog post about this same issue:

https://bonedaddy.net/pabs3/log/2014/02/17/pid-preservation-...

by GuB-42on 6/13/2025, 2:31 AM

system() is for running shell commands and the article complains that it runs shell commands...

I rarely use it, and almost never in production, but it has its place. Think of it as the eval() of the POSIX world. If you want to build pipelines, or anything a shell has to offer, and do it simply, then system() is for you.

Security-wise, if you are using system() with user input, you are essentially giving shell access to the user, which may or may not be a big deal. If the intended users are people who already have a shell, that's fine maybe even desitable, otherwise, use something else, like exec*().

As for OpenSSH, what is the problem? The "SH" at the end means "shell", it runs shell commands, what did you expect?

by degamadon 6/10/2025, 2:42 PM

(2021)

by kouteiheikaon 6/10/2025, 5:30 PM

> Wall of shame: Ruby's backtick feature -- provides easy access to system(3);

It also provides easy access to escape whatever arguments you want to pass:

    out = `bash -c #{arg.shellescape}`
...here "arg" will be always passed as a single argument.

by hello_computeron 6/10/2025, 3:13 PM

There are so many neo-shells that go crazy with colors, autocompletions, & SQL-like features while the most basic problems (like handling of newlines/spaces/international chars) are mostly swept under the rug with -null/-print0, which is more hack than solution. I think Tom Duff's rc shell was an excellent start in that direction, which sadly went nowhere.