Repository navigation
Description
Summary
When an argument isn't valid in its encoding (UTF-8 under a UTF-8 locale), OptionParser#parse! raises ArgumentError: invalid byte sequence in UTF-8 from a regexp match instead of an OptionParser::ParseError. It happens for an option (--name=\xff), for an option's separate value (--name \xff), and for a non-option argument that optparse would otherwise return untouched, such as a file name in Latin-1 (caf\xe9.txt). A program that rescues OptionParser::ParseError crashes with a backtrace instead, and so does ARGV.options { ... }, which documents that it rescues ParseError and prints the message. ri crashes the same way.
Reproduction
opt.rb:
require "optparse"
parser = OptionParser.new do |o|
o.on("-n", "--name=NAME", "a name")
o.on("-v", "--verbose")
end
begin
rest = parser.parse!(ARGV, into: opts = {})
p [:ok, opts, rest]
rescue OptionParser::ParseError => e
p [:parse_error, e.class, e.message]
endRun in bash, with a UTF-8 locale (LANG=C.UTF-8):
$ ruby -v
ruby 4.0.9 (2026-10-23 revision 3e93b23afc) +PRISM [x86_64-linux]
$ ruby -roptparse -e 'puts OptionParser::Version'
0.8.1
$ ruby opt.rb --bogus
[:parse_error, OptionParser::InvalidOption, "invalid option: --bogus"]
$ ruby opt.rb --name=$'\xff'
.../lib/ruby/4.0.0/optparse.rb:1738:in 'Regexp#===': invalid byte sequence in UTF-8 (ArgumentError)
from .../lib/ruby/4.0.0/optparse.rb:1738:in 'block in OptionParser#parse_in_order'
$ ruby opt.rb --name $'\xff'
.../lib/ruby/4.0.0/optparse.rb:566:in 'Regexp#match': invalid byte sequence in UTF-8 (ArgumentError)
$ ruby opt.rb $'caf\xe9.txt'
.../lib/ruby/4.0.0/optparse.rb:1738:in 'Regexp#===': invalid byte sequence in UTF-8 (ArgumentError)
$ ruby opt.rb -- $'caf\xe9.txt'
[:ok, {}, ["caf\xE9.txt"]]
$ ruby -roptparse -e 'ARGV.options { |o| o.parse! }' $'caf\xe9.txt'
.../lib/ruby/4.0.0/optparse.rb:1738:in 'Regexp#===': invalid byte sequence in UTF-8 (ArgumentError)
$ ri $'caf\xe9'
.../lib/ruby/4.0.0/optparse.rb:1738:in 'Regexp#===': invalid byte sequence in UTF-8 (ArgumentError)
(First backtrace lines only; ... is the Ruby installation's prefix.) lib/optparse.rb on master (39a3901) is the same as the 0.8.1 that Ruby 4.0.9 ships. Under the C locale, Ruby tags such arguments as ASCII-8BIT and they parse.
Expected
- For an option or an option's value with invalid bytes: an
OptionParser::ParseError, e.g.OptionParser::InvalidArgument, which the class tree describes as the "errors on parsing". - For a non-option argument, I'd propose returning it untouched (
[:ok, {}, ["caf\xE9.txt"]]): optparse doesn't otherwise look inside non-options, the argument may be a valid file name in another encoding, and after--it is already returned untouched. RaisingOptionParser::InvalidArgumentfor it would still be better than anArgumentError.
Cause
OptionParser#parse_in_order classifies each argument with a case over regexps, the first being the long-option one (L1738, when /\A--([^=]*)(?:=(.*))?/m). A regexp match on a string with invalid bytes raises ArgumentError, so every such argument fails there, option or not. An option's value given as the next argument fails the same way in Switch#parse_arg (L566). Nothing turns either into a ParseError.
RubyGems rejects such arguments up front: Gem::GemRunner#validate_encoding raises Gem::OptionParser::InvalidArgument for any argument that isn't valid_encoding?. The crash that prompted it was in its own config code, but the check also keeps such arguments away from its copy of optparse.
Activity
ihamzatarar commentedon Nov 16, 2026
I have opened a PR for this.
Two things turned up while fixing it that are worth recording here.
There are two sites, not one. Besides the
case arginparse_in_order,Switch#parse_argmatches the switch's own pattern against the value, so patching only the parser loop leaves--name=\xFFand-n\xFFstill raising.String#rindexis why the second site cannot simply convert back. On a string that is not valid in its encoding it does not raise; it quietly fails to find a substring the string does contain.parse_argcallsarg.rindex(s, 0)immediately after the match, so matching in binary and restoring the encoding before that line turns theArgumentErrorinto a spuriousInvalidArgumentinstead. Working in binary through that method avoids it.The change is two guards, one at each site: use
arg.bwhen the argument is not valid in its encoding. Option syntax is ASCII, so the structure still parses and the bytes are passed on as they arrived. Only arguments that raise today are affected, so nothing that currently works can see a difference.With
-n/--name=NAMEand-v/--verbosedefined, on Ruby 3.4.6:--bogusInvalidOptionInvalidOptioncaf\xE9.txtArgumentError--name=\xFFArgumentErrornameset to\xFF--name \xFFArgumentErrornameset to\xFF-n\xFFArgumentErrornameset to\xFF--verbose caf\xE9.txtArgumentErrorverbose, and the file name in the restcafé.txtOn the question of a
ParseError: passing the bytes through seemed closer to what Ruby already does, since it hands arguments toARGVas the bytes it was given without validating them, andFile.opentakes them back. AParseErrorwould also be an improvement over a backtrace, and the PR says I am happy to do that instead. Either wayARGV.options { ... }andristop crashing, since both rescueParseErrorand neither rescuesArgumentError.