Homebrew installed awk is incompatible

[aaronjensen]

This isn't necessarily a transcrypt issue as far as I can tell, so feel free to close this. I'm mainly opening for visibility and in the off chance that there is something that transcrypt should do to address.

If you initialize transcrypt in directory that already has encrypted files, it will fail to decrypt the files if you have awk installed from homebrew (version 20260426)

I reported the issue upstream here: onetrueawk/awk#270

[aaronjensen]

The issue was closed and they do not intend to support NUL as a separator. Given that, would it be worth considering an alternative implementation that does not rely on that?

[jmurty]

Thanks for reporting this.

Here is the affected code. It does some complicated rewriting of output from the git check-attr filter command.

transcrypt/transcrypt

Lines 181 to 185 in 51d5f87

	git -c core.quotePath=false check-attr filter -z --stdin 2>/dev/null |
	# Split null-delimited list of filename + filter entries to be one line
	# per file with space delimiters in that line by replacing every third
	# null character with newline and the rest with a space
	awk -v RS='\x00' 'NR % 3 == 0 {printf "%s\n", $0} NR % 3 != 0 {printf "%s ", $0}' |

An alternative implementation would be great, but it might be difficult to find a good one. It took us a few attempts to arrive at that awk command in the first place. The constraints at that point in the code are:

• we must use the difficult-to-handle NUL-delimited output from git check-attr filter with the -z option, to avoid bugs handling encrypted files with special characters in the file name (see (Re-)Initialization of a cloned repo fails #204 and Handle quotes and colon in filenames #173)
• the output from check-attr filter should be processed in bulk – or at least avoid use of subshells – to avoid terrible performance in a repo with many files (see performance: don't create subshells #193)
• ideally we wouldn't add a requirement for another shell tool.

We already use perl in some cases – as a fallback for when neither xxd nor printf with support for "%b" is available. So perhaps a perl command could be used as an alternative.

Even if there is a good perl alternative command to the awk one, I would prefer to continue using awk in most cases because that is almost certainly available on every system whereas perl may not be.

[jmurty]

Maybe the simplest solution would be for transcrypt to check for the availability of the GNU Awk gawk command and prefer that command when available, only falling back to awk when gawk isn't available?

On many (most?) systems these commands will be equivalent by default. At least, they are on my latest macOS installation.
But if the awk available on a system happens to be the https://github.com/onetrueawk version that doesn't support NUL characters, there would be a good chance that gawk will also be present and if so should be guaranteed to work the way we need.

[jmurty]

I think I have a fix on branch prefer-gawk (PR #214) using the simpler approach of preferring the gawk command to plain awk. Can you please test this @aaronjensen?

[aaronjensen]

@jmurty thank you for looking into this. I don't actually have gawk installed. I get "onetrueawk" awk because I compile/install emacs-plus via homebrew it has awk as a compile dependency. The only other awk I have is the macOS built-in awk as well at /usr/bin/awk, which does work with NUL bytes.

I think that points to version detection or special handling of macOS systems or preferring /usr/bin/awk. None of those sound great, honestly.