diff --git a/README.md b/README.md index 48116406..cc394605 100644 --- a/README.md +++ b/README.md @@ -323,9 +323,9 @@ If planned renames clash, the process is halted and no change is applied to any | `-s` | **`--dot-all`** Have `.` also match newline. [boolean] | | `-M` | **`--void-multiline`** Void multiline search pattern. Makes ^ and $ match start/end of whole content rather than each line. [boolean] | | `-u` | **`--unicode`** Treat pattern as a sequence of unicode code points. [boolean] | -| | **`--flags`** Set all the flags of the regex, like `gimsv`. Without it the flags are `gim` (`gimvxn` with `-E regex+`), changed by -G, -I, -M, -s and -u, which can not be used with --flags. `x` (free spacing) and `n` (only named groups capture) need `-E regex+`, which uses `v`, or `u` when it is given. [string] | +| | **`--flags`** The flags of the regex, like `gimsv`, in place of the defaults: `gim`, or `gimvxn` with `-E regex+`. `x` and `n` need `-E regex+`. [string] | | `-e` | **`--encoding`** Encoding of files/piped data. [default: "utf8"] | -| `-E` | **`--engine`** What regex engine to use. `rr` (also `default`) is Regex+ wrapped to read patterns like the native engine, with the extended syntax of Regex+ on top: atomic groups `(?>…)`, possessive quantifiers like `a++`, and subroutines `\g` with `(?(DEFINE)…)`. `regex+` (also `V`) is the Regex+ engine with flag v, free spacing (x), only named groups capture (n), atomic groups, possessive quantifiers and subroutines. On a runtime without v, like Node 18, it uses u. Use --flags to leave out x or n. `runtime` is the native engine of the runtime, and `V8` (Node) and `JSC` (JavaScriptCore, Bun) are the same, but stop the run on the other runtime. [choices: "rr", "default", "regex+", "V", "runtime", "V8", "JSC"] [default: "rr"] | +| `-E` | **`--engine`** The regex engine. `rr` (also `default`) reads patterns like the native engine and adds the extended syntax of Regex+: atomic groups `(?>…)`, possessive quantifiers like `a++`, and subroutines `\g`. `regex+` (also `V`) is Regex+ with the flags v, x and n. `runtime` is the native engine of the runtime. `V8` and `JSC` (JavaScriptCore) are the native engines of Node and Bun, and need that runtime. [choices: "rr", "default", "regex+", "V", "runtime", "V8", "JSC"] [default: "rr"] | | `-q` | **`--quiet`** Hide file names and warnings. Errors are still shown, and output from flags like --output and --output-match is still shown. [boolean] | | `-Q` | **`--quiet-total`** Never display errors or info [boolean] | | `-f` | **`--full-path`** Print the full path of the named files instead of the relative path. [boolean] | @@ -367,7 +367,7 @@ If planned renames clash, the process is halted and no change is applied to any | `-g` | **`--glob-pipe`** Each line of the piped data is a path or a glob of the files to work on. They are used together with the ones given as arguments. [boolean] | | | **`--null-in`** The piped paths and globs of --glob-pipe are separated by null bytes, like from `find -print0` or --null-out. [boolean] | | `-R` | **`--replacement-pipe`** Replacement will be piped in. You still need to provide a dummy value (like `_`) as replacement parameter. [boolean] | -| `-j` | **`--replacement-js`**, **`--js`** Treat replacement as javascript source code. The statement from the last expression will become the replacement string. Purposefully implemented the most insecure way possible to remove _any_ incentive to consider running code from an untrusted part. The full match will be available as a javascript variable named $0 while each captured group will be available as $1, $2, $3, ... and so on. At some point, the $ char _will_ give you a headache when used from the command line, so use €0, €1, €2, €3... instead. The code has access to the following variables: `r` as an alias for `require` with both expanded to understand a relative path even if it is not starting with `./`, `fs` from node, `path` from node, `globs` from npm, `pipe`: the data piped into the command with --js-pipe (null if no piped data, and `pipe_` is `❌ `), `find`: pattern searched for (the needle), `text`: full text being searched i.e. file content or piped data (the haystack), `bytes`: total size of the haystack in bytes, `size`: human-friendly representation of the total size of the haystack, `time`: String representing the local time when the command was invoked, `time_obj`: date object representing `time`, `now`: alias for `time`, `cwd`: current process working dir, `nl`: a new-line char, `_`: a single space char (for easy string concatenation). The following values defaults to `❌` if haystack does not originate from a file: `file`: contains the full path of the active file being searched (including full filename), `file_rel`: contains `file` relative to current process working dir, `dirpath`: contains the full path without filename of the active file being searched, `dirpath_rel`: contains `dirpath` relative to current process working dir, `filename`: is the full filename of the active file being searched without path, `name`: filename of the active file being searched with no extension, `ext`: extension of the filename including leading dot, `mtime`: ISO inspired representation of the last local modification time of the current file, `ctime`: ISO representation of the local creation time of the current file. `mtime_obj`: date object representing `mtime`, `ctime_obj`: date object representing `ctime`. The numbers: `$n`: the number of the file, `$t`: the number of files, `$i`: the number of the file among the files with a match so far, `$c`: among the files changed so far, `$m`: the number of the match in the file, `$M`: across all files. Each has cousins with zeros in front (`$n0` has 2 digits, `$n00` 3, and so on to 10) and with a space at the end (`$n_`). For speed, the code runs once per file unless it names the match, a group, `$m` or `$M`. Access a named group as `$` and the name: `(?...)` gives you `$year`. A group named like a number, like `n`, wins over the number. With the flag `n` of `-E regex+` only named groups capture, so there is no `$1`. All variables, except from module, date objects, `nl` and `_`, has a corresponding variable name followed by `_` where the content has an extra space at the end (for easy concatenation). [boolean] | +| `-j` | **`--replacement-js`**, **`--js`** Treat replacement as javascript source code. The statement from the last expression will become the replacement string. Purposefully implemented the most insecure way possible to remove _any_ incentive to consider running code from an untrusted part. The full match will be available as a javascript variable named $0 while each captured group will be available as $1, $2, $3, ... and so on. At some point, the $ char _will_ give you a headache when used from the command line, so use €0, €1, €2, €3... instead. The code has access to the following variables: `r` as an alias for `require` with both expanded to understand a relative path even if it is not starting with `./`, `fs` from node, `path` from node, `globs` from npm, `pipe`: the data piped into the command with --js-pipe (null if no piped data, and `pipe_` is `❌ `), `find`: pattern searched for (the needle), `text`: full text being searched i.e. file content or piped data (the haystack), `bytes`: total size of the haystack in bytes, `size`: human-friendly representation of the total size of the haystack, `time`: String representing the local time when the command was invoked, `time_obj`: date object representing `time`, `now`: alias for `time`, `cwd`: current process working dir, `nl`: a new-line char, `_`: a single space char (for easy string concatenation). The following values defaults to `❌` if haystack does not originate from a file: `file`: contains the full path of the active file being searched (including full filename), `file_rel`: contains `file` relative to current process working dir, `file_path` and `file_path_rel`: the same as `file` and `file_rel`, `dirpath`: contains the full path without filename of the active file being searched, `dirpath_rel`: contains `dirpath` relative to current process working dir, `filename`: is the full filename of the active file being searched without path, `name`: filename of the active file being searched with no extension, `ext`: extension of the filename including leading dot, `mtime`: ISO inspired representation of the last local modification time of the current file, `ctime`: ISO representation of the local creation time of the current file. `mtime_obj`: date object representing `mtime`, `ctime_obj`: date object representing `ctime`. The numbers: `$n`: the number of the file, `$t`: the number of files, `$i`: the number of the file among the files with a match so far, `$c`: among the files changed so far, `$m`: the number of the match in the file, `$M`: across all files. Each has cousins with zeros in front (`$n0` has 2 digits, `$n00` 3, and so on to 10) and with a space at the end (`$n_`). For speed, the code runs once per file, at its first match, unless it names the match, a group, `$m`, `$M` or `rr`. A file without a match does not run the code. Access a named group as `$` and the name: `(?...)` gives you `$year`. A group named like a number, like `n`, wins over the number. With the flag `n` of `-E regex+` only named groups capture, so there is no `$1`. `rr` is the object behind the short names: `rr.year` is `$year`, `rr[1]` is `$1` and `rr.n` is `$n`. Code that uses `rr` runs once per match. `rr._` has the values of RexReplace with long names: `rr._.fileInfo` (`filePath`, `filePathRel`, `dirPath`, `dirPathRel`, `dirName`, `fileName`, `name`, `ext`, `bytes`, `size`, `mtime`, `ctime`, `mtimeObj`, `ctimeObj`), `rr._.count` (`fileNumber`, `filesTotal`, `matchedFileNumber`, `changedFileNumber`, `matchThisFile`, `matchUntilNow`), and `text`, `find`, `pipe`, `cwd`, `time` and `timeObj`. `rr.stdin` is the piped data, and `rr.print()` and `rr.printErr()` print to stdout and stderr, also with --quiet-total. `pad(value, width = 3, char = '0')` pads a value, like `pad($1)` for `007`. A named group overwrites a name it shares, also in `rr`. All variables, except from module, date objects, `nl` and `_`, has a corresponding variable name followed by `_` where the content has an extra space at the end (for easy concatenation). [boolean] | | `-x` | **`--exclude-re`** Files with a path that matches this regular expression will be excluded from processing. Can be provided multiple times. [string] | | `-X` | **`--exclude-glob`** Files found with this glob will be excluded from being processed. Can be provided multiple times and will act as one group of globs, meaning you can use ! notation to negate. [string] | | `-h` | **`--help`** Display help. [boolean] | diff --git a/src/cli.ts b/src/cli.ts index d4b8cefe..ed9a0e20 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -119,11 +119,10 @@ const yargs = require('yargs') .alias('u', 'unicode') .string('flags') - // --flags sets all the flags, so the options that change a default flag can not go with it .conflicts('flags', ['I', 'G', 'M', 's', 'u']) .describe( 'flags', - 'Set all the flags of the regex, like `gimsv`. Without it the flags are `gim` (`gimvxn` with `-E regex+`), changed by -G, -I, -M, -s and -u, which can not be used with --flags. `x` (free spacing) and `n` (only named groups capture) need `-E regex+`, which uses `v`, or `u` when it is given.' + 'The flags of the regex, like `gimsv`, in place of the defaults: `gim`, or `gimvxn` with `-E regex+`. `x` and `n` need `-E regex+`.' ) .default('e', 'utf8') @@ -149,7 +148,7 @@ const yargs = require('yargs') .alias('E', 'engine') .describe( 'E', - 'What regex engine to use. `rr` (also `default`) is Regex+ wrapped to read patterns like the native engine, with the extended syntax of Regex+ on top: atomic groups `(?>…)`, possessive quantifiers like `a++`, and subroutines `\\g` with `(?(DEFINE)…)`. `regex+` (also `V`) is the Regex+ engine with flag v, free spacing (x), only named groups capture (n), atomic groups, possessive quantifiers and subroutines. On a runtime without v, like Node 18, it uses u. Use --flags to leave out x or n. `runtime` is the native engine of the runtime, and `V8` (Node) and `JSC` (JavaScriptCore, Bun) are the same, but stop the run on the other runtime.' + 'The regex engine. `rr` (also `default`) reads patterns like the native engine and adds the extended syntax of Regex+: atomic groups `(?>…)`, possessive quantifiers like `a++`, and subroutines `\\g`. `regex+` (also `V`) is Regex+ with the flags v, x and n. `runtime` is the native engine of the runtime. `V8` and `JSC` (JavaScriptCore) are the native engines of Node and Bun, and need that runtime.' ) .choices('E', ['rr', 'default', 'regex+', 'V', 'runtime', 'V8', 'JSC']) .default('E', 'rr') @@ -410,6 +409,7 @@ The code has access to the following variables: The following values defaults to \`❌\` if haystack does not originate from a file: \`file\`: contains the full path of the active file being searched (including full filename), \`file_rel\`: contains \`file\` relative to current process working dir, +\`file_path\` and \`file_path_rel\`: the same as \`file\` and \`file_rel\`, \`dirpath\`: contains the full path without filename of the active file being searched, \`dirpath_rel\`: contains \`dirpath\` relative to current process working dir, \`filename\`: is the full filename of the active file being searched without path, @@ -420,10 +420,12 @@ The following values defaults to \`❌\` if haystack does not originate from a f \`mtime_obj\`: date object representing \`mtime\`, \`ctime_obj\`: date object representing \`ctime\`. -The numbers: \`$n\`: the number of the file, \`$t\`: the number of files, \`$i\`: the number of the file among the files with a match so far, \`$c\`: among the files changed so far, \`$m\`: the number of the match in the file, \`$M\`: across all files. Each has cousins with zeros in front (\`$n0\` has 2 digits, \`$n00\` 3, and so on to 10) and with a space at the end (\`$n_\`). For speed, the code runs once per file unless it names the match, a group, \`$m\` or \`$M\`. +The numbers: \`$n\`: the number of the file, \`$t\`: the number of files, \`$i\`: the number of the file among the files with a match so far, \`$c\`: among the files changed so far, \`$m\`: the number of the match in the file, \`$M\`: across all files. Each has cousins with zeros in front (\`$n0\` has 2 digits, \`$n00\` 3, and so on to 10) and with a space at the end (\`$n_\`). For speed, the code runs once per file, at its first match, unless it names the match, a group, \`$m\`, \`$M\` or \`rr\`. A file without a match does not run the code. Access a named group as \`$\` and the name: \`(?...)\` gives you \`$year\`. A group named like a number, like \`n\`, wins over the number. With the flag \`n\` of \`-E regex+\` only named groups capture, so there is no \`$1\`. +\`rr\` is the object behind the short names: \`rr.year\` is \`$year\`, \`rr[1]\` is \`$1\` and \`rr.n\` is \`$n\`. Code that uses \`rr\` runs once per match. \`rr._\` has the values of RexReplace with long names: \`rr._.fileInfo\` (\`filePath\`, \`filePathRel\`, \`dirPath\`, \`dirPathRel\`, \`dirName\`, \`fileName\`, \`name\`, \`ext\`, \`bytes\`, \`size\`, \`mtime\`, \`ctime\`, \`mtimeObj\`, \`ctimeObj\`), \`rr._.count\` (\`fileNumber\`, \`filesTotal\`, \`matchedFileNumber\`, \`changedFileNumber\`, \`matchThisFile\`, \`matchUntilNow\`), and \`text\`, \`find\`, \`pipe\`, \`cwd\`, \`time\` and \`timeObj\`. \`rr.stdin\` is the piped data, and \`rr.print()\` and \`rr.printErr()\` print to stdout and stderr, also with --quiet-total. \`pad(value, width = 3, char = '0')\` pads a value, like \`pad($1)\` for \`007\`. A named group overwrites a name it shares, also in \`rr\`. + All variables, except from module, date objects, \`nl\` and \`_\`, has a corresponding variable name followed by \`_\` where the content has an extra space at the end (for easy concatenation). ` ) diff --git a/src/engine.ts b/src/engine.ts index 5868e1da..3ab5eadc 100644 --- a/src/engine.ts +++ b/src/engine.ts @@ -24,15 +24,14 @@ const re = { // With a name the location is in parentheses at the end, like `at fn (/path/file.js:2:15)`. stackLineWithoutName: /^\s*at (?:file:\/\/)?((?:\/|[A-Za-z]:[\\/]).*?)(:\d+:\d+)\s*$/, stackLineWithName: /^\s*at .*? \((?:file:\/\/)?(.+?)(:\d+:\d+)\)\s*$/, - // Subroutines of Regex+, which the rr engine only looks for when the pattern has them subroutineSyntax: /\\g<|\(\?\(DEFINE\)/, - // The flags only Regex+ knows: free spacing (x) and only named groups capture (n) + // The code uses the rr object, like rr.year or rr[1], so it can reach a group in a way a search can not see + usesRr: /(^|[^\p{ID_Continue}$])rr[.[]/u, + // v is a native flag too, but only Regex+ knows x and n regexPlusOnlyFlags: /[xn]/, - // Every flag of a regex, and x and n of Regex+ regexFlagNames: /[dgimsuvyxn]/g, // The flags Regex+ takes as options, and not as flags regexPlusFlags: /[uvxn]/g, - // A $ in a replacement: $$, $&, $`, $', $, or a group number of one or two digits replacementToken: /\$(\$|&|`|'|<[^>]*>|(\d)(\d?))/g, // Only the chars with a meaning outside a character class. Unicode mode does not allow to escape the others (#686). regexSpecialChars: /[\[\]{}()*+?.\/\\^$|]/g, @@ -197,7 +196,7 @@ function unpackConfig(config) { ); } - // The groups Regex+ adds for atomic groups and possessive quantifiers are not counted in the numbers of the replacement + // The user does not know the groups Regex+ adds for atomic groups and possessive quantifiers if ( hiddenCaptures.length && 'string' === typeof config.replacement && @@ -737,12 +736,10 @@ function codeReplacement(file, config, text, numbers) { }; } -// -E: default is another name for rr, and V for regex+ const engineNames = {default: 'rr', V: 'regex+'}; const engineName = name => engineNames[name] ?? name ?? 'rr'; -// The flags of the regex: --flags as they are, or the default flags with -G, -I, -M, -s and -u. -// regex+ also has v, free spacing (x) and only named groups capture (n) by default, and v makes -u needless. +// -u is needless with regex+, as it has v function regexFlags(config, regexPlus) { if (null != config.flags) { // The runtime does not always say which flag it does not know, so it is told here @@ -761,8 +758,6 @@ function regexFlags(config, regexPlus) { ); } -// The regex, the numbers of the groups the engine added to it, which the user does not know about, -// and if only named groups capture function compileRegex(pattern, config) { const engine = engineName(config.engine); if (!['rr', 'regex+', 'V8', 'JSC', 'runtime'].includes(engine)) { @@ -778,7 +773,7 @@ function compileRegex(pattern, config) { if (!regexPlus && re.regexPlusOnlyFlags.test(flags)) { panic('The flags x and n need -E regex+', flags); } - // Regex+ takes v, u, x and n as options. It uses v, unless --flags has u, or the runtime has no v. + // Regex+ takes v, u, x and n as options, and gives an error when they come as flags const nativeFlags = regexPlus ? flags.replace(re.regexPlusFlags, '') : flags; const unicode = !flags.includes('u') && supportsFlagV ? 'v' : 'u'; try { @@ -808,8 +803,7 @@ function compileRegex(pattern, config) { } } -// The rr engine: the native engine with the flags of today, and the extended syntax of Regex+ on top. -// Only the transforms of Regex+ run, in its own order, and not its flags, so a pattern that works natively gives the same regex. +// Only the transforms of Regex+ run, in its own order, and not its flags, so a pattern that works natively gives the same regex function compileRr(pattern, flags) { let hiddenCaptures = []; const transform = plugin => { @@ -826,9 +820,7 @@ function compileRr(pattern, flags) { return {regex: new RegExp(pattern, flags), hiddenCaptures}; } -// The group numbers in a replacement, like $1, count only the groups of the user, so they are moved past the hidden ones. -// It follows the rules of javascript for $n and $nn with the visible groups, and always gives two digits, -// so a digit after it is not read as part of the number. A number that is no visible group stays as text. +// Always two digits, so a digit after it is not read as part of the number. A number that is no group of the user stays text, as in javascript. function visibleGroupNumbers(replacement, hiddenCaptures, total) { const real = [0]; for (let group = 1; group <= total; group++) { @@ -880,19 +872,21 @@ function countFile(config, text, result) { function codeUses(code, regex) { const named = name => new RegExp('\\$' + name + '(?![A-Za-z])').test(code); const groups = namedGroups(regex); - // A named group is used as $year, or as $year_ with a space at the end const usesGroup = name => new RegExp( '(^|[^\\p{ID_Continue}$])\\$' + name.replace(/\$/g, '\\$') + '_?(?![\\p{ID_Continue}$])', 'u' ).test(code); const matchNumbers = named('m') || named('M'); + const rr = re.usesRr.test(code); return { groups, + // Making rr costs time at each match, so it is only made when the code uses it + rr, // $i, $c, $m and $M count what came before, so the files are done in order countsBefore: matchNumbers || named('i') || named('c'), - // The groups, the named groups, $m and $M are about the match, so the code runs once per match - perMatch: matchNumbers || re.capturedGroupRef.test(code) || groups.some(usesGroup), + // The groups, the named groups, $m, $M and rr are about the match, so the code runs once per match + perMatch: matchNumbers || re.capturedGroupRef.test(code) || groups.some(usesGroup) || rr, stat: re.mctime.test(code), size: re.byteOrSize.test(code), }; @@ -919,6 +913,8 @@ const numberNames = ['n', 't', 'i', 'c', 'm', 'M'].flatMap(name => ]) ); +const perFileNumbers = (numberNames.length / 6) * 4; + // The values in the order of numberNames, from {n, t, i, c, m, M} function numberValues(numbers) { return Object.values(numbers).flatMap(value => { @@ -1021,6 +1017,8 @@ function dynamicReplacement(file, config, text, numbers = {n: 1, t: 1, i: 1, c: text, file: '❌', file_rel: '❌', + file_path: '❌', + file_path_rel: '❌', dirpath: '❌', dirpath_rel: '❌', dirname: '❌', @@ -1044,6 +1042,8 @@ function dynamicReplacement(file, config, text, numbers = {n: 1, t: 1, i: 1, c: Object.assign(values, { file: full, file_rel: path.relative(cwd, full), + file_path: full, + file_path_rel: path.relative(cwd, full), dirpath: dir, dirpath_rel: path.relative(cwd, dir), dirname: (full.match(re.folderName) || ' _')[1], @@ -1090,6 +1090,7 @@ function dynamicReplacement(file, config, text, numbers = {n: 1, t: 1, i: 1, c: ctime_obj, nl: '\n', _: ' ', + pad: (value, width = 3, char = '0') => String(value).padStart(width, char), }; for (const [name, value] of Object.entries(values)) { params[name] = value; @@ -1097,22 +1098,87 @@ function dynamicReplacement(file, config, text, numbers = {n: 1, t: 1, i: 1, c: } codeFunction ??= new Function( ...Object.keys(params), + 'rr', ...numberNames, '__code_rr', 'return eval(__code_rr);' ); const args = Object.values(params); + // The values of RexReplace with long names, so code in a .js file can be read without the short ones + const fileInfo = { + filePath: values.file, + filePathRel: values.file_rel, + dirPath: values.dirpath, + dirPathRel: values.dirpath_rel, + dirName: values.dirname, + fileName: values.filename, + name: values.name, + ext: values.ext, + bytes: values.bytes, + size: values.size, + mtime: values.mtime, + ctime: values.ctime, + mtimeObj: mtime_obj, + ctimeObj: ctime_obj, + }; + const own = {text, find: values.find, pipe: values.pipe, cwd, time, timeObj: config.startTime}; + + // A new object for each match costs time, so the values of the file are made once, and each match builds on them. + // The groups come last, so a group overwrites a name it shares, also _. + let rrOfFile = null; + const rrObject = (allNumbers, shortNumbers, groups, named) => { + if (!rrOfFile) { + rrOfFile = { + stdin: values.pipe, + // The code of the user asks for it, so it is printed also with --quiet-total + print: value => print(String(value), null, true, process.stdout), + printErr: value => print(String(value), null, true, process.stderr), + }; + // $n, $t, $i and $c are the same in the whole file + for (let i = 0; i < perFileNumbers; i++) { + rrOfFile[numberNames[i].slice(1)] = shortNumbers[i]; + } + } + const object = Object.create(rrOfFile); + for (let i = perFileNumbers; i < numberNames.length; i++) { + object[numberNames[i].slice(1)] = shortNumbers[i]; + } + object._ = { + ...own, + fileInfo, + count: { + fileNumber: allNumbers.n, + filesTotal: allNumbers.t, + matchedFileNumber: allNumbers.i, + changedFileNumber: allNumbers.c, + matchThisFile: allNumbers.m, + matchUntilNow: allNumbers.M, + }, + }; + groups.forEach((value, i) => { + object[i] = value; + }); + for (const [name, value] of Object.entries(named)) { + object[name] = value; + object[name + '_'] = value + ' '; + } + return object; + }; + // The code of the user is in a block of its own, so it can declare the same names with let and const. // The value of the block is the value of its last statement, like before. The line break ends a comment at the end. const code = '{' + config.replacementOri + '\n}'; // When the code fails, only the message is told, and not the stack trace with all the code put in front of the code of the user - const runCode = (matchNumbers, variables) => { + const runCode = (matchNumbers, variables, groups = [], named = {}) => { + const allNumbers = {...numbers, ...matchNumbers}; + const shortNumbers = numberValues(allNumbers); try { return codeFunction( ...args, - ...numberValues({...numbers, ...matchNumbers}), + config.codeUses.rr ? rrObject(allNumbers, shortNumbers, groups, named) : undefined, + ...shortNumbers, variables + code ); } catch (e) { @@ -1137,9 +1203,12 @@ function dynamicReplacement(file, config, text, numbers = {n: 1, t: 1, i: 1, c: // The value is given back from a function, so `$&` and `$$` in it are inserted as they are and not expanded if (!config.codeUses.perMatch) { - // The code runs once for the file, so it is the first match - const result = runCode({m: 1, M: config.numbers.matches + 1}, ''); + // At the first match, so a file without a match does not run the code + let result; return () => { + if (!found) { + result = runCode({m: 1, M: config.numbers.matches + 1}, ''); + } foundMatch(); return result; }; @@ -1152,17 +1221,15 @@ function dynamicReplacement(file, config, text, numbers = {n: 1, t: 1, i: 1, c: const {groups, named} = splitMatch(matchArgs, config.hiddenCaptures); let variables = ''; // $0 is the match. With the flag n the groups have no numbers, only names. - groups.forEach((value, i) => { - if (!i || !config.namedOnly) { - variables += 'var $' + i + '=' + JSON.stringify(value) + '; '; - } + const numbered = config.namedOnly ? groups.slice(0, 1) : groups; + numbered.forEach((value, i) => { + variables += 'var $' + i + '=' + JSON.stringify(value) + '; '; }); - // A named group is $ and the name, like $year for (?…), and $year_ has a space at the end. - // It comes after the numbers, so a group named like a number, like n, wins over $n. + // After the numbers, so a group named like a number, like n, wins over $n for (const [name, value] of Object.entries(named)) { variables += `var $${name} = ${JSON.stringify(value)}, $${name}_ = $${name} + ' '; `; } - return runCode({m: match, M: config.numbers.matches}, variables); + return runCode({m: match, M: config.numbers.matches}, variables, numbered, named); }; } diff --git a/test/fix/856.test.js b/test/fix/856.test.js new file mode 100644 index 00000000..164aee52 --- /dev/null +++ b/test/fix/856.test.js @@ -0,0 +1,123 @@ +// https://github.com/mathiasrw/rexreplace/issues/856 +// --js code gets an rr object behind the short names, the values of RexReplace with long names in rr._, +// rr.stdin, rr.print() and rr.printErr(), pad(), and file_path with file_path_rel +import {describe, test, expect} from 'bun:test'; +import fs from 'fs'; +import path from 'path'; +import {rr} from '../helper.js'; + +const js = (code, config = {}) => + rr('(?\\d)(\\d)', code, {pipedData: '12', replacementJs: true, ...config}); + +describe('#856 the rr object', () => { + test.each([ + ['rr.y for $y', 'rr.y + $y', '11'], + ['rr[1] for $1, and rr[0] for $0', 'rr[1] + rr[2] + rr[0]', '1212'], + ['rr.y_ for $y_', 'rr.y_ + "|"', '1 |'], + ['rr.n and its cousins for $n', 'rr.n + rr.n00 + rr.n_', '10011 '], + ['rr.m and rr.M for $m and $M', 'rr.m + "/" + rr.M', '1/1'], + ])('has %s', async (name, code, out) => { + expect((await js(code)).out).toBe(out); + }); + + test('has the groups as text and the numbers as numbers', async () => { + expect((await js('typeof rr.y + typeof rr.n')).out).toBe('stringnumber'); + }); + + test('makes the code run once per match', async () => { + const config = {pipedData: 'a1b2', replacementJs: true}; + expect((await rr('\\d', 'rr.m', config)).out).toBe('a1b2'); + }); + + test('is not found inside another name, like brr.n', async () => { + const config = {pipedData: 'a1b2', replacementJs: true}; + expect((await rr('\\d', 'const brr = {n: 7}; brr.n', config)).out).toBe('a7b7'); + }); + + test('is overwritten by a named group of the same name, also _', async () => { + const config = {pipedData: 'ab', replacementJs: true}; + expect((await rr('(?a)', 'rr.n + $n', config)).out).toBe('aab'); + expect((await rr('(?<_>a)', 'rr._', config)).out).toBe('ab'); + }); +}); + +describe('#856 rr._', () => { + test('has the values of the file with long names', async () => { + fs.writeFileSync('my.file.txt', 'foo'); + const code = + '[rr._.fileInfo.fileName, rr._.fileInfo.name, rr._.fileInfo.ext, rr._.fileInfo.filePathRel, rr._.fileInfo.dirName].join("|")'; + await rr('foo', code, {includeGlob: ['my.file.txt'], replacementJs: true}); + const dir = path.basename(process.cwd()); + expect(fs.readFileSync('my.file.txt', 'utf8')).toBe( + `my.file.txt|my.file|.txt|my.file.txt|${dir}` + ); + }); + + test('has the counts with long names', async () => { + ['a.txt', 'b.txt', 'c.txt'].forEach(name => + fs.writeFileSync(name, 'b.txt' === name ? 'x' : 'oo') + ); + const code = 'Object.values(rr._.count).join("")'; + await rr('o', code, {includeGlob: ['*.txt'], replacementJs: true}); + // fileNumber, filesTotal, matchedFileNumber, changedFileNumber, matchThisFile, matchUntilNow + expect(['a.txt', 'b.txt', 'c.txt'].map(name => fs.readFileSync(name, 'utf8'))).toEqual([ + '131111131122', + 'x', + '332213332224', + ]); + }); + + test('has text, find and time', async () => { + expect((await js('rr._.text + rr._.find + typeof rr._.time')).out).toBe( + '12(?\\d)(\\d)string' + ); + }); +}); + +describe('#856 printing from the code', () => { + test('rr.print goes to stdout with a line break, also with --quiet-total', async () => { + const result = await js('rr.print("hi"); "x"', {quietTotal: true, output: true}); + expect(result.out).toStartWith('hi\n'); + }); + + test('rr.printErr goes to stderr', async () => { + expect((await js('rr.printErr("oops"); "x"')).err).toBe('oops\n'); + }); + + test('rr.stdin is the piped data', async () => { + fs.writeFileSync('a.txt', 'o'); + const config = {includeGlob: ['a.txt'], replacementJs: true, pipe: true, pipedData: 'P'}; + await rr('o', 'rr.stdin', config); + expect(fs.readFileSync('a.txt', 'utf8')).toBe('P'); + }); +}); + +describe('#856 pad()', () => { + test.each([ + ['pad($1)', '001'], + ['pad($2, 5)', '00002'], + ['pad($y, 2, " ")', ' 1'], + ])('%s', async (code, out) => { + expect((await js(code)).out).toBe(out); + }); +}); + +describe('#856 file_path and file_path_rel', () => { + test('are the same as file and file_rel', async () => { + fs.writeFileSync('a.txt', 'o'); + await rr('o', '[file_path === file, file_path_rel, file_path_rel_].join("|")', { + includeGlob: ['a.txt'], + replacementJs: true, + }); + expect(fs.readFileSync('a.txt', 'utf8')).toBe('true|a.txt|a.txt '); + }); +}); + +describe('#856 code that runs once per file', () => { + test('only runs for a file with a match', async () => { + ['a.txt', 'b.txt'].forEach(name => fs.writeFileSync(name, 'a.txt' === name ? 'foo' : 'bar')); + const code = 'fs.appendFileSync("log", name); "x"'; + await rr('foo', code, {includeGlob: ['*.txt'], replacementJs: true}); + expect(fs.readFileSync('log', 'utf8')).toBe('a'); + }); +});