------------------------------------------------------------------- Mon Jun 05 15:07:25 UTC 2023 - scott.bradnick@suse.com - Update to version 6.8.0+git20230604.4050f566: * fix mis-spelling for head docs * Update readthedocs notes in the how-to-release page (#1308) * release 6.8.0 docs * post-6.8.0 * Update go version in .github/workflows/release.yml * go mod tidy * miller 6.8.0 * Unify the `case` verb, and add options (#1306) * Values-only `-a` option for `mlr grep` (#1305) * Bump github/codeql-action from 2.3.5 to 2.3.6 (#1303) * Bump github.com/stretchr/testify from 1.8.3 to 1.8.4 (#1301) * Bump github/codeql-action from 2.3.4 to 2.3.5 (#1300) * Bump github/codeql-action from 2.3.3 to 2.3.4 (#1299) * Bump github.com/mattn/go-isatty from 0.0.18 to 0.0.19 (#1296) * Bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (#1295) * Bump actions/setup-go from 4.0.0 to 4.0.1 (#1294) * Let `mlr help` take pre-flags, such as `--always-color` (#1292) * New `mlr json-parse -k` flag (#1291) * Bump golang.org/x/term from 0.7.0 to 0.8.0 (#1285) * Bump github/codeql-action from 2.3.2 to 2.3.3 (#1284) ------------------------------------------------------------------- Wed May 03 15:13:01 UTC 2023 - scott.bradnick@suse.com - Update to version 6.7.0+git20230501.6c86ef19: * Bump codespell-project/actions-codespell (#1282) * Bump github/codeql-action from 2.3.1 to 2.3.2 (#1279) * Bump github/codeql-action from 2.3.0 to 2.3.1 (#1277) * Bump github/codeql-action from 2.2.12 to 2.3.0 (#1274) * Implement `--csv-trim-leading-space` flag (#1272) * Fix precedence of coalesce operators `??` and `???` (#1270) * Treaet data-file numbers with leading + as numeric (#1269) * Bump github/codeql-action from 2.2.11 to 2.2.12 (#1265) * Bump actions/checkout from 3.5.1 to 3.5.2 (#1264) * Bump actions/checkout from 3.5.0 to 3.5.1 (#1263) * Bump github/codeql-action from 2.2.10 to 2.2.11 (#1261) * Bump github/codeql-action from 2.2.9 to 2.2.10 (#1259) * Bump golang.org/x/term from 0.6.0 to 0.7.0 (#1256) * Fix typo (#1252) * Bump codespell-project/actions-codespell (#1253) * Better error message on unparseable TZ environment variable (#1249) ------------------------------------------------------------------- Wed Mar 29 20:26:19 UTC 2023 - scott.bradnick@suse.com - Update to version 6.7.0+git20230328.dc14420f: * Bump github/codeql-action from 2.2.8 to 2.2.9 (#1250) * Fix issue links in README-dev.md (#1248) * `index` DSL function [WIP] (#1247) * Bump github.com/mattn/go-isatty from 0.0.17 to 0.0.18 (#1243) * Bump golang.org/x/term from 0.0.0-20210927222741-03fcf44c2211 to 0.6.0 (#1222) * Bump minimum compiler version from Go 1.15 to 1.18 (#1246) * Bump actions/checkout from 3.4.0 to 3.5.0 (#1245) * Add optional second base argument to `int` DSL function (#1244) * Bump github/codeql-action from 2.2.7 to 2.2.8 (#1242) * Bump actions/checkout from 3.3.0 to 3.4.0 (#1234) * Bump actions/setup-go from 3.5.0 to 4.0.0 (#1233) * Bump github/codeql-action from 2.2.6 to 2.2.7 (#1232) * Bump actions/cache from 3.2.6 to 3.3.1 (#1229) * Bump github/codeql-action from 2.2.5 to 2.2.6 (#1230) * Add `mlr step -a rprod` for running products (#1228) * Label head docs as "Miller Dev Documentation" * doc note for #1218 * Run go mod tidy (#1220) * Include tools in the release tarball (#1221) * Add new `upcase` and `downcase` verbs (#1217) * Docs re `tail -f` and `--records-per-batch 1` (#1218) * typofix * Fix bug on DSL comment with no final newline (#1216) * release docs * build artifacts * miller 6.7.0-dev * miller 6.7.0 * Support more backslashed special characters in DSL strings (#1212) * unit-test files * fix build error * Unit-test updates * tools/build-dsl artifacts * Update DSL-build instructions * go.mod go.sum * BNF mod for `\^` `\$` et al. * --ofmte, --ofmtf, --ofmtg (#1206) * Bump github/codeql-action from 2.2.4 to 2.2.5 (#1207) * Bump github.com/stretchr/testify from 1.8.1 to 1.8.2 (#1208) * New `leftpad` and `rightpad` DSL functions (#1205) * Bump golang.org/x/sys in /cmd/experiments/cli_parser (#1203) * Bump actions/cache from 3.2.5 to 3.2.6 (#1196) * Bump github/codeql-action from 2.2.3 to 2.2.4 (#1193) * Bump github/codeql-action from 2.2.1 to 2.2.3 (#1191) * Bump actions/cache from 3.2.4 to 3.2.5 (#1192) * Bump actions/cache from 3.2.3 to 3.2.4 (#1186) * Bump goreleaser/goreleaser-action from 4.1.1 to 4.2.0 (#1187) * Bump goreleaser/goreleaser-action from 4.1.0 to 4.1.1 (#1185) * Complete #1181 (#1184) * Bump github/codeql-action from 2.1.39 to 2.2.1 (#1183) * Fixed missing double quote in documentation sample (#1181) * Bump github/codeql-action from 2.1.38 to 2.1.39 (#1179) * Bump github/codeql-action from 2.1.37 to 2.1.38 (#1176) * Bump actions/upload-artifact from 3.1.1 to 3.1.2 (#1175) * Bump actions/cache from 3.2.2 to 3.2.3 (#1174) * Bump actions/checkout from 3.2.0 to 3.3.0 (#1173) * Bump codespell-project/actions-codespell (#1172) * Add Go LICENSE file (#1171) ------------------------------------------------------------------- Mon Jan 02 04:44:33 UTC 2023 - scott.bradnick@suse.com - Update to version 6.6.0+git20230101.7495a8845: * link to 6.6.0 docs * doc-neaten * post-6.6.0 * 6.6.0 (#1169) * Fix #1164: regression on CSV blank-line handling (#1168) * mlr unspace verb (#1167) * Add doc info on DSL code-comment syntax (#1165) * Fix #1102 (#1163) * Fix #1146 (#1157) * Bump github.com/mattn/go-isatty from 0.0.16 to 0.0.17 (#1162) * Bump actions/cache from 3.2.1 to 3.2.2 (#1160) * Bump actions/cache from 3.2.0 to 3.2.1 (#1156) * Bump actions/cache from 3.0.11 to 3.2.0 (#1155) * Bump github/codeql-action from 2.1.36 to 2.1.37 (#1151) * Bump actions/setup-go from 3.4.0 to 3.5.0 (#1148) * Bump goreleaser/goreleaser-action from 3.2.0 to 4.1.0 (#1147) * Bump actions/checkout from 3.1.0 to 3.2.0 (#1145) * Bump github/codeql-action from 2.1.35 to 2.1.36 (#1143) * Fix typo in `mlr put` documentation (#1140) * Bump actions/setup-go from 3.3.1 to 3.4.0 (#1136) * Bump github/codeql-action from 2.1.33 to 2.1.35 (#1137) ------------------------------------------------------------------- Wed Nov 30 17:33:12 UTC 2022 - scott.bradnick@suse.com - Update to version 6.5.0+git20221130.636ca8873: * Fix typos (#1135) * Release notes * Post-6.5.0 * Miller 6.5.0 (#1134) * Miller 6.5.0 * Convert mlrval polymorphism from struct to unionish interface (#1133) * Clarify locations of performance info * More mlrval size-reduction (#1132) * More mlrval size-reduction * Exclude median from summary default (#1131) * Exclude median from summary default * Bump github.com/stretchr/testify from 1.8.0 to 1.8.1 (#1113) * Bump actions/upload-artifact from 3.1.0 to 3.1.1 (#1112) * Bump actions/cache from 3 to 3.0.11 (#1109) * Bump github/codeql-action from 2.1.28 to 2.1.33 (#1126) * Use int8 for mvtype (memory reduction) (#1130) * make fmt * forgot to set between-releases version to 6.4.0-dev after 6.4.0 * Bump github.com/pkg/profile from 1.6.0 to 1.7.0 (#1110) * [StepSecurity] ci: Harden GitHub Actions (#1107) * fix an old typo ------------------------------------------------------------------- Thu Oct 13 23:36:39 UTC 2022 - scott.bradnick@suse.com - Renaming buildmode-pie.patch to buildmode-pie.diff - Update to version 6.4.0+git20221003.39286923d: * Don't double-quote a CSV field only for having a leading space (#1101) ------------------------------------------------------------------- Sun Oct 02 16:57:42 UTC 2022 - scott.bradnick@suse.com - Update to version 6.4.0+git20220929.36d0ed760: * type-safety in exec (#1099) * add source-checkout info to build-from-source instructions * Check -- terminator on --mfrom (#1098) * todo * mlr reshape multiple regex via multiple -r (#1091) * Fix labels for mlr histogram --auto (#1089) * Continue #1086 (#1087) ------------------------------------------------------------------- Fri Sep 2 21:29:11 UTC 2022 - Scott Bradnick - Patch buildmode-pie.patch was added per go's request during build. ------------------------------------------------------------------- Fri Aug 26 18:47:10 UTC 2022 - Scott Bradnick - Excluding patch from ppc64: "-buildmode=pie not supported on linux/ppc64" ------------------------------------------------------------------- Fri Aug 26 16:03:07 UTC 2022 - Scott Bradnick - Adding Group definition - Removing unnecessary 'golang' requirement which isn't present in 15.3 ------------------------------------------------------------------- Tue Aug 23 20:43:11 UTC 2022 - scott.bradnick@suse.com - Update to version v6.4.0+git20220823.cdbe8b82e: * Account for varying mlr locations (#1086) * Restore --quote-all for CSV output (#1084) * Restore the --jvquoteall flag (#1083) * forgot to make a branch/PR for --quote-all * make dev: for doc-build artifacts * unit-test case * Restore --quote-all flag for CSV output * post-6.4.0 * 6.4.0 release (#1081) * codespell ------------------------------------------------------------------- Wed Nov 17 06:58:21 UTC 2021 - Luigi Baldoni - Update to version 5.10.3 * autoreconf.fiv; manpage & docs w/ 5.10.3 * Replace NDEBUG with MLR_DSL_NDEBUG ifdef to avoid name clash with conda build * precompute lemon artifacts ------------------------------------------------------------------- Wed Mar 24 11:40:14 UTC 2021 - Luigi Baldoni - Update to version 5.10.2 Fix: * Restore mlr manpage to distro file ------------------------------------------------------------------- Mon Mar 22 08:20:46 UTC 2021 - Luigi Baldoni - Update to version 5.10.1 Fix: * NR is stale after apply remove-empty-columns * Segfault with --tsv --implicit-csv-header * Default value of uninitialized variables for multiplication: docs vs awk * Bug with uniq + sort ------------------------------------------------------------------- Mon Nov 30 11:35:45 UTC 2020 - aloisio@gmx.com - Update to version 5.10.0 Features: * The unsparsify -f feature fulfills #387 from @sjackman . * The new sort-within-records verb is an old ask, underway from the Go port, backported to C. * Likewise the truncate DSL function. Bugfixes: * The count -n feature was not implemented as intended. * Pretty-print format now works correctly with --headerless-csv-output. * The seqgen verb now correctly tracks NR and FNR in the records it emits. * An intermittent JSON-parsing bug has been fixed. ------------------------------------------------------------------- Thu Sep 3 09:50:47 UTC 2020 - Luigi Baldoni - Update to version 5.9.1 * Disallow prepipe within .mlrrc (fixes CVE-2020-15167) ------------------------------------------------------------------- Wed Aug 19 20:26:12 UTC 2020 - aloisio@gmx.com - Update to version 5.9.0 * You can now save common defaults in a ~/.mlrrc. For example, if you normally process CSV files, you can say that in your ~/.mlrrc and you can leave off the --csv flag from your mlr commands. ------------------------------------------------------------------- Tue Aug 4 06:57:04 UTC 2020 - aloisio@gmx.com - Update to version 5.8.0 Features: * The new count verb is a keystroke-saver for stats -a count -f {some field name}`. * --jsonx and --ojsonx are keystroke-savers for --json --jvstack and --ojson --jvstack, which is to say, multi-line pretty-printed JSON format. * The new -s name=value feature for mlr put and mlr filter gives you simpler access to environment variables in your Miller script, as requested in #315. Bugfixes: * mlr format-values is no longer SEGVing on CSV/TSV input. This was reported on #330. * #313 fixes a corner case when field names within command-line arguments have embedded newlines. * Line/column indicators for JSON-formatting error messages are now correct (previously they were showing up as 0). * end {print NF} no longer SEGVs. This was reported in #330. * Several broken doc links were fixed up as reported on #329. - Drop miller-5.3.0-gcc43.patch (no longer necessary) - Spec cleanup ------------------------------------------------------------------- Tue Mar 17 06:56:30 UTC 2020 - aloisio@gmx.com - Update to version 5.7.0 Features: * The new remove-empty-columns and skip-trivial-records are keystroke-savers for things which would other require DSL syntax, as tracked in #274. Bugfixes: * A bug regarding optional regex-pattern groups was fixed in #277. * As of #294 you can now specify --implicit-csv-header for the join-file in mlr join. * A bug with spaces in XTAB-file values was fixed on #296. * A bug with missing final newline for XTAB-formatted files using MMAP files was fixed on #301. - Drop group tag ------------------------------------------------------------------- Sun Sep 22 04:53:59 UTC 2019 - Luigi Baldoni - Update to version 5.6.2 * #271 fixes a corner-case bug with more than 100 CSV/TSV files with headers of varying lengths. ------------------------------------------------------------------- Fri Sep 13 05:54:23 UTC 2019 - Luigi Baldoni - Update to version 5.6.0 Features: * The new system DSL function allows you to run arbitrary shell commands and store them in field values. Some example usages are documented here. This is in response to issues #246 and #209. * There is now support for ASV and USV file formats. This is in response to issue #245. * The new format-values verb allows you to apply numerical formatting across all record values. This is in response to issue #252. Documentation: * The new DKVP I/O in Python sample code now works for Python 2 as well as Python 3. * There is a new cookbook entry on doing multiple joins. This is in response to issue #235. Bugfixes: * The toupper, tolower, and capitalize DSL functions are now UTF-8 aware, thanks to @sheredom's marvelous https://github.com/sheredom/utf8.h. The internationalization page has also been expanded. This is in response to issue #254. * #250 fixes a bug using in-place mode in conjunction with verbs (such as rename or sort) which take field-name lists as arguments. * #253 fixes a bug in the label when one or more names are common between old and new. * #251 fixes a corner-case bug when (a) input is CSV; (b) the last field ends with a comma and no newline; (c) input is from standard input and/or --no-mmap is supplied. ------------------------------------------------------------------- Sun Sep 1 06:34:42 UTC 2019 - Luigi Baldoni - Update to version 5.5.0 * Positional indexing and other data-cleaning features Features: * The new positional-indexing feature resolves #236 from @aborruso. You can now get the name of the 3rd field of each record via $[[3]], and its value by $[[[3]]]. These are both usable on either the left-hand or right-hand side of assignment statements, so you can more easily do things like renaming fields progrmatically within the DSL. * There is a new capitalize DSL function, complementing the already-existing toupper. This stems from #236. * There is a new skip-trivial-records verb, resolving #197. Similarly, there is a new remove-empty-columns verb, resolving #206. Both are useful for data-cleaning use-cases. * Another pair is #181 and #256. While Miller uses mmap internally (and invisibily) to get approximately a 20% performance boost over not using it, this can cause out-of-memory issues with reading either large files, or too many small ones. Now, Miller automatically avoids mmap in these cases. You can still use --mmap or --no-mmap if you want manual control of this. * There is a new --ivar option for the nest verb which complements the already-existing --evar. This is from #260 thanks to @jgreely. * There is a new keystroke-saving urandrange DSL function: urandrange(low, high) is the same as low + (high - low) * urand(). * There is a new -v option for the cat verb which writes a low-level record-structure dump to standard error. * There is a new -N option for mlr which is a keystroke-saver for --implicit-csv-header --headerless-csv-output. Documentation: * The new FAQ entry http://johnkerl.org/miller/doc/faq.html#How_to_escape_'%3F'_in_re gexes%3F resolves #203. * The new FAQ entry http://johnkerl.org/miller/doc/faq.html#How_can_I_filter_by_date% 3F resolves #208. * #244 fixes a documentation issue while highlighting the need for #241. Bugfixes: * There was a SEGV using nest within then-chains, fixed in response to #220. * Quotes and backslashes weren't being escaped in JSON output with --jvquoteall; reported on #222. ------------------------------------------------------------------- Mon Oct 15 07:23:57 UTC 2018 - Luigi Baldoni - Update to version 5.4.0 Features: * The new clean-whitespace verb resolves #190 from @aborruso. Along with the new functions strip, lstrip, rstrip, collapse_whitespace, and clean_whitespace, there is now both coarse-grained and fine-grained control over whitespace within field names and/or values. See the linked-to documentation for examples. * The new altkv verb resolves #184 which was originally opened via an email request. This supports mapping value-lists such as a,b,c,d to alternating key-value pairs such as a=b,c=d. * The new fill-down verb resolves #189 by @aborruso. See the linked-to documentation for examples. * The uniq verb now has a uniq -a which resolves #168 from @sjackman. * The new regextract and regextract_or_else functions resolve #183 by @aborruso. * The new ssub function arises from #171 by @dohse, as a simplified way to avoid escaping characters which are special to regular-expression parsers. * There are new localtime functions in response to #170 by @sitaramc. However note that as discussed on #170 these do not undo one another in all circumstances. This is a non-issue for timezones which do not do DST. Otherwise, please use with disclaimers: localdate, localtime2sec, sec2localdate, sec2localtime, strftime_local, and strptime_local. * Travis builds at https://travis-ci.org/johnkerl/miller/builds now run on OSX as well as Linux. * An Ubuntu 17 build issue was fixed by @singalen on #164. Documentation: * put/filter documentation was confusing as reported by @NikosAlexandris on #169. * The new FAQ entry http://johnkerl.org/miller-releases/miller-head/doc/faq.html#How_ to_rectangularize_after_joins_with_unpaired? resolves #193 by @aborruso. * The new cookbook entry http://johnkerl.org/miller/doc/cookbook.html#Options_for_dealing_ with_duplicate_rows arises from #168 from @sjackman. * The unsparsify documentation had some words missing as reported by @tst2005 on #194. * There was a typo in the cookpage page http://johnkerl.org/miller/doc/cookbook.html#Full_field_renames_a nd_reassigns as fixed by @tst2005 in #192. Bugfixes: * There was a memory leak for TSV-format files only as reported by @treynr on #181. * Dollar sign in regular expressions were not being escaped properly as reported by @dohse on #171. ------------------------------------------------------------------- Sun Jan 7 07:56:34 UTC 2018 - aloisio@gmx.com - Update to version 5.3.0 (see draft-release-notes.md for a changelog) - Added miller-5.3.0-gcc43.patch ------------------------------------------------------------------- Thu Aug 24 15:18:41 UTC 2017 - aloisio@gmx.com - Updated license ------------------------------------------------------------------- Thu Jul 20 09:29:23 UTC 2017 - aloisio@gmx.com - Update to 5.2.2 * This bugfix release delivers a fix for #147 where a memory allocation failed beyond 4GB. ------------------------------------------------------------------- Tue Jun 20 06:34:53 UTC 2017 - aloisio@gmx.com - Update to version 5.2.1 * Fixes (gh#johnkerl/miller#142) build segfault on non-x86 architectures ------------------------------------------------------------------- Tue Jun 13 08:06:28 UTC 2017 - aloisio@gmx.com - Update to version 5.2.0 This release contains mostly feature requests. Features: * The stats1 verb now lets you use regular expressions to specify which field names to compute statistics on, and/or which to group by. Full details are here. * The min and max DSL functions, and the min/max/percentile aggregators for the stats1 and merge-fields verbs, now support numeric as well as string field values. (For mixed string/numeric fields, numbers compare before strings.) This means in particular that order statistics -- min, max, and non-interpolated percentiles -- as well as mode, antimode, and count are now possible on string-only fields. (Of course, any operations requiring arithmetic on values, such as computing sums, averages, or interpolated percentiles, yield an error on string-valued input.) * There is a new DSL function mapexcept which returns a copy of the argument with specified key(s), if any, unset. The motivating use-case is to split records to multiple filenames depending on particular field value, which is omitted from the output: mlr --from f.dat put 'tee > "/tmp/data-".$a, mapexcept($*, "a")' Likewise, mapselect returns a copy of the argument with only specified key(s), if any, set. This resolves #137. * A new -u option for count-distinct allows unlashed counts for multiple field names. For example, with -f a,b and without -u, count-distinct computes counts for distinct pairs of a and b field values. With -f a,b and with -u, it computes counts for distinct a field values and counts for distinct b field values separately. * If you build from source, you can now do ./configure without first doing autoreconf -fiv. This resolves #131. * The UTF-8 BOM sequence 0xef 0xbb 0xbf is now automatically ignored from the start of CSV files. (The same is already done for JSON files.) This resolves #138. * For put and filter with -S, program literals such as the 6 in $x = 6 were being parsed as strings. This is not sensible, since the -S option for put and filter is intended to suppress numeric conversion of record data, not program literals. To get string 6 one may use $x = "6". Documentation: * A new cookbook example shows how to compute differences between successive queries, e.g. to find out what changed in time-varying data when you run and rerun a SQL query. * Another new cookbook example shows how to compute interquartile ranges. * A third new cookbook example shows how to compute weighted means. Bugfixes: * CRLF line-endings were not being correctly autodetected when I/O formats were specified using --c2j et al. * Integer division by zero was causing a fatal runtime exception, rather than computing inf or nan as in the floating-point case. ------------------------------------------------------------------- Sat Apr 15 07:48:57 UTC 2017 - aloisio@gmx.com - Update to 5.1.0 (see changelog at https://github.com/johnkerl/miller/releases/tag/v5.1.0) ------------------------------------------------------------------- Sun Mar 12 21:04:27 UTC 2017 - aloisio@gmx.com - Update to version 5.0.1 Minor bugfixes: * As described in #132, mlr nest was incorrectly splitting fields with multi-character separators. * The XTAB-format reader, when using multi-character IPS, was incorrectly splitting key-value pairs, but only when reading from standard input (e.g. on a pipe or less-than redirect). ------------------------------------------------------------------- Tue Feb 28 10:28:05 UTC 2017 - aloisio@gmx.com - Initial package (v5.0.0)