hledger-flow should exit with a non-zero error code if a journal construct invocation results in ExitFailure

[lestephane]

Background
I have PDF statements from my bank which i convert using a construct script (combination of pdftotext and awk script).

During the hledger-flow import invocation, I see some error, and indeed the corresponding journal is never created.

Collecting input files...
Found 23 input files in 0.134708928s. Proceeding with import...
...
hledger: -:161:1:
    |
161 | 2018-02-30 "Abschluss lt. Anlage 1 PN:905"
    | ^^^^^^^^^^
well-formed but invalid date: 2018-2-30

2019-04-15 17:04:58.505274683 EEST	hledger-flow End:   executing 'girokonto/construct' on 'girokonto/1-in/2018/Kontoauszug_vom_28.02.2018_05.03.2018.pdf' ExitFailure 1 (0.082273261s)
...
Imported 23 journals in 2.203744436s

the problem is twofold:

• there is now a YYYY-include.journal file that contains the !include of a file that does not exist on the filesystem in 3-journal
• the importcommand did not return a non-zero exit code (despite the construct ExitFailure 1). So, one will only see a problem later (potentially days later), when loading the files with hledger. By then the user won't know what made his file disappear.

$ hledger-ui -f all-years.journal 
...
in file included from /Finance/import/personal/bbbank/2018-include.journal,
/Finance/import/personal/bbbank/girokonto/2018-include.journal:4:97:
  |
4 | !include 3-journal/2018/Kontoauszug_vom_28.02.2018_05.03.2018.journal
  |                                                                                                 ^
No existing files match pattern: 3-journal/2018/Kontoauszug_vom_28.02.2018_05.03.2018.journal

[lestephane]

Another example of ExitFailure that did not cause hledger-flow to exit with a non-zero code (a parsing error in a .rules file)

hledger: user error (/Finance/import/personal/n26/checking/n26-checking.rules:23:1:
   |
23 | else
   | ^
unexpected 'e'
expecting blank or comment line, conditional block, directive, end of input, field assignment, or field name list
)
...
Imported 24 journals in 0.9541295s
$ echo $?
0

[apauley]

Nice catch! I'll look at this as soon as I can make some time.

[apauley]

I've made a change that I think should fix the issue, but I have only tested it briefly.

There should be a pre-release uploaded soon, if the build succeeds:
https://github.com/apauley/hledger-flow/releases

Otherwise you can build this using branch exit-on-err

[apauley]

This should be fix now, please let me know if you see any other issues

[lestephane]

• I get a non zero exit code when a .rules file has a syntax error (stray 'else', invalid date)
• I get a non zero exit code when a construct script exits with a non-zero exit code.

As an optional usability improvement bonus, I would suggest adding a final one-line log statement like

command `<insert command line that failed here>` exited with code `<insert code here>`, aborting...

Just before hledger-flow exits.

[apauley]

@lestephane The error output in the latest release now looks something like this:

=== Begin Error: importing 'import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-05-28.csv' using rules file 'import/gawie/bogart/cheque/bogart-cheque.rules' ===
External command:
/usr/bin/hledger print --rules-file import/gawie/bogart/cheque/bogart-cheque.rules --file import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-05-28.csv --output-file import/gawie/bogart/cheque/3-journal/2016/123456789_2016-05-28.journal
Exit code 1
Error output:
hledger: error: could not parse "'29 Apr'" as a date using date format "YYYY/M/D", "YYYY-M-D" or "YYYY.M.D"
the CSV record is:  "5", "1", "'29 Apr'", "#Monthly Bank Fee", "", "", "-599.06", "37394.41", ""
the date rule is:   %3
the date-format is: unspecified
you may need to change your date rule, add a date-format rule, or change your skip rule
for m/d/y or d/m/y dates, use date-format %-m/%-d/%Y or date-format %-d/%-m/%Y


=== End Error: importing 'import/gawie/bogart/cheque/2-preprocessed/2016/123456789_2016-05-28.csv' using rules file 'import/gawie/bogart/cheque/bogart-cheque.rules' ===

Let me know if you can think of more usability improvements.