A way to add custom options to the hledger print command that is run on the construct output might make sense

[lestephane]

Context (what brought about headscratching)

• Create a depreciation import directory

$ mkdir -p import/business/automation/depreciation/1-in
$ cat <<EOF > import/business/automation/depreciation/1-in/Dell-XPS-9500-20210112.journal
~ every 01-12 from 2022 to 2026
    Business-CY:Assets:Fixed-Assets:Laptop-Computers:Dell-XPS-9500-20210112     -389.4375 EUR
    Business-CY:Expenses:Depreciations
EOF
$ for year in $(seq 2021 2026); do
  mkdir -p "import/business/automation/depreciation/1-in/${year}"
  ln -srf import/business/automation/depreciation/1-in/Dell-XPS-9500-20210112.journal \
    "import/business/automation/depreciation/1-in/${year}/"
done

• create the simplest construct script

$ cat <<EOF > "import/business/automation/depreciation/1-in/construct"
#!/usr/bin/env bash

cat "$1" > "$2"
EOF
$ chmod u-x "import/business/automation/depreciation/1-in/construct"

• run hledger-flow import
• the generated journals are empty << -- Unexpected

$ cat 3-journal/202*/*
$

• as I've experienced in the past that periodic transactions may not be output if present in a file that has no other transactions, I add a noop transaction dated 2023 at the top of import/business/automation/depreciation/1-in/Dell-XPS-9500-20210112.journal

2023-01-01 ; depreciation

• run hledger-flow import
• the generated journals still do not contain the result of the periodic transactions, only the 'real' transactions << -- Unexpected

$ cat 3-journal/202*/*
2023-01-01  ; depreciation

2023-01-01  ; depreciation

2023-01-01  ; depreciation

2023-01-01  ; depreciation

2023-01-01  ; depreciation

2023-01-01  ; depreciation

What I ended up doing

• In the end I found out that all I need is to add the --begin and --end argument to match the current year, and I don't even need to use a dummy transaction. But this forces me to create an additional hledger print ahead prior to outputting the result of construct.

$ cat "import/business/automation/depreciation/1-in/construct"
#!/usr/bin/env bash

readonly year="$(basename "$(dirname "$1")")"

cat "$1" | hledger print -f - --forecast --begin "${year}" --end "$(( year + 1 ))"

$ cat "import/business/automation/depreciation/1-in/construct"
~ every 01-12 from 2022 to 2026
    Business-CY:Assets:Fixed-Assets:Laptop-Computers:Dell-XPS-9500-20210112     -389.4375 EUR
    Business-CY:Expenses:Depreciations  
$ hledger-flow import
$ grep ""  3-journal/*/*
3-journal/2022/dell-xps15-20210112.journal:2022-01-12
3-journal/2022/dell-xps15-20210112.journal:    ; generated-transaction: ~ every 01-12 from 2022 to 2026
3-journal/2022/dell-xps15-20210112.journal:    Business-CY:Assets:Fixed-Assets:Laptop-Computers:Dell-XPS-9500-20210112    -389.4375 EUR
3-journal/2022/dell-xps15-20210112.journal:    Business-CY:Expenses:Depreciations
3-journal/2022/dell-xps15-20210112.journal:
3-journal/2023/dell-xps15-20210112.journal:2023-01-12
3-journal/2023/dell-xps15-20210112.journal:    ; generated-transaction: ~ every 01-12 from 2022 to 2026
3-journal/2023/dell-xps15-20210112.journal:    Business-CY:Assets:Fixed-Assets:Laptop-Computers:Dell-XPS-9500-20210112    -389.4375 EUR
3-journal/2023/dell-xps15-20210112.journal:    Business-CY:Expenses:Depreciations
3-journal/2023/dell-xps15-20210112.journal:
3-journal/2024/dell-xps15-20210112.journal:2024-01-12
3-journal/2024/dell-xps15-20210112.journal:    ; generated-transaction: ~ every 01-12 from 2022 to 2026
3-journal/2024/dell-xps15-20210112.journal:    Business-CY:Assets:Fixed-Assets:Laptop-Computers:Dell-XPS-9500-20210112    -389.4375 EUR
3-journal/2024/dell-xps15-20210112.journal:    Business-CY:Expenses:Depreciations
3-journal/2024/dell-xps15-20210112.journal:
3-journal/2025/dell-xps15-20210112.journal:2025-01-12
3-journal/2025/dell-xps15-20210112.journal:    ; generated-transaction: ~ every 01-12 from 2022 to 2026
3-journal/2025/dell-xps15-20210112.journal:    Business-CY:Assets:Fixed-Assets:Laptop-Computers:Dell-XPS-9500-20210112    -389.4375 EUR
3-journal/2025/dell-xps15-20210112.journal:    Business-CY:Expenses:Depreciations
3-journal/2025/dell-xps15-20210112.journal:

Getting to the point

1. it feels like --end should be set automatically in the hledger print command that is used internally by hledger-flow
2. it feels like --begin should be set automatically in the hledger print command that is used internally by hledger-flow
3. if feels like --forecast should be settable by the person writing the construct script, so that even if hledger-flow does not want to activate it by default, the user can add his custom flags without having to create an additional hledger print command in the construct pipeline.

I feel strongly about 1 and 2. Should it not be the default?

As for 3, I feel it so much easier to:

• always add --forecast
• ensure periodic transactions that are actual forecast transactions (most of them aren't) have an additional forecast: tag
• use not:tag:forecast when i want to switch off forecasting. The --forecast option name has always felt funny to me.

To make the hledger-flow ignorant of all the idiosyncracies of this or that user, maybe hledger-flow could have a way to pass additional options to the hledger print command that runs on construct output (this would cover adding the --forecast option for me, if forecast is to remain switched off by default).

[apauley]

Hi @lestephane, I have finally sat down and tried out your examples and made sure I understood it.

My general feeling is that the context here is construct scripts which gives users all the power to generate transactions with minimal interference (just formatting). I wouldn't want to get into a position where we add options that work in a specific instance, only to find out later that it interferes with some other use case.

I liked the fact that you were able to solve this issue with a more intelligent script. It fits the "more power but more work" intent of construct. It does however feel inefficient that this means we'll do an additional hledger print when it could have been done with one. How do you feel about removing the pipe to hledger print which was intended just for formatting?

Specifically about your 3 points:

Regarding limiting transactions to the relevant year, this would have immediate consequences on one of my accounts.
A bank sent me monthly statements which goes from the middle of one month to the middle of the next month, and I'm currently using these unchanged as input. If construct excludes transactions based on the year I'll suddenly have excluded some overlapping January and December transactions of each year.

Yes, it will probably be better to modify my input statements so that it only contains transactions which fit the year, but I don't think users would want to be forced by the construct script to do that 😃

Regarding your --forecast suggestion, I added that option to hledger-flow's internal print, and it didn't break anything on any of my accounts. But it also doesn't solve this issue you described (not until you also add --begin and --end).

Can you think of other scenarios where adding just a --forecast would be helpful?

As a side note, one other possible solution for these forecasted transactions could be to designate some _manual_ journals for this purpose, and maybe have one of your own scripts ensure that it is populated.

But I do like what you did with the dedicated account.