beancount-txn-elisp: A library to read/parse and write/insert individual Beancount transactions, implemented in Emacs Lisp.

beancount-txn-elisp

#beancount-txn-elisp

#Introduction

A library to read/parse and write/insert individual Beancount transactions, implemented in Emacs Lisp.

N.B.: The package is listed on Sourcehut as beancount-txn-elisp, however the file and (Emacs) feature names are simply beancount-txn (further info in Naming section).

#Installation

The easiest way (considering potential updates) for the time being is probably just to clone this repo:
```
~/ $ cd git/ext
~/git/ext $ git clone https://git.sr.ht/~trs-80/beancount-txn-elisp
```

Then say something like the following in your init file:

(add-to-list 'load-path "~/git/ext/beancount-txn-elisp")
(require 'beancount-txn)

Why the differences in naming? See Naming section.

#Getting started

The main entry points (see docstrings for further details) are:
- bean-txn-parse - Parse Beancount txn at point (returns TXN-FORM).
- bean-txn-format - Compose args into a Beancount txn (via format, which returns a string).
- bean-txn-insert - Insert Beancount txn at point, from args.
The format and insert functions also have a form that accepts a TXN-FORM (instead of args):
- bean-txn-format-txn-form - Compose TXN-FORM into a Beancount txn, returning it as a string.
- bean-txn-insert-txn-form - Insert Beancount txn at point, from TXN-FORM.
Studying the tests file (beancount-txn-tests.el) along with the accompanying sample txns (in beancount-txn-tests.bc) might be the easiest way to visualize the TXN-FORM.
I also wrote some debugging commands during development. Put point on a txn and run them and have a look.

#A word about types

#When are metadata values returned with quotes?

Or, more accurately:

Why are some metadata values returned with (extra/escaped) quotes around them, and some are not?
Meanwhile date, flag, payee, narration, etc. are always returned without (extra/escaped) quotes?

The short answer is because those values (date, flag, payee, narration, etc.) are unambiguous, but metadata values can be alomst anything.

We have no idea what the person who wrote the ledger had in mind, so we have to take whatever is there at face value. This is the only sane way.

This is also required to do proper round-tripping.

#Everything are strings

Keep in mind that we are not doing any kind of type inference nor conversion. From the point of view of our parser and writer (and Emacs), everything are just strings.

This is not true with other (external) tools (most notably, Beancount itself, which is written in Python and has types).

#An example

With the above in mind, consider the following example:

2024-07-28 * "Acme" "widgets"
  order-id:     "123"
  confirmation: 321
  Assets:Bank:Checking                 -10.00 USD
  Expenses:Whatever

If you were to run bean-txn-parse-debug-message on it, you would get:

date:        |"2024-07-28"|
txn-flag:    |"*"|
payee:       |"Acme"|
narration:   |"widgets"|
tags-links:  |nil|
txn-metadata:|(("order-id" . "\"123\"") ("confirmation" . "321"))|
postings:    |((nil "Assets:Bank:Checking" "-10.00" "USD" nil nil nil nil) (nil "Expenses:Whatever" nil nil nil nil nil nil))|

Consider the difference between:

The value of order-id which is written in the ledger file as "123" (and thus returns "\"123\"" i.e., with surrounding quotes included (and escaped))),
and the value for confirmation, which is written in the ledger file as the number 321, so that is what is returned (technically, "321", as again, "everything is a string").

#What works

The txn parser is quite complete (it even has tests!).
- With the exception of supporting math in postings (see Roadmap).

#Round-tripping

We have achieved the project goal to insure that "round-tripping" works (i.e., txn in file -> parser -> variable/memory -> wrtiter -> txn form again), without losing any information.

#Roadmap

#Development status

I currently consider this "alpha" software.

Not because it doesn't work (it works completely reliably for me and even has some tests).¹

But rather because it is still in active development, and may not have a stable interface.

For example, if/when I implement parsing comments before the txn, that will probably become the first part of TXN-FORM. Which will, of course, require updating anything else which relies on that. And there are likely other similar cases.

I am billing this as "a library" and so I wanted to point this out. Since I decided to make a public release, from now on I do promise to point out any breaking changes in the release notes.

#Hopefully soon

These are higher priority things that I want to complete soon:

[ ] Create a function (and settings) to insure an empty line before (and/or after) inserting txns.
[ ] Have bean-txn-parse return comments immediately before the txn.
[ ] Write tests for the writer (and/or "round trip" tests).

#Someday maybe

The following do not work (yet) but I would like to get them working "someday/maybe"² (roughly in descending order of priority):

[ ] Further parse tags-links into another nested list.
[ ] Working with (parsing/writing) multiple txns at a time.
- It shouldn't be too hard to write some kind of a map function (famous last words? lol).
[ ] Write some getter wrappers for specific components of TXN-FORM (e.g., payee, narration, date, txn-metadata, etc.;is this necessary?).

#Probably never?

[ ] Parse comments at end of main txn line.
- We already parse comments at end of posting lines, this would be the equivalent for txn lines. Does anyone need this (I don't think I do)?
[ ] Parsing directives other than txns.
- Not a feature I personally think I need, but who knows what the future brings. I suppose I would accept a patch (but please discuss first)?
[ ] Math in postings.
- This is supported in Beancount, however… Does anyone actually do this? Honestly I will likely never get around to it, unless someone speaks up.

#Out of scope

The following are not desired, for the reason(s) listed:

Working with multiple files. This is functionality I also desire, but this will be³ implemented in a separate project.

#Extras

#Features

Some other quality of life improvements in this package:

Hooks into bean-txn-insert function:
- bean-txn-insert-pre-hook
- bean-txn-insert-post-hook

#Expanded from upstream

We expand on some features from the upstream beancount-mode project:

We correctly parse escaped quotes in Payee and Narration (bean-txn-string-quoted-regexp).
Being a regexp based parser⁴, it's probably no surprise that our regexps actually parse quite a lot more than upstream, for examples see these variables:
- bean-txn-posting-regexp
  - We correctly parse flags on postings.
  - Capable of parsing comments at end of posting lines.
- bean-txn-transaction-regexp
We indent posting metadata an additional 2 spaces when writing out txns (configurable in variable bean-txn-write-indent-posting-metadata (which see)).

#Reporting bugs

If anything does not work, I am very interested in receiving bug reports!

I would like more people to battle test this software (especially the parser).

The project has a public ticket tracker here where you can submit reports by way of plain old email.⁵

#Hacking

#Terminology

I have tried very hard to follow upstream Beancount project terminology precisely, in order to avoid confusion (e.g., flag, payee, narration, posting, cost, metadata, etc.). This applies to all parts of the project, from this README, to function and variable names (even internal ones), even comments in regexp and elsewhere in the code.

#Naming

I tried to put some thought into my naming convention.

Namespace (function and variable name prefix)

Originally I was using beancount-txn- as a prefix for my function and variable names. However (while still in development, before publishing) I decided to switch to using bean-txn- instead, for a couple reasons:
1. To more readily distinguish from the beancount- prefix that upstream beancount-mode uses (in particular, we shadow some of their variable names).
2. As an additional benefit, bean-txn- is also shorter.
Emacs feature (and file name)

N.B.: The Emacs feature (and file name) are still beancount-txn (and beancount-txn.el, respectively). This is more appropriate within the context of Emacs (e.g., package managers, or inside Emacs).
Repository name

However the repository is listed on Sourcehut as beancount-txn-elisp. This is more appropriate:
1. Not only within the context of SourceHut (where there are many kinds of software), but also
2. The wider world (as maybe some day we get a txn parser in some other language).
"Corporate we" / "our"

So far it's just been me hacking away on this. Perhaps this is wishful thinking. Or trying to will it into existence? :)

#Semantic Versioning

Yes I intend to (more or less) follow Semantic Versioning.

The plan is to release the 0.0.x series as "alpha". Once I think the API is fairly stable, I guess I will call that 0.1.0 (or maybe even 1.0.0 I suppose, but we will see).

I am also going to try and keep the version number the same across the project (i.e., in both beancount-txn.el and beancount-txn-tests.el).

#Debugging commands

I wrote some debugging commands during development. I figured I would leave them in, they might be useful:

bean-txn-parse-debug-message - Parse txn at point and return debug message.
bean-txn-parse-debug-round-trip-message - Parse txn at point, pass back thru format function, show result in minibuffer.

#Running tests

In case you are unfamiliar with ert (but it's actually pretty easy, as you will see):

load the file beancount-txn-tests.el.
1. This file will, in turn, require the ert package (which has been included with Emacs since version 24).
Run (M-x) the interactive command ert (ert-run-tests-interactively).
It should pop up a list of all available tests (assuming you did #1, above).
But you will want to enter "^bean-txn-test-*" (including the quotes) then press RET to select all tests from this package.
1. There are, of course, other ways to select which tests to run (see 'Test Selectors' section of manual ([Info](ert#Test Selectors) web)).

#Footnotes

¹ In fact, please submit a bug report if you find anything amiss!

² This is a concept from Getting Things Done (GTD), in case you never heard of it. In my personal Orgmode-based project and information management system it's an actual (and often used) todo state as well. :)

³ Actually, is already (mostly), just unpublished. Working title beancount-multi-file. I use it almost every day, just needs some polish for release. Let me know if you are interested.

⁴ "and now we have 2 problems" :D

⁵ This is something I really appreciate about Sourcehut (one of their explicitly stated goals is not trying to lock you in to their silo). I find this a quite refreshing change from (what seems to be) the norm these days.

~trs-80/beancount-txn-elisp