ledgerpy

Scripts to transform different input formats (CSV and OFX) into ledger accounting files. The scripts allow manipulating ledger transactions based on CSV mapping files.

Other scripts attempt to handle the same use-case. I have tried a couple of them, as well as the integrated CSV import of hledger, and ran into issues with all of them. That's why I wrote yet another CSV to ledger tool.

There are two main scripts, getofx, and toldg. The former uses the Python ofxtools library to download bank transactions via OFX and stores them into CSV files. The latter takes CSV files and transforms them into ledger accounting files.

The OFX script works well for my workflow, but I am not sure if it would be beneficial to other people. My premise is that I update my accounting files at least once a week. Therefore, the OFX script downloads the instructions for the last thirty days and then merges them into the CSV file (if it exists).

You might object that it makes more sense to download all the data for the first run and then update for consecutive runs. That was my idea in the beginning, but it turns out that the OFX credit card interface of my bank only returns the transactions of the last 60 days. Hence, I downloaded all transactions manually and then set up in the incremental updates. The examples directory contains an example configuration for this script.

On the other hand, I am pretty happy with the CSV transformation script. Most of my workflows are editor based, and the mapping file-based approach makes it easy to manipulate transactions. Also, the script relies on a single configuration file, which makes the configuration clearer.

The directory tools contains additional scripts for personal use. Currently their main use-case is to generate a PDF report with figures. I use hledger to generate CSV data, plot figures with pandas, and then generate a report with pandoc. A shell script executes this process. This workflow fulfills my requirements, but is currently not useful for anybody else.

Dependencies

The scripts rely on a couple of newer Python features, such as data-classes, format-strings, and the walrus-operator. Python 3.8 or later is therefore required to run the scripts. Additionally, the OFX script relies on ofxtools (pip install ofxtools). Transforming CSV files into the ledger format does not require additional packages.

Usage

Invoke python3.8 with either of the scripts and provide a configuration file as the first argument.

git clone https://git.felixm.de/felixm/ledgerpy.git
cd ledgerpy/example
python3 ../toldg.py configs/toldg.json # or `make` alternatively

You can see that toldg copies and transforms the input files in processed directory using the mappings from the CSV files in the mappings directory. For transactions that do not have a matching mapping toldg creates a default mapping into mappings/unmatched.csv, as shown in the following listing.

expenses,UBER ABO13,credit=-29.37;date=2018/12/03

The first part is the new payee (or account2) for the transaction. The second part is the string used to match the description of the transaction. Normally, the script uses a string compare. If the string starts and ends with a frontslash / toldg compiles it into a regex and tries to match the description: regex.match(descriptio). The last field is a query specification string in the following form:

field1=string1;field2=string2

I have added this feature to specify different payees for the same store. For example, sometimes a get groceries from Target and other times furniture (household expenses). In case multiple mappings match a transaction the script uses the first match. I might change that into the mapping that has the most query parameters. That way one could have one default mapping (usually I get groceries from Target), but then override on a case per case basis without getting warnings.

Todo

• Write this readme
• Use OFX parser from ofxtools instead of parsing the XML
• Autoappend latest OFX data to CSV file
• Include example workspace with mock data to demo my workflow
• Write script to generate PDF reports