Bills/transactions matching algorithm¶
The matching algorithm is in charge of finding matchings between
io.cozy.bank.operations documents. This document explains
how the matching algorithm works.
What are matchings?¶
Matchings are links that can be made between
For example, if the user ordered something on Amazon, his Amazon connector will
io.cozy.bills document representing this order, and his banking
connector will import a
io.cozy.bank.operations representing the bank
transaction associated to this order. In banks’ UI, it is used to show a CTA
with the transaction on which the user can click to show the linked bill PDF
without leaving banks app.
The algorithm is held by the
and more precisely its
linkBillsToOperations method. This method takes 3
- An array of
- An array of
- An options object
The options object can contain the following properties:
minAmountDelta: defaults to
maxAmountDelta: defaults to
pastWindow: defaults to 15 (days)
futureWindow: defaults to 29 (days)
These options will determine the default date and amount windows in which a transaction should be to match with a bill.
When called, this method will do the following (each part will be explained in details after):
- Filter bills that are declared as third party payer: these bills have no matching transation since it has been paid by somebody else
- Then for each bill:
- Try to find a matching debit transaction (amount < 0)
- Try to find a matching credit transaction (amount > 0) if the bill is a refund
- Try to combine bills that didn’t match anything to see if they match something when combined
- Update transactions that matched in database
Finding debit and credit operations for a bill¶
The algorithm is globally the same to find a debit or a credit operation for a bill, but there are some differences at some steps. Each step is explained below.
Find neighboring transactions¶
First, we look for transactions that are « neighbors » of the bill. It means the ones that fits in the amount and date windows of the bill.
The date window is
[bill date - dateLowerDelta; bill date + dateUpperDelta]. Where:
bill.dateif we are looking for a credit;
bill.originalDatewith a fallback on
bill.dateif we are looking for a debit
bill.matchingCriterias.dateLowerDeltaif it exists,
bill.matchingCriterias.dateUpperDeltaif it exists,
The amount window is
[bill amount - amountLowerDelta; bill amount + amountUpperDelta]. Where:
bill.groupAmountwith a fallback on
bill.amountif we are looking for a credit;
-bill.originalAmountwith a fallback on
-bill.amountif we are looking for a debit
bill.matchingCriterias.amountLowerDeltaif it exists,
bill.matchingCriterias.amountUpperDeltaif it exists,
All transactions that matches these two criterias are kept.
Apply filters to neighboring transactions¶
There are other filters a transaction must pass to match with a bill.
Health bills are a little special because they can only match with health transactions. So if the bill is an health bill, we keep only health transactions. Otherwise, we keep all but health transactions.
You can pass a
allowUncategorized option to the algorithm, in which case
uncategorized transactions will also be able to match bills.
This filter is applied only if we are looking for a debit transactions.
If we try to link a bill as a reimbursement to a debit transaction, we check that the bill’s amount and transaction’s reimbursements sum does not overflows the transaction’s amount.
This filter is applied only if we are looking for a credit transaction or if the bill is not of health type.
This filter checks if a transaction matches a regex that is built upon the bill’s properties. Here is how the regexp is built:
- If the bill has a
matchingCriterias.labelRegexproperty, this is used
- Otherwise we try to get the brand corresponding to the bill’s vendor in our brands dictionary. If it exists, we use its
- Otherwise, we create a regexp containing the bill’s vendor
- Lastly, if the bill has no vendor, we can’t do anything and the filter will be
falsefor every transaction and the bill will not match any transaction
Sort remaining transactions¶
A score is given to the remaining transactions according to their « distance » to the bill’s date and amount. The closest one wins and continues through the algorithm.
Link bills to debit and credit transactions¶
When transactions were found for a given bill, we need to link the bill to
them, either as a
bill or as a
When a bill matched with a transaction (debit or credit), it is added to the
When a bill matched with a debit and a credit transactions, it is also added to
the debit transaction’s
A last safety net is applied by checking if adding the bill to the transaction
would make the bills sum overflowing the transaction amount or not. It it
overflows, the bill is not added. For the
bills property, a removed bill will
be ignored. But for the
reimbursements property, since we don’t fetch the
bill because the amount is already present in the transaction, a removed bill
will not be ignored.
Sometimes, bills need to be combined to match with a debit transaction. It is generally the case for health bills. For example, when your paid multiple medical procedures with a single transaction. It will generate multiple bills for a single debit transaction. In this case, we try to combine bills and find debit transaction for them.
This combination process is not required to find debit transaction because in
that case the bill’s
can be used.
The bills that are candidates to be combined are the ones that:
- Didn’t match with a debit transaction
- Are health costs or are from a vendor that groups bills
These bills are then grouped by date and vendor, so we don’t mix bills that have nothing in common. Then all possible combinations are generated for each group. For each combination, a « meta bill » is created from the grouped bills and this meta bill is passed in the matching algorithm to find a debit transaction that matches with it. If a transaction is found, then all the bills making the meta bill are linked to it using the same logic as before.