Data Lake

The Data Lake makes your ledger data available for reporting and analysis through Insights and the Data Lake API. It maintains a separate reporting copy, so you can explore historical activity without adding analytical workloads to your live ledger. Your ledger remains the source of truth. Data Lake lets finance, operations, and audit teams explore unlimited history without adding reporting load to that path.

The Data Lake is available only on managed instances and to Enterprise customers. It is not included with connected self-hosted Cores without an Enterprise license.

What you can query

The lake exposes four tables: transactions, balances, ledgers, and identity. Open Schema to see the columns and how they join. Records are written to your ledger before they become available in the Data Lake; this means that a payment can already exist on the ledger without appearing in Insights yet. Keep that delay in mind when you report on recent activity.

Question	Where to get it
Money that moved over a period	Data lake
Wallets, ledgers, and identities that appear in transaction history	Data lake
Amount in a wallet right now	Ledger API
Amount in a wallet at a past time	Historical balances

How the date range affects your query

The date range in the Insights toolbar determines which period of history your query can access. Your SQL then filters and analyzes the records within that selection. On every table, that range applies to created_at:

On transactions, created_at is when that lifecycle row was created in your ledger. A September range is payments that moved through a status in September.
On balances, ledgers, and identity, created_at is when the wallet, ledger, or identity was created. A September range pulls records created in September, not every record that existed in September.

Choose a range that covers the activity you want to analyze. A monthly volume report might only need that month. An investigation into pending transactions may need a wider period so commit or void rows are included. If the toolbar is set to September, an SQL filter for August will not retrieve August records. If you want to include August records, change the toolbar range to include August.

How transactions appear

The Data Lake can contain multiple records for a single payment, one per lifecycle stage. A payment may have a QUEUED row followed by an APPLIED row. Those describe the same payment at different stages. Counting both incorrectly counts the payment twice. For example, if you want to report on completed transactions, your query must filter for APPLIED records only:

SELECT
  currency,
  COUNT(*) AS txn_count,
  SUM(
    TRY_CAST(precise_amount AS DECIMAL(38, 8))
    / NULLIF(TRY_CAST(precision AS DECIMAL(38, 8)), 0)
  ) AS volume
FROM transactions
WHERE status = 'APPLIED'
GROUP BY currency

Without the WHERE filter, the same payment is counted once as QUEUED and again as APPLIED. For other reports, change the status to the stage you care about. See Transaction lifecycle. To follow an inflight payment to its outcome, connect related rows with parent_transaction or meta_data.QUEUED_PARENT_TRANSACTION. Those rows can fall on different dates. A hold on September 30 that commits on October 1 needs a range that includes both days.

SELECT
  hold.transaction_id,
  hold.created_at AS held_at,
  hold.status,
  outcome.transaction_id AS outcome_id,
  outcome.created_at AS resolved_at,
  outcome.status AS outcome_status
FROM transactions hold
LEFT JOIN transactions outcome
  ON outcome.parent_transaction = hold.transaction_id
  AND outcome.status IN ('APPLIED', 'VOID')
WHERE hold.status = 'INFLIGHT'

If the toolbar ends on September 30, an October 1 commit is outside the range. outcome_id comes back empty and the hold looks unresolved. Widen the range to include both days.

How balances appear

A wallet keeps one balance_id for its lifetime. Use the balances table to count wallets, filter by ledger or currency, and join transactions to the wallet, ledger, or identity they belong to. The toolbar range follows created_at, so you can also report on wallets created in that period:

Wallets created by currency

SELECT
  currency,
  COUNT(*) AS wallet_count
FROM balances
GROUP BY currency

Join balances to transactions when you need ledger or owner context on volume. This totals applied volume that touched wallets in one ledger:

Applied volume for a ledger

SELECT
  t.currency,
  SUM(
    TRY_CAST(t.precise_amount AS DECIMAL(38, 8))
    / NULLIF(TRY_CAST(t.precision AS DECIMAL(38, 8)), 0)
  ) AS volume
FROM transactions t
JOIN balances b
  ON t.source = b.balance_id
  OR t.destination = b.balance_id
WHERE b.ledger_id = 'ldg_049495c6-356e-4ebc-a45e-60d1e1e16afb'
  AND t.status = 'APPLIED'
GROUP BY t.currency

Set the toolbar to the period you want to measure. Filter to source or destination only if you want outflow or inflow. Use ledgers and identity the same way: join them when a report needs a ledger name or an owner.

Write queries that match the data

Keep these in mind when you write SQL:

The lake is for history, not live holdings. Do not treat balances.balance as the current amount in a wallet. Use the Ledger API for that.
Each transaction status is its own row. Filter by status, and use parent_transaction when you need commit or void outcomes.
The date range follows created_at. On wallets, that is creation time. A short range will omit older wallets, even if they still exist on the ledger.
Widen the range when you follow a parent and its children. Inflight commits and voids create new rows. If the outcome sits outside the range, the report can miss the resolution.
Dates and amounts can be typed differently across days. Wrap them in TRY_CAST so one mismatched row does not fail the whole query. See Accepted SQL.

Need help?

We are very happy to help you make the most of Blnk, regardless of whether it is your first time or you are switching from another tool. To ask questions or discuss issues, please contact us or join our Discord community.

Start Here

Instances

Ledgers

Balances

Transactions

Identities

Reconciliation

Filters & Views

Alerts

Monitoring

Insights

Apps

Settings

What you can query

How the date range affects your query

How transactions appear

How balances appear

Write queries that match the data

Need help?

​What you can query

​How the date range affects your query

​How transactions appear

​How balances appear

​Write queries that match the data

​Need help?

What you can query

How the date range affects your query

How transactions appear

How balances appear

Write queries that match the data

Need help?