Using Blockchain Information as Audit Evidence
Key risks, controls, and vendor considerations for using blockchain data received from third party vendors as audit evidence.
Overview
Audit procedures to support the completeness, existence, and accuracy of crypto assets account balances and transactions often use blockchain information as audit evidence. Auditors use various methods to acquire relevant data from public blockchains, including obtaining it from third-party data vendors. The “Accounting for and auditing of digital assets” Guide by AICPA briefly mentions some considerations relevant for auditors engaging a third party to obtain blockchain information. But today, we are going to try to dig a little bit deeper and understand the blockchain information acquisition process in more detail.
We will focus only on service organizations that supply blockchain data but do not authorize or initiate transactions. These providers often bridge blockchain data and tidy, tabular reports suitable for accounting systems, a step frequently overlooked in the literature.
As a valid alternative to running their own node infrastructure, auditors can use a third-party vendor. Firms might engage a third-party vendor directly or obtain it from the audit client. Respectively, auditors using the blockchain information received from third-party data vendors will follow a testing approach consistent with the selected approach to data acquisition:
- Information received directly from the external source when using a substantive testing approach.
- Information received from the entity when testing controls.
The sections below outline relevant considerations.
Blockchain Data Observability
Most third-party blockchain APIs alone rarely provide enough information to reconcile an account balance due to non-transaction state changes and other factors. Auditors must use alternative methods.
Now, let’s briefly discuss raw blockchain data structures and outline the steps needed to access raw blockchain data.
Not all data recorded on blockchain is relevant to auditors, and not all pertinent data for auditors is directly captured on the blockchain. There are two generalized data elements relevant during the audit of financial statements of companies that are involved in digital assets:
- State, which is a value assigned to a record at a specific block height (i.e., the number of a block in the blockchain). On Ethereum, includes:
“In Ethereum, state includes:
Account balances and nonces
Contract code
Contract storage
Consensus-related data (some recent block hashes, uncles, proof-of-stake consensus data such as validator pubkeys and activity records on the beacon chain, etc)”
[“A Theory of Ethereum State Size Management” by V. Buterin on 02/12/2021]
- State Transition, which is a change in a value assigned to a record between the block height specified and the immediately preceding block height.
Blockchain does not always capture every state and state transition because doing so would be computationally expensive. Some blockchains record both; others track only one, or neither, but allow recomputation. Each blockchain, and each smart contract, has its own way of recording state and transitions.
The process of data acquisition includes the following steps:
Downloading the blockchain node software
Installing the node software and spinning up the node. This includes connecting and syncing the node with the network. During a sync, depending on the storage mechanism used by the node software, there are risks of incomplete data download, partial loss, or corruption of downloaded data, and unaccessibility or unreadability of downloaded raw blockchain information. This might happen due to:
Storing individual blocks in separate files (Bitcoin and Bitcoin Cash).
Selecting the wrong sync mode results in a loss of information about transactions that occurred prior to the period covered by a light node scope.
Direct Query. Once the node is synced, it might be possible to query the node directly using its native RPC interface. When querying the node RPC interface, we do not access or observe raw blockchain data. We rely on the node software code to be designed appropriately and to return the data it promises based on the documentation (if such documentation exists, which is not always the case). However:
Bugs in the node client software that eliminate the ability to use standard commands to extract needed data via the node RPC interface, such as querying the blockchain state at a specific block height.
Data Indexing. Raw blockchain blocks are encoded and unreadable. Auditors needing to manipulate or filter data must preprocess it with indexer software. Indexers organize the data, but this requires understanding both on-chain data structures and the result of executing blockchain code on that data. Risks include:
Inappropriate indexing code logic results in incomplete or inaccurate data being returned. This is not a trivial task at all. Here is a post about a prominent security research firm trying to determine the number of all smart contracts deployed on Ethereum using different approaches and receiving drastically different results.
Missing or incorrect reference files, including Genesis files containing the initial allocation of tokens to wallets on Day 0 and Application Binary Interfaces (ABI) of smart contracts. This data is not stored on the chain, and it might not be possible to index a smart contract without knowing its ABI or to determine the account balance without a genesis file.
Indexed Data Storage. Indexed data is typically saved in a modern fast database (“serving database”), which is convenient for querying using SQL to extract human-readable data with the required fields and values. Risks arising at this point include:
Data loss or corruption occurs during the initial operation performed to record indexed data in the serving database.
Data loss or corruption can occur while storing data in the database. For example, physical disk damage or malfunction can cause data loss.
Incorrect logic of SQL queries from the indexed database results in incomplete or inaccurate data being returned.
Transmission. Finally, auditors receive data via email or API in the form of a PDF report, Excel or CSV file, or other format. The main risk is inappropriate modification of data during transmission for various reasons.
Understanding these risks and the responses in place to address them is key to the successful use of blockchain information as audit evidence.
1) Information received directly from an External Source
In the context of an individual audit engagement, a third-party blockchain data vendor used by the audit firm is an external source of information. Hence, auditors need to consider the requirements for using information received from external sources as audit evidence.
PCAOB Staff Guidance [1] provides some helpful reminders about how auditors should approach audit evidence obtained from external sources. When using information about historical balances received directly from a third party as audit evidence, auditors need to consider the relevance and reliability of the information.
Relevance measures how well audit information aligns with the audit procedure’s purpose. When testing digital asset balances, the quantity of tokens in the wallet at the end of the period is directly relevant.
Reliability of audit evidence depends on sufficiency (quantity) and appropriateness (quality). Information about asset quantities at year-end may or may not be reliable. How do we assess reliability?
Many practitioners believe a SOC 1 or SOC 2 from the blockchain data vendor is enough to assess the reliability of the information received.
SOC I Type 2, if it tests controls over report completeness and accuracy, can help. When auditors use vendor reports directly, a SOC report alone generally is not enough to ensure client-specific information is complete and accurate because:
a) Service auditor reporting on the blockchain data vendor performs audit procedures using sampling at the level of the vendor. The sample size might not be sufficient to provide the level of assurance consistent with the tolerable misstatement threshold used for the purposes of the audit of the reporting entity.
b) If the reporting entity relied on the information received from the blockchain data vendor with reference to the SOC I Type 2 report, management would have to do a considerable amount of additional work to avoid the deficiency in its internal controls (refer to the next section for further details). It would not be appropriate to state that audit firms can rely on the same information without needing to perform additional work to validate it.
Key considerations for reliable external audit evidence include:
Independence, competence, expertise, and capabilities, including the infrastructure reliability of the vendor and the use of third-party vendors.
Limitations and disclaimers that affect the use of the vendor’s data.
Cybersecurity, known vulnerabilities (unresolved), and responses implemented by the vendor.
The risks associated with a specific blockchain network.
Responses to data integrity risks occurring throughout the full lifecycle of data (see the previous section).
2) Information Received from the Entity (Client Using a Service Organization)
Reporting entities (e.g., web3 startups) may rely on a blockchain data provider if both of the following are true:
a) During the relevant period, the vendor maintained effective controls over relevant processes and specific data reports used. This assessment is based on the SOC 1 Type 2 report received from the service organization and bridge letters showing results of subsequent testing during the interim periods between the service organization’s year-end and the period-end date of the user’s financial statements being audited.
b) Management designed and implemented:
Complementary user-entity controls identified in the vendor’s SOC 1 Type 2.
Compensating controls to mitigate any deficiencies identified in that report; and
Entity-specific controls over risks at the reporting entity are not sufficiently covered by the vendor’s SOC 1 Type 2.
Accordingly, auditors may rely on blockchain data that:
is received from the client,
which the client obtained from a service organization,
with a SOC 1 Type 2 report,
covering processes, controls, and reports relevant to the entity, and
is supported by effective controls implemented at the level, allowing us to ensure that relevant information to the reporting entity is complete and accurate for financial statement purposes.
When these conditions are met, auditors might be able to rely on blockchain data received from an external vendor. To rely, the auditors’ testing approach should be designed assuming the auditors’ reliance on the effective design and operation of relevant internal controls of the audit client. To rely on controls, auditors should test those controls, including those over the accuracy and completeness of blockchain information received from the service organization (blockchain data vendor).
If these conditions are not met, reliance on data received from an external vendor API is generally not appropriate.
Conclusion
Unfortunately, blockchain data cannot be directly observed by auditors. The data transformation process is complicated and prone to errors. Regardless of the data source and testing approach, there are numerous risks related to the completeness and accuracy of the human-readable representation of blockchain information used by auditors. To address these risks and obtain sufficient audit evidence, auditors will likely require further audit procedures.
[1] Spotlight Reports “Data and Reports” and “Evaluating Relevance and Reliability of Audit Evidence Obtained from External Sources”
This was some top notch article ! Ive always been fascinated on blockchain data , been trying to learn about it ,I’d say through your work I’ve learned a couple of things 👏👏👏.Im looking forward to more of this and learning from your work 😊