How to test dApps (decentralized applications)
If you’re a test engineer and you’re keen to get involved in Web3, it can be hard to know where to start. There are literally thousands of tutorials aimed at blockchain developers but few so far that are focused on testing.
A few people have asked me questions about this recently, so I thought it might be useful to write a post. Please add your thoughts/criticisms/ideas in the comments.
This is not a tutorial. Instead, it’s a starting point, with resources you can use to start exploring this new and expanding sector.
It helps if you’ve already got some interest in crypto or decentralized technologies and are comfortable using a wallet and interacting with token economies, but the great news is that with some curiosity and application, the right resources are only a few clicks away.
First of all, what will you actually be testing? This post specifically covers projects that are decentralized applications (dApps) built on Ethereum, or EVM (Ethereum Virtual Machine) compatible networks.
There are various jobs that involve testing/quality assurance at the protocol level, or testing applications built on other networks written in different languages, but they are outside the scope of this post.
Getting started
If you have had no exposure to Ethereum at all, read this for an overview.
Then read this to understand what a smart contract does, as this will form the basis of nearly everything you need to test. You need to know the basics of how to read a smart contract and understand what the functions do, at least at a high level..
Remix is an in-browser IDE that allows you to write, explore and test smart contracts without worrying about dependencies. It’s a great place to start.
You’ll also need to be able to run a dApp locally so you can test it against a simulated mock blockchain. Frameworks such as Hardhat and Truffle walk you through this process.
Want to play with the simulated blockchain on its own to start with? Read the documentation and watch a video tutorial for Ganache.
Exploratory testing
The processes are going to be similar for any kind of consumer-facing dApp, whether it’s mobile or web. Remember that some wallets don’t work on mobile, and vice versa.
Put yourself in the user’s shoes and imagine seeing and using your application for the first time. Does the user interface and general user experience make sense or does it assume pre-existing knowledge of the Web3 ecosystem? This is a new area of technology and users will need more signposting and guidance than usual.
You will also need to have a wallet such as Metamask installed and practise switching between networks, adding accounts and importing tokens.
Do not neglect the basics, such as accessibility testing. This should go without saying: there is no excuse for inaccessible applications, decentralized or otherwise.
The sort of things you need to think about are:
- Does the dApp throw any unexplained errors when you go through the normal user flow? What about if there is a loss of connectivity? Does it revert to the correct state?
- Does the messaging inform the end user that they are transacting on a blockchain? How informative is the messaging?
- Does the dApp advise users which parts of the service are centralized or make claims that do not stand up?
- If you are testing a wallet, is there sufficient messaging around the importance of keeping your private key or recovery phrase safe?
- If the network is busy and the transaction takes a long time, does the transaction time out and result in an error? Or ask the user to retry?
- Is any payment deducted from the wallet the right amount? What happens if you make multiple requests or try to make transactions that exceed the balance of tokens in your wallet?
- What happens to the state of the app if you cancel the transaction in your wallet?
- Does the web page or mobile screen automatically refresh in response to changes in your wallet or the network or do you need to manually refresh?
- If using an external API are prices etc updating correctly?
- Is the user warned that transactions are irreversible?
Once you have tested the application locally, You will probably need to use a testnet such as Ropsten or Rinkeby. This is like a scaled down version of the live Ethereum network, which has its own (valueless) tokens. To test transactions on one of these test nets you will need to get some test ETH (from a faucet like https://faucet.ropsten.be/).
Other tools you should know about are EtherScan and EthGasStation. EtherScan is a block explorer for the Ethereum blockchain. This is where you can see addresses and transactions. Ethereum’s testnets have their own instances of EtherScan.
Gas fees (the price you have to pay to make a transaction on Ethereum) are currently a pain point. When gas fees are high, you should think about your dApp’s requirements and whether it makes business sense for users to carry out certain transactions. EthGasStation provides a useful overview of current and predicted gas fees.
Automated testing
I cannot overstate the importance of a well written, comprehensive test suite. Developers should write their own contract tests, but you should be able to read the tests and suggest alternative test cases that they may not have thought of.
For example, here are the tests for the Boson Protocol smart contracts.
Working with the rest of the development team to identify edge cases and improvements in coverage should be part of the test engineer’s job. Coverage tools such as sol-coverage are useful to a certain extent, but will not necessarily identify particular scenarios that need to be accounted for — and this is where a test engineer’s skills are invaluable.
For end-to-end automated testing, it can be tricky because many popular testing frameworks make it difficult to access elements outside the DOM, such as wallets in browser extensions.
Dappeteer is a fork of Puppeteer that actually incorporates Metamask interactions. The original version of Dappeteer is no longer maintained, but there is a fork here created and maintained by ChainSafe. Otherwise, you may consider using something like Playwright, which allows interactions with browser extensions and supports multiple tabs (thanks to Krzysztof Paliga for the tip).
Bug bounties
Most decentralized projects tend to be open source, and bug bounties are a popular way to engage the entire community in improving the quality of a project. This is the one we run at Boson Protocol. Whatever type of testing you do, it is impossible to know and do everything and to spot all vulnerabilities.
Bug bounties can never be a substitute for good testing practices, but they are an essential tool in modern quality processes for Web3 projects.
Read this thread for the account of how a huge Coinbase bug was found and fixed.
