Don’t fight your external problems, laugh at them

In my recent blog posts, I’ve primarily focused on hardcore crypto topics, which may be too specialized for a broader audience. In this post, I aim to address issues that resonate with the experiences of most software engineers and demonstrate how they can be solved when writing code with Elixir. Specifically I’ll describe how to write maintainable and easy to test code which makes requests to external services.

By the end of this post, the choice of the title will become clear. So, buckle up!

Journey begins

A contemporary web application rarely functions in isolation; in most instances, it engages in communication with other web services. Consequently, developers frequently find themselves tasked with crafting intricate logic for interacting with numerous external services.

Let’s examine the following example. When it comes to synchronizing crypto transactions, such as those on the Ethereum blockchain, the simplest and least error-prone approach involves fetching each block along with its transactions individually and then checking if any of the transactions within a block belong to one of your addresses.

def sync_transactions(block_number) do
  with {:ok, block_hash} <- fetch_block_hash(block_number),
       {:ok, block_with_transactions} <- fetch_block_with_transactions(block_hash),
       {:ok, found_transactions} <- find_transactions(block_with_transactions),
       {:ok, inserted_transactions} <- insert_transactions(found_transactions),
       {:ok, inserted_additional_data} <- insert_additional_data(found_transactions) do
     {:ok, inserted_transactions}
  else
    # handle each error separately
    {:error, :request_timeout} -> ...
    {:error, :db_error} -> ...
    ...
  end
end

# fetches block hash from a blockchain node
defp fetch_block_hash(block_number) do
  EthereumClient.fetch_block_hash(block_number)
end

# fetches block with its transactions from a blockchain node
defp fetch_block(block_hash) do
  EthereumClient.fetch_block(block_hash)
end

# finds transactions that belong to our addresses
defp find_transactions(block_with_transactions) do
  to_addresses = Enum.each(block_with_transactions["transactions"], fn transaction ->
    transaction.to_address
  end)

  found_addresses = Addresses.find_addresses(to_addresses)

  found_transactions = Enum.filter(block_with_transactions["transactions"], fn transaction ->
    transaction.to_address in found_addresses
  end)

  {:ok, transactions}
end

# insert transactions
defp insert_transactions(transactions) do
  Transactions.insert_transactions(transactions)
end

The problems

In the provided example, there are several challenges:

1. Complex Logic Tracking: Understanding the sequence of steps in the code can be quite challenging, particularly when the logic is intricate and involves multiple processes. This complexity can make it difficult for anyone reading the code to follow what’s happening.

2. Error Handling Complexity: Dealing with errors is not straightforward. To handle errors effectively, you need to be well-versed in all the potential error types that different functions could generate. This complexity can slow down the troubleshooting and debugging process.

3. Error Ambiguity: When various functions return the same type of error, it can be confusing to distinguish between them. This ambiguity can lead to misinterpretation of the source of errors and make it harder to find and fix issues.

4. Maintenance Difficulty: Keeping this code up-to-date is a significant challenge. If any of the functions change their structure or behavior, you have to modify the error-handling part of the code to match. This close connection between function signatures and error handling makes maintenance a demanding task.

5. Ensuring Data Consistency: A critical requirement is to execute the final two steps within the same database transaction. Failure to do so would result in data inconsistency within the database. This is vital for maintaining the integrity of the database’s contents.

6. Testing Complexity: Testing this code is not straightforward, particularly when it involves numerous interactions with an external service like an Ethereum node. Coordinating and validating these interactions can be time-consuming and prone to unexpected issues, making testing a complex and resource-intensive process.

7. Unstructrured response: Using raw maps instead of structured data types like structs can introduce unpredictability, especially when dealing with external services. In the context of our example, this unpredictability becomes evident when making requests to the Ethereum node. Any alterations in the responses from the external service can lead to unexpected consequences, making it crucial to consider the reliability of the data structures used.

Sagas to the rescue

The Saga design pattern helps keep data consistent in distributed systems when dealing with multiple small services. It’s like a story with a series of steps. At each step, it updates a service and sends a message to start the next step. If something goes wrong in a step, the saga performs actions to undo what happened in previous steps.

There is a great implementation of the saga pattern in elixir - sage. Let’s rewrite our initial example using it

def sync_transactions(block_number) do
  Sage.new()
  |> Sage.run(:block_hash, &fetch_block_hash/2, &handle_and_log_block_hash_error/3)
  |> Sage.run(:block_with_transactions, &fetch_block_with_transactions/2, &handle_and_log_block_error/3)
  |> Sage.run(:found_transactions, &find_transactions/2)
  |> Sage.run(:inserted_transactions, &insert_transactions/2)
  |> Sage.run(:additional_data, &insert_additional_data/2)
  |> Sage.transaction(Repo, %{block_number: block_number})
  |> case do
    {:ok, _final_effect, %{inserted_transactions: inserted_transactions}} -> {:ok, inserted_transactions}
    error -> error
  end
end

defp fetch_block_with_transactions(_effects, %{block_number: block_number}) do
  Application.get_env(__MODULE__, :eth_client).fetch_block_hash(block_number)
end

defp handle_and_log_block_hash_error(effect_to_compensate, _effects, _attrs) do
  Logger.error("Failed to fetch block hash")

  {:retry, retry_limit: 5, base_backoff: 10, max_backoff: 30_000, enable_jitter: true}
end

...

The updated version addresses the first five problems in the original code:

1. The code is now easy to understand. We gave our sage steps names that clearly describe what they do.
2. Dealing with errors is simpler because sage offers compensations. For instance, in the ‘block_hash’ step, you can observe that the request will be retried.
3. Error handling is clear and specific. Each compensation is associated with a particular step.
4. Maintenance is much easier compared to the initial version.
5. Sage enables bundling all its steps into a database transaction using the Sage.transaction function. If any step fails, the transaction will be undone.

For the remaining two problems we will use addtional external libraries

Mox

To solve the testing problem, we’re going to use the mox library, which you can find here. In the updated version of the fetch_block_with_transactions function, you’ll notice that we’ve started getting the client from the configuration. This is intentional.

We did this so that in our test code, we can use a mock client. We define how this mock client responds to different situations based on the responses we set up.

However, in the actual code used in production, we’ll use a real client. You can learn how to set up a mock client by checking the readme.

Parameter

The last issue is unstructred responses. To fix it we’re going to use the paramter library. It will validate responses from the external service - ethereum node. it’s going to protect us from api changes and unexpected responses by not failing in the actual domain logic but failing on the client module level.

For example, the api client is going to return the Block struct defined by the following code

defmodule Block do
  use Parameter.Schema

  param do
    field :block_hash, :string, required: true
    field :block_number, :integer, required: true
    ...
  end
end

The main features of this library:

• Schema creation and validation
• Input data validation
• Deserialization
• Serialization

Conclusion

In conclusion, we embarked on a journey to address common challenges faced by software engineers when dealing with external services in the context of cryptocurrency transactions, using the Elixir programming language. The initial example demonstrated complex logic, intricate error handling, ambiguity, and maintenance difficulties.

By introducing the Saga design pattern, implemented with the sage library, we streamlined the code, making it more understandable, simplifying error handling, and easing maintenance. This pattern also ensured data consistency by bundling all steps into a database transaction.

To tackle testing complexity, we incorporated the Mox library, enabling the use of mock clients for testing while preserving the ability to use real clients in the production code.

Lastly, to address the issue of unstructured responses, we integrated the parameter library. By validating responses with a defined data structure, we enhanced reliability and protected against unexpected changes in the external service’s API.

In embracing these tools and patterns, we’ve not only addressed the identified problems but also enhanced the overall robustness, maintainability, and testability of our code when interacting with external services in Elixir.