Today I learned how to use CubDB and some alternatives.
CubDB is a disk-based key-value database written in the Elixir language and it can be used as part of an Elixir application without any configuration.
Its API is so simple and intuitive to Elixir programmers. This type database is a perfect fit for Nerves-powered embedded Elixir projects.
Here are the versions Erlang and Elixir that I use as of writing.
elixir 1.12.1-otp-24
erlang 24.0.2
Playing with CubDB in IEx shell
With Elixir 1.12, we can play with it using Mix.install in the IEx shell.
# Start an Interactive Elixir shell.
❯ iex
iex> :ok = Mix.install([{:cubdb, "~> 1.0"}])
:ok
iex> {:ok, cubdb} = CubDB.start_link(data_dir: "tmp")
{:ok, #PID<0.166.0>}
iex> CubDB.put(cubdb, :word, "hello")
:ok
iex> CubDB.get(cubdb, :word)
"hello"
iex> CubDB.delete(cubdb, :word)
:ok
iex> CubDB.get(cubdb, :word)
nil
iex> ls "tmp"
0.cub data
It is very intuitive.
Starting as a child when the app starts
According to the official documentation:
Important: avoid starting multiple CubDB processes on the same data directory. Only one CubDB process should use a specific data directory at any time.
so it seems to be a good idea to name the database process. Also why not start the database when the application is starting?
In a Nerves project, we can write a file in /data directory. Don't forget the leading thrash (/). It is not data.
Because the root filesystem is read-only, we also add a read/write partition by default, called app_data and mounted at /data (the root user's home directory).
defmodule HelloNerves.Application do
...
@nerves_data_dir "/data"
def children(_target) do
[
# Children for all targets except host
{CubDB, [data_dir: @nerves_data_dir, name: CubDB]}
]
end
...
Then we can use CubDB anywhere in the app anytime.
CubDB.put(CubDB, :word, "hello")
It is worth noting that we will get an error when we cannot access the specified file.
iex> CubDB.start_link(data_dir: "/secret_dir", name: CubDB)
{:error, :erofs}
** (EXIT from #PID<0.105.0>) shell process exited with reason: :erofs
Alternatives to CubDB
The CubDB author is so kind that he lists some alternative solutions for similar use cases.
- ETS
- DETS
- Mnesia
- SQLite, LevelDB, LMDB, etc
- Writing to plain files directly
The list explains the key characteristics of each item succinctly, which is a great educational resource to me.
I also found a few Elixir wrappers of ETS:
Wrapping ETS and DETS
If all we want is a simple key-value store, we could just write a plain Elixir module that thinly wraps ETS and/or DETS. This might suffice in many situations.
defmodule HelloNerves.MemoryStore do
@ets_config [
{:read_concurrency, true},
{:write_concurrency, true},
:public,
:set,
:named_table
]
def create_table() do
:ets.new(__MODULE__, @ets_config)
end
def get(key) do
case :ets.lookup(__MODULE__, key) do
[] -> nil
[{_key, value} | _rest] -> value
end
end
def put(key, value) do
:ets.insert(__MODULE__, [{key, value}])
|> ok_or_error_response
end
def delete(key) do
:ets.delete(__MODULE__, :word)
|> ok_or_error_response
end
def delete_table do
:ets.delete(__MODULE__)
|> ok_or_error_response
end
defp ok_or_error_response(ets_result) do
if ets_result, do: :ok, else: :error
end
end
defmodule HelloNerves.FileStore do
def open(opts \\ []) do
data_dir = opts[:data_dir] || "tmp"
file = :binary.bin_to_list(Path.join(data_dir, "file_store"))
:dets.open_file(__MODULE__, file: file, type: :set)
end
def get(key) do
case :dets.lookup(__MODULE__, key) do
[] -> nil
[{_key, value} | _rest] -> value
end
end
def put(key, value) do
:dets.insert(__MODULE__, [{key, value}])
end
def delete(key) do
:dets.delete(__MODULE__, key)
end
def close do
:dets.close(__MODULE__)
end
end
But when something goes wrong, ETS argument error is very unfriendly. We might end up wanting more robust features.
# When table does not exist for example
** (ArgumentError) argument error
(stdlib 3.15.1) dets.erl:1259: :dets.delete(:my_table, :name)
Final thoughts
I think CubDB is one of the most intuitive to many Elixir programmers among all the solutions. It is written in Elixir. Although we have Erlang builtin solutions like ETS or DETS, we might need some cognitive overhead for understanging how they work unless we are already familiar with them. While there are some Elixir library that wrap ETS, I could not find anything similar for DETS that is actively maintained.
If one is not sure which one to use, CubDB can be a good default for file-based key-value store in Elixir. It can help us develop things quickly and it just works.
After playing with CubDB, ETS, DETS etc and ended up with this library DBKV, which is inspired by CubDB.
That's it!