Elixir

Elixir

Introduction

References

• Bal15 - Learning Elixir; Kenny Ballou

• Cai18 - Mastering Elixir; André Albuquerque, Daniel Caixinha

• Tho18 - Programming Elixir; Dave Thomas

• Cheat sheet: https://devhints.io/elixir

• https://livebook.manning.com/book/elixir-in-action/chapter-1/

• https://serokell.io/blog/learn-elixir

• https://elixir-lang.org/learning.html

• http://elixirkoans.io/

• http://www.elixircards.co.uk/

  • https://github.com/cleverbunny/elixir-flashcards

• Blogs around the net

  • https://medium.com/flatiron-labs/perfect-match-pattern-matching-in-elixir-9d49ced20b07

To read:

• https://elixir-lang.org/getting-started/typespecs-and-behaviours.html

• use and __using__ https://brooklinmyers.medium.com/using-use-usefully-in-elixir-and-phoenix-b59a5ea08ad2

Training

• https://exercism.org/tracks/elixir

• https://alchemist.camp/episodes

• https://www.openmymind.net/

Vocabulary

• BEAM - Bogdan/Björn’s Erlang Abstract Machine.
  • The Erlang Virtual machine.
  • BEAM is a single OS process.
  • Can contain millions of Erlang processes.
  • BEAM can straddle multiple logical CPUs.
  • BEAM runs on a single OS process, and then spawns threads under that process, by default, based on the number of cores available(,p101).
    • Each thread will run a scheduler that will preemptively manage the CPU time of each process.
    • The BEAM is a soft real-time system, and this preemptive scheduling promotes fairness and liveness.
  • The max processes by default, it's capped at around 256,000.
    • You can increase this limit by passing options with the --erl flag when starting elixir or iex , as in: iex --erl "+P 10000000"
      • http://erlang.org/doc/man/erl.html
• behaviours - define a set of functions that a module needs to implement, and are thus tied to a module.
• ephemeral - lasting for a very short time.
• ETC - Erlang Term Storage
• Expressions - each instruction will return something. (in contrast to statements, that does not necessarily return anything.)
  • we gain some distinct advantages if everything is an expression, namely, in terms of composability[Bal15,p16].
    • That is, with expressions, the concept of program flow and execution becomes more evident because we can compose parts of our programs in a way that is more readable and natural.
• GenServer -
• lexical scope - Lexical scoping, also known as static scoping, refers to setting the scope, or range of functionality, of a variable so that it may be called (referenced) from within the block of code in which it is defined. The scope is determined when the code is compiledlexical scoping.
  • A variable declared in this fashion is sometimes called a private variable.
• OTP - Open Telecom Platform. a set of libraries
• Protocols - Allows independent developers to add new type for handling by a module.
  • define a set of functions that a data type must implement(,p53).
  • This means that, with protocols, we have data type polymorphism, and we're able to write functions that behave differently depending on the type of their arguments.
• Statements - typically refer to instructions where the programmer specifies to the computer or runtime to perform some action
  • the code, itself, which instructs the performance of such actions, does not necessarily, nor inherently return anything[Bal15,p15].

Overview

Erlang

• BEAM
• Erlang processes are completely isolated from each other. They share no memory, and a crash of one process doesn’t cause a crash of other processes.
• Sharing no memory, processes communicate via asynchronous messages.
• Communication between processes works the same way regardless of whether these processes reside in the same BEAM instance or on two different instances on two separate, remote computers.
• Runtime is specifically tuned to promote overall responsiveness of the system.
  • dedicated schedulers that interchangeably execute many Erlang processes.
  • A scheduler is preemptive - it gives a small execution window to each process and then pauses it and runs another process.
  • Because the execution window is small, a single long-running process can’t block the rest of the system.
  • I/O operations are internally delegated to separate threads, or a kernel-poll service of the underlying OS is used if available.
    • This means any process that waits for an I/O operation to finish won’t block the execution of other processes.
• Garbage collection is specifically tuned to promote system responsiveness.
  • per-process garbage collection: instead of stopping the entire system, each process is individually collected as needed.

Elixir

• everything in Elixir is an expression–there are no statements.
  • always having a return value is very useful because you can chain functions together and define the program flow according to the values being returned.

Tools

• Dialyzer - Performs static analysis of the compiled .beam code(,p50)
  • Dialyxir lib - https://github.com/jeremyjh/dialyxir
  • The error 'Function xxxx/X has no local return.' usually just means there is an error, reported, further down the list that has to be fixed.
  • ':0:unknown_type\nUnknown type: XXX.some_type/0.' - Did you do an Alias for XXX ?
  • ':invalid_contract\The @spec for the function does not match the success typing of the function.' - is some default data out of range of the type def.
    • E.g. your default is 0, but the type is defined as 1..8.
    • add '| {0,0,0} to the spec.
• ExUnit - unit testing
• iex - interactive interpreter
  • 'b' - describe callbacks?
    • b Module.callback/arity
  • 'break!' - set breakpoint
    • e.g. 'break! StringHelper.palindrome?/1'
  • 'h' - provide help
    • e.g. 'h Enum'
  • 'i' - find out more information about a data type.
  • whereami
  • IEx.Helpers.flush/0 function, which will remove all the messages from the mailbox of the shell process and print them out to the terminal(,p106).
• hex - packaging thing
• mix - make/deploy tool
  • mix help
• observer - allows you to monitor your server and provides you with tons of useful information about the Erlang VM.
  • See: Chapter 11, Keeping an Eye on Your Processes

iex

See: http://blog.plataformatec.com.br/2016/03/how-to-quit-the-elixir-shell-iex/

• iex -S mix
• exit: press Ctrl-C twice.

mix

• mix help [TASK]
  • e.g. mix help new
• mix compile
• mix docs - generate html and epub documentation.
  • See: https://hexdocs.pm/ex_doc/Mix.Tasks.Docs.html
  • See also: the 'code documentation' section further down
• mix release - generate release files.
  • It will explain how to run the application
    • e.g. _build/dev/rel/my_server/bin/my_server start
• mix format [LIST_OF_FILES]
  • Alternatively, the project's root folder can have a .formatter.exs configuration file, indicating which files should be formatted.
• mix new sample_project --sup
  • create a OTP project with a supervisor.
• mix new elixir_drip --module ElixirDrip.Umbrella --app elixir_drip_umbrella --umbrella
• mix archive.install hex phx_new
  • mix phx.new.ecto elixir_drip --database postgres
    • See: https://hexdocs.pm/phoenix/Mix.Tasks.Phx.New.Ecto.html
• mix clean
• mix credo - (,p92)
  • it is recommended to have a configuration file by project to ensure each project is checked against the same static analysis criteria, independently of where the code is analyzed.
  • We can generate a local configuration file by running: mix credo.gen.config
• mix xref
  • mix xref warnings
  • mix xref graph --format dot - generate usage graphs
    • mix xref graph --format dot --source lib/elixir_drip/storage/storage.ex - get all the files that are referenced by the lib/elixir_drip/storage/storage.ex file.
  • mix xref graph --format dot --sink lib/elixir_drip/storage
    • --include-siblings option. When passed, this option considers all the umbrella dependencies of the project, so it will show the interdependencies between modules living in different umbrella applications as well.
• 'mix help compile.app' to learn about applications.

You can define your own aliases for 'mix' e.g. 'lint'(,p97):

cat mix.exs
defmodule ElixirDrip.Umbrella.Mixfile do
use Mix.Project
def project do
[
# ...
aliases: aliases()
]
end
defp deps do
[
# ...
]
end
defp aliases do
[
"lint": ["format", "credo"]
]
end
end

• If you need to create a Mix task from scratch, you can define a new module whose name starts with the Mix.Tasks. prefix. This module needs to use the Mix.Task module and define a run/1 function(,p97):

defmodule Mix.Tasks.MyCustomTask do
use Mix.Task
def run(_args), do: IO.puts("Hello from custom task")
end

Various commands/ops

• :application.which_applications
  • is defined as {Application, Description, Vsn} , where
    • Application is the name of the application,
    • Description is either the application name in string form or an explanatory text of the application,
    • Vsn is the loaded version of the application
• Process.list

Project

umbrella projects

• Umbrella projects - allows you to have more than one application under the same Elixir project.

  • Mix lets you achieve this by placing your individual applications under an apps folder in your umbrella project, while still allowing you to run each application separately inside the same project.
  • mix new elixir_drip --module ElixirDrip.Umbrella --app elixir_drip_umbrella --umbrella

• Keep in mind that you should always be able to use every feature of your project without having to pass through the web layer.

  • This will only be possible if your web interface is really thin.
  • A good rule of thumb is to design your project in a way that lets you use every feature from the comfort of an IEx session.

Project structure - general

• mix.exs - defines your Mix project and shows, among other things, which applications are started when you run your project.
• mix.lock - stores the specific versions of every dependency our project depends on and should be placed under your version control system along the rest of the code.
  • created when running mix deps.get
• test - All the tests you create should live here.
  • Every test file on the test folder has to end with _test.exs.

Project structure - ?

• config/config.exs - This configuration file is only considered by the current application running.
  • If your application is used by another application as a dependency, its configuration file won't be considered. If your application needs to fetch some configuration entries from the config/config.exs file, your documentation should be clear about that, because those configurations will need to be explicitly added to the configuration file of the main application.

Project structure - umbrella

• apps - the dir under which each app is created (with mix new my_app)

Project dependencies

See: https://hexdocs.pm/mix/Mix.Tasks.Deps.html

• If you just set the dependency name and version in the mix.exs file (that is to say, {:library, "~> 1.0.1"} ), it will be fetched from hex.pm.
  • If your dependency lives in a private repository, you should add the organization option to its dependency entry in the deps list(,p72).
  • git - {:gettext, git: "https://github.com/elixir-lang/gettext.git", tag: "0.1"}
  • only - this dependency is only active in the specified environment(s)
  • override - this version selection will then be the active one even in dependency packages,
    • e.g. http_client might only need ~> 2.0 og a lib, if there is an override ~> 3.0 for that lib, then the ~> 3.0 will override the ~> 2.0
  • path
  • in_umbrella - if your dependency lives under the same umbrella project, you can use it by setting the in_umbrella option to true.
  • If you just set the dependency name and version in the mix.exs file (that is to say, {:library, "~> 1.0.1"} ), it will be fetched from hex.pm . If your dependency lives in a private repository, you should add the organization option to its dependency entry in the deps list(,p72).

mix.exs

TODO Describe this whole

This is actually an elixir script in itself.

Shows which applications are started when you run your project[Cai18,p67]

There are a number of recognized functions

• project - Defines the current project, name, version, Elixir/Erlang requirements etc.
• application - specify which applications are started when you run your project[Cai18,p67]. Configuration for the OTP application[Bal15,64]
  • The return value of the application/0 callback in a Mix.Project is used to build the actual OTP-application manifest.
  • :mod is required if you want an “active” application that has its own supervision tree. It actually tells the runtime how to start and stop the application.(https://elixirforum.com/t/why-do-we-use-application-function-in-mix-exs/23834)
• deps - modules this app is dependent upon
• app_mod -

mix.exs deps

TODO git branches, min versions etc.

Basic

• Comment lines start with '#'

• Atom - ':foo' - An atom is a constant whose value is its name.

  • Names of modules in Elixir are also atoms. MyApp.MyModule is a valid atom, even if no such module has been declared yet.
  • :true is the boolean true.
  • true, false and nil are atoms where you can skip the leading ':'
  • They are not garbage collected
  • Atoms are kept in the atom table, and upon compilation, their value is replaced by a reference to their entry on this table.
    • This makes comparing atoms very efficient.
    • Since atoms are not garbage collected, don't create atoms dynamically from sources you can't control. For instance, if you're parsing a JSON response and creating a map out of it, don't use atoms for its keys—use strings instead
  • a name ended with ':' is a key in e.g. map.
  • once you define (read use) an atom, it will point to the same memory as all the other occurrences of that atom[Bal15, p20].
    • they are not garbage-collected,
    • nor are they mutable.
    • The memory used by atoms will never be freed up until the termination of the program.

• Aliases start in upper case and are also atoms:

• Strings in Elixir are UTF-8 encoded and are wrapped in double quotes

• boolean operators:

  • 'and', 'or', 'not' they require something that evaluates to a boolean (true or false) as their first argument. '1' is not 'true'
  • ||, &&, and ! - these can have any value on either side.
    • false || 11 - returns 11
    • || - it returns the first value that's truthy
    • && - it returns the first falsey value
    • in both cases, in the event the stated conditions never happen, they return the last value.
  • In Elixir, boolean expressions return the last evaluated value.
  • iex(12)> 1 || false
  • 1
  • iex(13)> true && 42
  • 42

• match operator (=): is like an assertion. It succeeds if Elixir can find a way of making the left-hand side equal the right-hand side(Mcc15,p15).

  • a = 1 Elixir can make the match true by binding the variable a to value 1.
  • Elixir will only change the value of a variable on the left side of an equals sign.
    • thus 1 = a will fail.

• comparison: Elixir comes with all the comparison operators we’re used to: ==, !=, ===, !==, <=, >=, <, and >.

  • For strict comparison of integers and floats, use ===:
    • 2 == 2.0 - true
    • 2 === 2.0 - false
      • Compares both the value and the type[Bal15, p22].

• naming:

  • Module, record, protocol, and behavior names start with an uppercase letter and are BumpyCase. ' All other identifiers start with a lowercase letter and is snake_case

• Sort order: number < atom < reference < function < port < pid < tuple < map < list < bitstring

• variables

  • Elixir allows you to rebind a variable, which is why the following works:
    • iex> a = 1
      • 1
    • iex> a = 7
      • 7
  • If you prepend a variable name with '_' the you indicate that there is no intention to using the variable.
    • If you use the variable you will get a compiler warning.

• ~ sigils - see: http://elixir-lang.github.io/getting-started/sigils.html

• ~s - create a string

• ~S - create a string but don't interpolate or escape characters

• ~c - create a char list

  • ~c(a charlist created by a sigil)

• ~C - create a char list but don't interpolate or escape characters

• ~r - create a regex

• @ - directives

  • @behaviour - TODO (I think it marks the start of @impl implementations?
  • @callback - defined 'Behaviour' (sort of abstract class kind of a thing)
  • @derive - TODO ?
  • @doc - document function?
  • @fallback_to_any - TODO ?
  • @impl [CALLBACK] - directive to mark the functions that are implemented as callbacks for a behaviour(,p52).
    • CALLBACK is optional and the name of the callback function that is being implemented(,p52).
  • @moduledoc - document module?
  • @optional_callbacks to mark one or more functions as optional when adopting a certain behaviour(,p52).
  • @spec - for specifying return types to functions(,p49)
    • @spec function_name(argument_type) :: return_type
  • @target - TODO ?
    • used for delegate/wrapping/mapping generic function calls to a generic .ex
    • See (,p87)
      • https://hexdocs.pm/elixir/Kernel.html
  • @type - define your own types to use in typespecs. Defining a type in a module will export it.
    • @type country_with_population :: {String.t, integer}
    • https://hexdocs.pm/elixir/typespecs.html
  • @typedoc - document type specification(,p49)
  • @typep - define a private/non-exported type spec.

• float - 64 bit

• << >> - binary

  • a group of consecutive bytes, << >> containing a comma separated list of bytes.
  • If you want to add a byte sequence from a number larger then 255 then you can specify the bit length to use.
    • e.g. << 256::16 >> becomes << 1, 0 >>
  • '<>' concatenate binaries.
  • (Tho18,p32)

• [] - list - see: https://hexdocs.pm/elixir/List.html

  • Values can be of any type, and can be mixed.
    • e.g. [1, 2, true, 3]
  • Get ':c' from list: Enum.at([:a, :b, :c, :d], 2)
  • '++/2' and '--/2' are used for concatenated or subtracted of lists.
  • head of the list 'hd/1' is the first element.
    • Does not change the list.
  • tail of the list 'tl/1' is the remainder.
    • Does not change the list.
  • Getting head or tail of an empty list throws an error.
  • Lists are stored in memory as linked lists, meaning that each element in a list holds its value and points to the following element until the end of the list is reached.
    • This means accessing the length of a list is a linear operation: we need to traverse the whole list in order to figure out its size.
  • 'in' operator check for the presense: 1 in [0, 1, 1, 2, 3, 5] : true
  • or if Enum.member?(["sip","sips"], sip_uri_tag) do
  • length/1 returns number of elements
  • Keyword lists are just special syntax for lists of two-element tuples: assert [foo: "bar"] == [{:foo, "bar"}] (koans)
    • Maps allow only one entry for a particular key, whereas keyword lists allow the key to be repeated(Thi18,p31).

• [] - array

  • add element to the end: [1,2,3] ++ [4]
  • add element to at the beginning: [0 | 1,2,3]
  • cut first element [ head | rest ]

• {} - tuple

  • used to store multiple items in a single variable
  • Every operation on a tuple returns a new tuple, it never changes the given one.
  • Tuples are stored contiguously in memory.
    • This means getting the tuple size or accessing an element by index is fast.
    • However, updating or adding elements to tuples is expensive because it requires creating a new tuple in memory.
  • elem(tuple,x) is zero based.
  • The general recommendation in Elixir is that tuples should hold up to four elements—anything more than that and you probably should be using another type.
  • If you put more than four elements in a tuple you might want to start looking at Maps or Structs(Tho18,p28)

• %{} - Map

  • similar to hashes in Ruby and dictionaries in Python.
  • To create a map, you enclose your key-value pairs in %{} , and put a => between the key and the value.
    • %{:name => "Gabriel", :age => 1}
    • %{name: "Gabriel", age: 1}
    • retrieve a value: map[:name] gives "Gabriel"
    • If the key is an atom you can do: map.name
  • fetch
    • When you try to fetch a key that doesn't exist in the map, using 'map[key]', a KeyError error will be raised
    • when using the map.key syntax it will return nil.
  • To update a key in a map, you can use
    • %{map | key => new_value}
    • %{map | age: 2} (if the key is an atom)
  • insert a new key
    • use the put function from the Map module: Map.put(map, :gender, "irrelevant")
  • has key
    • Map.has_key(dictionaryOfMessage, :via)
  • guard has key
    • def via_response(dictionaryOfMessage, current_message) when :erlang.is_map_key(:via, dictionaryOfMessage) do
  • Maps allow only one entry for a particular key, whereas keyword lists allow the key to be repeated(Thi18,p31).
  • In general, use keyword lists for things such as command-line parameters and passing around options, and use maps when you want an associative array(Tho18,p31).

• %StructName{} - Structs.

  • Structs are maps where the keys are pre-determined.
  • the StructName is (freaky) module.
  • none of the protocols implemented for maps are available for structs.Getting Started - Structs
  • are an abstraction built on top of maps.
  • See the 'Structs' sub-section further down.
  • struct_var.__struct__ will return the name of the module that defines the struct.

• #{} - ? can be used in a string as '${}' in bash.

• """ - heredoc. Documentation, as in python.

  • The closing delimiter of a heredoc string/charlist must be on its own line.

• binary - 0b1010

• octal -0o777

• hexadecimal - 0x1F

• |> - (pipe operator) This operator allows you to chain function calls.

  • This operator takes the term that's at its left, and injects it as the first argument on the function at its right.
  • i.e. the output, of the statement before the |>, becomes the first parameter in the function after the |>.
  • you should always use parentheses around function parameters in pipelines(Tho18, p64).

• '->' - same as '|>'

• size used when calculation will be constant time. (tuple?)

• length if the operation is linear (list)

• keyword list - is a list in which its elements have a specific format: they are tuples where the first element is an atom.

  • you can access values in a keyword list using the same syntax as you would in maps.
  • you can use the get function from the Keyword module.
  • Internally this still is a list of tuples–which means that searching for an item in a keyword list is O(n) , and not O(1) as in maps.
  • In a keyword list, contrary to what happens in maps, you can have more than one value for a given key.
  • Also, you can control the order of its elements. Usually, keyword lists are used to allow functions to receive an arbitrary number of optional arguments.

• Ranges - represent an interval between two integers.

  • To create a range, we use this: 17..21
  • Similar to what we do with a list, we can use the 'in' operator to check whether a number is between the start and the end of a range.
    • 19 in 17..21 : true

• MapSet - ?

• Port - A port is a reference to a resource. The Erlang VM uses it to interact with external resources, such as an operating system process.

  • maybe like a handle?
  • See. (,p62)

• PID - A PID is the type used to identify processes in the Erlang VM.

• Reference: A reference is a type created by the Kernel.make_ref function.

  • This functions creates an almost-unique reference, which gets repeated around every 2^82 calls.

Syntax

->

'->' designates a relation between the left side and the right side.

    case open_connection(parent, debug, state) do
      {:ok, new_state} ->
        debug = Utils.sys_debug(debug, :connected, state)
        module_init(parent, debug, new_state)

      {:error, error, _} ->
        state.reply_fun.({:error, error})
    end

• open_connection(parent, debug, state) - returns a tupple
  • of two entries, if the first entry is ':ok'
  • or two or three entries if the first entry is ':error'
• {:ok, new_state} - assign the value of the second tupple, returned from open_connection the 'new_state' variable
  • The right side can now use 'new_state' as a variable.

Pattern matching

• pattern matching, when it is successful, we always get back the term that was matched on the right-hand side.
  • in pattern matching, when the match succeeds, the right-hand side of the expression is returned.
    • Due to this behavior and taking into account that Elixir rebinds the variables on the left-hand side,
    • we can write expressions such as the following one, binding multiple variables to the same value:
      • iex> x = y = 100
      • both x and y are now 100.
• we don't care about the second element of the list? That's where the '_' (underscore) anonymous variable is convenient:
  • iex> [first, _, third] = ["X", "Y", "Z"]
  • you can never read from '_'
• Getting the head of a list:
  • [first | rest_of_list] = ["X", "Y", "Z"]
    • first: "X"
  • this technique is a fundamental piece to operate on a list using recursion.
• your pattern must only contain keys that exist on the map that's being matched on, otherwise MatchError will be raised
• Sometimes, you may want to match on the value of a variable, instead of rebinding it to a new value.
  • To this end, you can use the pin operator, represented by the ^ character:
    • iex> name = "Gabriel"
      • "Gabriel"
    • iex> %{name: ^name, age: age} = %{name: "Gabriel", age: 1}
      • %{age: 1, name: "Gabriel"}
    • iex> %{name: ^name, age: age} = %{name: "Jose", age: 1}
      • ** (MatchError) no match of right hand side value: %{age: 1, name: "Jose"}
  • When working with the pin operator, the variable you're using must already be bound, as it will not bind the variable in case it doesn't exist.
  • If you use the pin operator on a non-existent variable, you'll get a CompileError.
• Pattern matching on binaries
  • iex> <<first_byte, second_byte>> = << 100, 200 >>
  • parsing a packet from a given network protocol. By applying pattern matching to binaries, you can extract bits or bytes as necessary,
• Pattern matching on strings
  • Since strings are just binaries underneath, we can use the same strategy as in the preceding snippet:
    • iex> <<first_byte, second_byte>> = "YZ"
  • To match on strings, the best approach is to use the functions from the String module, such as starts_with? , ends_with? , or contains?

Build and release

• we usually name the file with the name of the module we're defining in it.

• Elixir source code files may have two extensions:

  • .ex files are compiled to disk (creating .beam files).
  • .exs files are compiled only in memory.
    • the test files that use the .exs extension (as there's no point in compiling them to disk)(,p31).

• elixirc - compiles files.

• iex string_helper.ex in our case. This will make Elixir compile your file, which will make our StringHelper module (and its functions) available in the IEx session

• If you're already inside the IEx session and want to compile a new file, you can use the c command, passing the filename as a string:

  • iex> c("examples/string_helper.ex")

no 'while'

• Elixir doesn't have 'while' or 'do ... while' constructs.
  • The way to iterate in Elixir is by using recursion, through functions that call themselves.

String

• Internally represented as an contiguous sequence of bytes.
• 'byte_size' of a string will count bytes not characters, non-ascii characters are two bytes.
• double quotes delimit strings.
• single quotes delimit charlists.
  • The convention in the Elixir community is to only use the term string when referring to the double-quote format.
  • This distinction is important, since their implementation is very different.
  • Functions from the String module will only work on the double-quote format.
• You can use ~s to create a string and ~c to create a charlist (their uppercase versions, ~S and ~C , are similar but don't interpolate or escape characters):
  • ~s(a string created by a sigil)

String Interpolation

iex> name = "Sean"
iex> "Hello #{name}"
"Hello Sean"

String Concatenation

iex> name = "Sean"
iex> "Hello " <> name
"Hello Sean"

Structs

• https://elixir-lang.org/getting-started/structs.html

• Structs are, underneath, bare maps with a fixed set of fields.

  • you can't add or remove keys from the map
  • you define a default value for each key
  • none of the protocols implemented for maps are available for structs.

• A struct is defined in it's own module,

  • each module can only have one struct
  • the struct is accessed via the module name

• you can't, out of the box, iterate on a struct, as it doesn't implement the Enumerable protocol(,p57).

• if you don't provide a default value when defining the fields of a struct, nil will be assumed as its default value(,p57).

  • You can define a structure combining both fields with explicit default values, and implicit nil values. In this case you must first specify the fields which implicitly default to nil:.

• you can enforce that certain fields are required when creating your struct(,p57).

  • You do that with the '@enforce_keys' module attribute, before the 'defstruct'.
  • If field is not provided, when creating the struct, an 'ArgumentError' will be raised(,p57)

• Structs alongside protocols provide one of the most important features for Elixir developers: data polymorphism.

defmodule User do
  @enforce_keys :name

  defstruct name: "",
            age: 0
end

• The name of the struct is 'User'
• You access 'name' as: 'User.name'

Examples

iex(1)> defmodule User do
...(1)>   defstruct name: "", age: 0
...(1)> end
{:module, User,
 <<70, 79, 82, 49, 0, 0, 6, 164, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 181,
   0, 0, 0, 18, 11, 69, 108, 105, 120, 105, 114, 46, 85, 115, 101, 114, 8, 95,
   95, 105, 110, 102, 111, 95, 95, 10, 97, ...>>, %User{age: 0, name: ""}}
iex(2)> usera =  %User{name: "baby"}
%User{age: 0, name: "baby"}
iex(3)> userb = %{usera | name: "tut", age: 42}
%User{age: 42, name: "tut"}

pipeline operator |>

The pipeline operator |> takes the result of the previous expression and feeds it to the next one as the first argument.

def process_xml(model, xml) do
  model
  |> update(xml)
  |> process_changes
  |> persist
end

Does the same as:

process_xml(Model, Xml) ->
  Model1 = update(Model, Xml),
  Model2 = process_changes(Model1),
  persist(Model2).

List

TODO sort these '###' sections

• https://inquisitivedeveloper.com/lwm-elixir-39/
• https://hexdocs.pm/elixir/1.13/List.html

List.keytake

• List.keytake(list_of_tuple, value, tuple_index)

  • list_of_tuple - list of tuples
    • e.g. [:a 8, :b 9, :c 10]
  • value - value to look for, in each of the tuple, at the tuple_index position of each tuple
    • e.g. 8
  • tuple_index - the index inside the tuple
    • e.g. 1
  • e.g. List.keytake([:a 8, :b 9, :c 10], 8, 1)
    • will look for 8 at the second element of each tuple (looking at 8,9,10)

• If a match occurs return a tuple with two elements;

  • the tuple that matched is returned in the first index of a tuple
  • the list without the tuple is returned in the second index of that same tuple.

Map

• https://inquisitivedeveloper.com/lwm-elixir-49/
• https://hexdocs.pm/elixir/1.13.4/Map.html

Map.pop

Map.put

• Add key-value, if key does not exist
• Update key-value, if the key does exist

Process

• spawn(FUNCTION_NAME) - spawns a process that runs that function(,p103)
  • The caller of the spawn function resumes its work as soon as the process is created.
  • All variables are copied/rebound to the new process.
    • So x in the parent process is independent of the x in the spawned process.
• IO.puts "I'm running in process #{inspect(self())}"
• Task - abstraction provided by Elixir(,p110)
• No looping - to make processes run in a loop you use recursion.
  • Particularly, we have to use endless-tail recursion, to ensure that no additional memory is consumed as a result of the process calling itself over and over again.
  • If the last thing a function does is call another function, or itself, the usual stack push doesn't occur and instead a simple jump is performed.
    • This means that using tail recursion won't consume any additional memory or cause a stack overflow.

Detect a process exit

• Link - When two processes are linked, and one exits, the other one will receive an exit signal, which notifies us that a process has crashed(,p118).
  • One link always contains two processes, and the connection is bidirectional.
  • One process may be linked to an arbitrary number of other processes, and there's no predefined limit in the system to the number of created links.
  • By default, unless the exit reason is :normal , when a process receives an exit signal, it's also terminated, along with the linked process.
    • When a process is trapping exits, it's not terminated upon receiving an exit signal. Instead, this exit signal is delivered, in the form of a message, to the mailbox of that process. That way, a trapping process can receive this message and act on it accordingly(,p120).
  • This technique is very powerful and may come in handy if, for instance, we want to always ensure that a certain group of related processes are killed as a whole, leaving no dangling processes behind.
• Monitor - In the event that the monitored process exits, the monitoring process receives a message with the following format: {:DOWN, monitor_reference, :process, from_pid, exit_reason}(,p121)

Message passing

• These messages contain Elixir terms–basically anything you can store in a variable can be sent in a message(,p104).
• The mailbox of a process is unbounded–however, in practice, it is bounded by the available memory.
• message passing is asynchronous.
• If you only want to process a certain kind of messages, you can use pattern-matching on the clauses provided to receive:

iex> send(self(), {:result, 2+2-1})
{:result, 3}
iex> receive do
...>
{:result, result} -> IO.puts "The result is #{result}"
...> end
The result is 3
:ok

• if a message doesn't match any clause in the receive block, it will stay at the mailbox to be processed at a later stage.

  • if a process is frequently receiving messages that are never matched on receive blocks, its mailbox will grow indefinitely and this may ultimately cause the BEAM to crash, due to the excessive memory usage.
    • To avoid it add _ -> {:error, :unexpected_message}(,p105)

• receive will block waiting for messages, if there are no messages in the mailbox.

  • use after to continue after some time has passed(,p106).
    • after can be 0, which means no waiting.
    • see: https://samuelmullen.com/articles/elixir-processes-send-and-receive/

iex> receive do
...> message -> IO.puts "Message is #{message}"
...> after
...> 2000 -> IO.puts "Timed out. No message received in 2 seconds."
...> end
Timed out. No message received in 2 seconds.
:ok

Naming processes

• You can register a process PID to an Atom[Bal15,p142]
  • Process.register(pid, :kv)
    • Now you can send messages via: send :kv {:put, :a, 42, self}
  • I assume you can use this so you app code can always use nameing(atom) when sending data, and then it is only during init that the PID has to be registered.
    • and the the PID doesn't have to be passed around to all other processes that needs it.

Registry

• {:via, Registry, {ElixirDrip.Registry, {:cache, <media_id>}}}
  • The ':via' tuple informs the GenServer.start_link/3 function how it should register the soon-to-be spawned process.
  • Indicate that a local register must be used instead of the ':global'
  • See: Process registry in Elixir: a practical example
    • via tuple is basically a way to tell Elixir that we will use a custom module to register our processes.
    • And the tuple always follow the same format: {:via, module_name, term}

If you want to use ':via' and implement your own register, then it must support the four base functions[Cai18,p178], [Cai18,p176]:

• register_name/2
• unregister_name/1
• whereis_name/1
• send/2

See also the source code: https://github.com/elixir-lang/registry/blob/master/lib/registry.ex

Supervisors

A supervisor is a process whose sole responsibility is to supervise other processes[Cai18,p123].

• https://hexdocs.pm/elixir/Supervisor.html
• http://erlang.org/doc/design_principles/des_princ.html

After having its process created, a supervisor will

• configure itself to trap exit signals[Cai18,p123].

• Then, create links between itself and each one of its children[Cai18,p123].

  • With this in place, if a child crashes, the supervisor will receive an exit signal, allowing it to react to this event, according to the configuration that was provided when it started[Cai18,p123].
  • Supervisors use links rather than monitors because the bidirectional communication means that if a supervisor crashes, all of its children will also be immediately terminated, which is useful to avoid dangling processes in a system[Cai18,p123].

• By default, a supervisor allows for three restarts in a five-second interval.

  • If this limit is exceeded, the supervisor will terminate itself, along with all of its children. Having the supervisor terminate itself in these conditions is very important, as it allows the error to propagate in a supervision tree(Cai18,p127).

• You can specify child config at the supervisor definition of the configuration can be a map in the child module (def child_spec(_) do)

  • You may not have to define a child_spec/1 function on every occasion. For instance, the Agent and Task modules.

Supervisor integration

• To plug a module into the supervisor's behaviour, we only need to implement one callback: init/1(Cai18,p124) implemented in the child.

  • This callback will provide the specification of the processes that are started by this supervisor, along with some configuration. To interact with supervisors, we use the Supervisor module provided by Elixir.

• to implement 'supervisor' functions in a child add use Supervisor to the child code.

Other example of how the child code could look:

See also: https://stackoverflow.com/questions/46994003/how-do-i-properly-return-from-a-worker-processs-init

# If you use GenServer, then this function is not needed, since it is part of GenServer
  def child_spec(opts) do
    %{
      id: __MODULE__,
      start: {__MODULE__, :start_link, [opts]},
      type: :worker,
      restart: :permanent,
      shutdown: 500
    }
  end

  def start_link(args) do
    IO.puts("DDD start_link()")
    {:ok, self() }
  end

Supervision trees

(,p133)

• If designed properly, supervision trees allow you to minimize the effects of an error on a certain part of your system, allowing the rest to operate as usual.
  • The point of supervision trees is to first try to recover from the error locally, affecting only the truly required processes.
  • If then this doesn't work, because the process being restarted keeps crashing, the supervisor itself will crash, moving up in the tree, and delegating the next action to be taken to the supervisor's supervisor.
• Usually, root supervisors are defined in the application.ex file, which is the starting point of the application.
  • Note that if you use the --sup flag when creating a new project with Mix, this is automatically done by the generators(,p134).

Supervisor children

There are three ways to add a child to a supervisor: https://hexdocs.pm/elixir/1.12/Supervisor.html

XXX

code documentation

See: https://hexdocs.pm/ex_doc/Mix.Tasks.Docs.html

1. edit mix.exs
2. mix deps.get
3. mix docs
4. chromium doc/index.html

• to display the documentation

Edit mix.exs

Add the follwoing to the def project do section

   # Docs
   name: "My App",
   source_url: "https://github.com/USER/PROJECT",
   homepage_url: "http://YOUR_PROJECT_HOMEPAGE",
   docs: [
           main: "MyApp", # The main page in the docs
           logo: "path/to/logo.png",
           extras: ["README.md"]
         ]

Add the following to the defp deps do section

      {:ex_doc, "~> 0.24", only: :dev, runtime: false}

if mix docs fails with '** (Mix) The task "docs" could not be found. Did you mean "do"?' then it is because you forgot to add :ex_doc to the mix.exs file.

Functions

• functions in Elixir are a type as well; in fact, they are a first-class citizen, as they can be assigned to a variable and passed as arguments to other functions(,27)
• There is no return keyword in Elixir, the return value of a function is the value returned by its last expression.
• Functions must be created inside modules
• Function names, like variable names, start with a lowercase letter,
  • if they contain more than one word, they are separated by underscore(s).
  • They may end in ! and ? .
    • The convention in the Elixir community is that
      • function names ending in ! denote that the function may raise an error,
      • function names ending in ? indicate that that function either returns true or false.
  • Note that unlike anonymous functions, we don't need to put a dot between the function name and the parenthesis when calling named functions. This is deliberate and serves to explicitly differentiate calls to anonymous and named functions(,p31).
• A version of a function is referred to in Elixir as a "function clause". A single function can have multiple clauses.
• Named functions in Elixir are identified by:
  • their module name,
  • the function's own name,
  • their arity.
    • The arity of a function is the number of arguments it receives.
    • e.g. Helpers.StringHelper.palindrome?/1 takes on argument.
    • This concept is important because functions with the same name but different arities are, in effect, two different functions.
• The common pattern in Elixir is to have lower-arity functions being implemented as calls to functions of the same name but with a higher arity(,p32).
• def defines a public function.
• defp defines a private function.
• optional typespec return value
  • function_name(argument_type) :: return_type (,p49)
• default arguments.
  • We do that by using the '\' operator after argument name, followed by the default value:
    • 'def emphasize(phrase, number_of_marks \ 3) do'
  • This will generate two functions with the same name and different arities.
• Elixir will search from top to bottom for a clause that matches(,p34).
• we can use pattern matching in named functions.

def emphasize(_phrase, 0) do
"This isn't the module you're looking for"
end

• guard clauses, which extend on the pattern matching mechanism and allow us to set broader expectations on our functions.
  • To use a guard clause on a function, we use the when clause after the list of arguments.
    • e.g. def palindrome?(term) when is_bitstring(term) do
    • def palindrome?(_term), do: {:error, :unsupported_type} (must come after the 'when' claused function definition.
  • See also: https://hexdocs.pm/elixir/guards.html
  • and: (Elixir Guards)[https://kapeli.com/cheat_sheets/Elixir_Guards.docset/Contents/Resources/Documents/index]
• defguard - construct, which allows us to define clauses that can be reused.
  • e.g. defguard is_string(term) when is_bitstring(term)
• File.stat! , which works similarly to File.stat , but, instead of returning a {:ok, result} tuple, it returns the result itself.

All functions are, by design, expected to return values

Function parameters

Example of a function parameter set (for stun response message):

• {:stun_binding, :response = _class, _address, _port, stun_msg}
  • first parameter must be the atom: ':stun_binding'
  • second parameter must be the atom ':response' (the '= _class' is there to inform the reader that is is the 'class' of the stun that it is picked from.
    • since the '_' in front of 'class' means it will not be used.
  • third parameter: ignored
  • fourth parameter: ignored
  • rest of the data put into 'stun_msg'

Function overload

  # If compare_test() is not called with two identical values, then this function instance will not be used.
  def compare_test(alpa, alpa) do
    IO.puts("DDD compare_test() with vars")
  end

  def compare_test(_, _) do
    IO.puts("DDD compare_test() no care")
  end

recursion

• The Erlang runtime employs tail-call optimization whenever it can, which means that a recursive call won't generate a new stack push.
  • For the runtime to do this optimization, you have to ensure that the last thing our function does is call another function (including itself)–or, in other words, make a tail call(,p39).
  • Note that there's a trade-off here: On one hand, this optimization is important when dealing with large collections (since function calls don't consume additional memory);
    • on the other hand, code that doesn't use this optimization is usually easier to read and comprehend, as it's usually more concise.
• When doing recursion, consider the advantages and disadvantages of each solution.
• First look for the simplest possible case, one that has a definite answer. This will be the anchor(Tho18,p56). Then look for a recursive solution that will end up calling the anchor case(Tho18,p56).

def multiply(list, accum \\ 1)
def multiply([], accum), do: accum
def multiply([head | tail], accum) do
 multiply(tail, head * accum)
end

Anonymous functions, usually called lambdas

• delimited by the keywords 'fn' and 'end:'
• Parentheses around arguments of an anonymous function are optional.
• anonymous functions can also access variables from the outer scope.
  • the variable can be bound to another value, and our function will still hold a reference to the value that the variable had when the anonymous function was defined.
    • This is usually called a closure: the function captures the memory locations of all variables used within it.
    • Since every type in Elixir is immutable, that value residing on each memory reference will not change.
    • However, this also means that these memory locations can't be immediately garbage-collected, as the lambda is still holding references to them(,29).
• capture operator (represented by & )
  • &n will represent the n th argument of the function.
    • Similar to what happens in the fn notation, the parentheses are optional.
    • However, it's better to use them, as in a real-world application, these lambda functions become hard to read without them. *the capture operator can also be used with named functions.

This defines 'my_function/2'

my_function = fn a, b -> a + b end

• my_function.(1, 2)
  • returns: 3
  • the '.' between the variable and parentheses is required to invoke an anonymous function.
    • The dot ensures there is no ambiguity between calling the anonymous function matched to a variable add and a named function add/2.
  • is_function(my_function) returns true.
  • is_function(my_function, 1) returns false.

An anonymous function can also have multiple implementations

An anonymous function can also have multiple implementations , depending on the value and/or type of the arguments provided. Let's see this in action with an example:

iex> division = fn
...>   (_dividend, 0) -> :infinity
...>   (dividend, divisor) -> dividend / divisor
...> end

The & and anonymous functions

• The & operator converts the expression that follows into a function(Tho18,p49).
  • Inside that expression, the placeholders &1, &2, and so on correspond to the first, second, and subsequent parameters of the function.
  • So &(&1 + &2) will be converted to fn p1, p2 -> p1 + p2 end.
• There’s a second form of the & function capture operator(Tho18,p50).
  • You can give it the name and arity (number of parameters) of an existing function, and it will return an anonymous function that calls it.

Protocols?

• https://elixir-lang.org/getting-started/protocols.html
  • define the protocol using defprotocol
    • its functions and specs may look similar to interfaces or abstract base classes in other languages.
  • add implementation using defimpl.
    • defimpl PROTOCOL, for: TYPE do
  • The output is exactly the same as if we had a single module with multiple functions.

Example

defprotocol MyProtocol do
  @spec mysize(t) :: int.t()
  def mysize(input)
end

defimpl MyProtocol, for: BitString do
  def mysize(string), do: byte_size(string)
end

Call the above: MyProtocol.size("test")

Enumerable

• The Enum module is referred to as being eager.
  • This means that when processing a collection,
  • this module will load the entire collection into memory.
  • Furthermore, if you have a chain of functions you want to apply to a collection, the Enum module will iterate through your collection as many times as the functions are applying to it.

Stream

See also: Streams are a really nimble way to process large, or even infinite, collections.

• lazy processing

• Streams are a really nimble way to process large, or even infinite, collections.

• When working with lazy enumerables,

  • the entire collection never gets loaded into memory,
  • the computations aren't made right away.
    • The results are produced as they are needed.
  • the functions that will be applied on it are saved into a structure (along with the collection we're working on). We can then pass this structure into the next function, which will further save a new function to be applied to our list.

• It's certainly great for very large collections, or even if you have a big chain of functions being applied to a collection and only want to traverse it once.

• However, for small or even medium collections, the Stream module will perform worse, as you're adding a lot of overhead, for instance, by having to save the functions you'll apply instead of applying them right away

• how do we make it return the result we're expecting? Just treat it as a regular (eager) enumerable, by applying a function from the Enum module, and it will start to produce results.

Collectable

is used to create a new structure from the values of an existing collection(,p41).

Control flow

• In Elixir, they aren't used as often as in traditional imperative languages, because we can fulfill our control-flow needs, using a mix of pattern matching, multi-clause functions, and guard clauses.

if

If you are using 'if' in your code, you are possibly doing something wrong. Check that you are using function clauses and pattern matching, where appropriate.

if <expression> do
# expression was truthy
else
# expression was falsy
end

On a single line. if <expression>, do: # expression was truthy, else: # expression was falsy

'else' is optional.

• The only things that er falsy are:
  • nil
  • 0

unless

unless <expression> do
# expression was falsy
else
# expression was truthy
end

cond

• cond can be seen as a multi-way if statement, where the first truthy condition will run its associated code.
• Different from 'switch' in the sense that switch can only have cases on one type, like integer or tupple
  • cond can have completey different types or unrelated comparisons.
• You have to be very diligent with the order of the entries.

e.g. (figuring out if the planet is habitable

cond do
  planet == "earth" -> :livable
  vehicle == "spaceship" -> :livable
  goldilocks_place() -> :livable
  true -> :uninteresting
end

x = 5
cond do
  x * x == 9 -> "x was 3"
  x * x == 16 -> "x was 4"
  x * x == 25 -> "x was 5"
  true -> "none of the above matched"
end
"x was 5"

case

• case accepts an expression, and one or more patterns, which will match against the return value of the expression.
• These patterns may include guard clauses.
• These patterns are matched (from top to bottom), and will run the code associated with the first expression that matches.

case Enum.random(1..10) do
  2 -> "The lucky ball was 2"
  7 -> "The lucky ball was 7"
  _ -> "The lucky ball was not 2 nor 7"
end
"The lucky ball was not 2 nor 7"

with

• It allows you to use pattern matching on the return value of each expression,
• running the do block if every pattern matches.
• If one of the patterns doesn't match, two things may happen:
  • If provided, the else block will be executed;
  • otherwise, it will return the value that didn't match the expression.
• In practice, with allows you to replace a chain of nested instances of case or a group of multi-clause functions.
• Moreover, you can control how to handle each error separately, using pattern matching inside the else block.

Exceptions

(,p47)

• raise - raise an error
  • If you provide only one argument, it will raise a RuntimeError , with the argument as the message.
  • If you provide two arguments, the first argument is the type of error, while the second is a keyword list of attributes for that error (all errors must at least accept the message: attribute).
• rescue - construct (you can rescue from a try block or from a whole function, pairing it with def ).
  • You define patterns on the rescue clause.
    • You can use _ to match on anything.
  • If none of the patterns match, the error will not be rescued and the program will behave as if no rescue clause was present.
  • you can also pass an 'else' and/or an 'after' block to the try/rescue block.
  • The 'else' block will match on the results of the try body when it finishes without raising any error.
  • The 'after' construct, it will always get executed, regardless of the errors that were raised. This is commonly used to clean up some resources (closing a file descriptor, for instance).
• use throw + catch
  • using 'throw' instead of 'raise' , and 'catch' instead of 'rescue'.
  • this should be used in situations where it is not possible to retrieve a value unless by using throw and catch.
    • See: http:/​ / ​ elixir-​ lang.​ github.​ io/​ getting-started/​ try-​ catch-​ and-​ rescue.​ html

Modules

• alias - is used to create aliases for other modules.
  • alias Helpers.StringHelper, as: StrHlp
    • lets you reference functions via 'StrHlp'
  • alias Helpers.StringHelper
    • lets you reference functions via 'StringHelper'
• require - This ensures that the macro definitions are available when your code is compiled(Tho18,p67).
• import - When we import a given module, we're also automatically requiring it.
  • import String, only: [reverse: 1].
    • Now we can just use reverse directly in our module
    • You provide a list of function name, arity pairs(Tho18,,p67)
      • e.g. import List, only: [ flatten: 1, duplicate: 2 ]
  • except: to import all but a given number of functions from a module.
  • only: and except: also accept :modules and :functions.
  • always try to pass the only: option when using import.
• use , which is a macro. This is commonly used to bring extra functionality to our modules.
  • Beneath the surface, use calls the require directive and then calls the using/1 callback, which allows the module being used to inject code into our context.

defmodule ModuleName do
end

• Modules in Elixir may contain attributes.

  • They're normally used where you'd use constants in other languages.
  • You define a module attribute with the following syntax:
    • @default_mark "!"
  • The attribute only exists at compile time, as it's replaced by its value during this process.
  • you can register attributes, which makes them accessible at runtime. For instance, Elixir registers the @moduledoc and @doc attributes, which can be used to provide documentation for modules and functions, respectively.
  • See also (Tho18, p67)

• Internally, module names are just atoms(Tho18,p68).

  • When you write a name starting with an uppercase letter, such as IO, Elixir converts it internally into an atom of the same name with Elixir. prepended(Tho18,p68).
    • So IO becomes Elixir.IO(Tho18,p68).

• nested modules are an illusion - all modules are defined at the top level. When we define a module inside another, Elixir simply prepends the outer module name to the inner module name, putting a dot between the two(Tho18, p66).

Module directives

Modules have three directives

• the executed as your program runs,

  • the effect of all three is lexically scoped; starting at the point the directive is encountered, and stops at the end of the enclosing scope(Tho18, p66)
  • a directive in a function definition runs to the end of the function(Tho18, p66).

• alias directive - creates an alias for a module(Tho18, p67).

  • to cut down on typing(Tho18, p67).
  • alias My.Other.Module.Parser
  • alias My.Other.Module.SuperRunner, as: Runner

• import directive - brings a module’s functions and/or macros into the current scope(Tho18, p66)

  • eliminating the need to repeat the module name time and again(Tho18, p66)
  • import List, only: [ flatten: 1, duplicate: 2 ]

• require directive - ensures that the macro definitions are available when your code is compiled(Tho18, p67).

Module attributes

@name value(Tho18, p67)

• works only at the top level of a module—you can’t set an attribute inside a function definition.
• You can set the same attribute multiple times in a module(Tho18, p67).
  • If you access that attribute in a named function in that module, the value you see will be the value in effect when the function is defined(Tho18, p68).

Sub-modules

• Sub-modules are for the convenience of the developer.
  • Behind the scene the interpetrer or compiler??? splits sub-modules out to modules and then put the proper alias in place. (So the developer wont be the wiser)
• Sub-modules are not real. They are actually full modules, that run in their own beam.
• So e.g. @type's defined in the parent module, are not available in the sub-module, unless you use the parent type with the full parent module path.

Behaviours

• Behaviours provide a way to describe a set of functions that have to be implemented by a module, while also ensuring that the module implements the functions in that set.
  • If you come from an object-oriented programming background, you can think of behaviours as abstract base classes that define interfaces. * After declaring the behaviour, we can then create other modules that adopt this behaviour.
• A behaviour creates an explicit contract, which states what the modules that adopt the behaviour need to implement.
• A behaviour is created using the @callback directive inside a module, providing a typespec for each function this behaviour contains.

Definition:

defmodule Presenter do
  @callback present(String.t) :: atom
end

Implementation/Usage:

defmodule CLIPresenter do
  @behaviour Presenter
  @impl true
  def present(text) do
    IO.puts(text)
  end
end

Protocols

Seems to be like behaviors but for data sets?

See: http://elixir-lang.github.io/getting-started/protocols.html

defprotocol Size do
  @doc "Calculates the size of a data structure"
  def size(data)
end

Testing

• Run specific test module: MIX_ENV=dev mix test test/my_module.exs
• Run a speficic test within a module MIX_ENV=dev mix test test/my_module.exs:14
  • With '14' being the line the test starts on.

Unit testing

See: Chapter 9 , Finding Zen Through Testing

• https://medium.com/blackode/elixir-testing-running-only-specific-tests-746cfc24d904
• https://hexdocs.pm/ex_unit/ExUnit.html

Running unit tests

Running all unit tests

• MIX_ENV=test mix test

Creating a new unit test

• test files ends in '_test.exs'
• test files are under the 'test' subdirectory of the project.
• The directory structure is the same in both the source and 'test' directories.
  • e.g.
    • source: apps/elixir_drip/lib/elixir_drip/storage/media.ex
    • test: apps/elixir_drip/test/elixir_drip/storage/media_test.exs

Example from: 'Mastering Elixir'

defmodule ElixirDrip.Storage.MediaTest do
  use ExUnit.Case, async: true

  @subject ElixirDrip.Storage.Media

  test "is_valid_path?/1 returns {:ok, :valid} for a valid path" do
    path = "$/abc"
    assert {:ok, :valid} == @subject.is_valid_path?(path)
  end

end

• '@subject' used for "linking" to the source directory.

Writing unit tests

ExUnit info

• ExUnit.CaptureLog.capture_log([level: :info], fun)
  • store info and above in the log.
  • I guess all levels are prevented from showing

Integration testing

Application lifecycle

• when a process dies, the "supervisor" is informed about it
  • TODO investigate how this is handled, and by whom
  • What happens if there is no supervisor? perhaps cai18 has some insights into that.

Libraries

Logging

• https://timber.io/blog/the-ultimate-guide-to-logging-in-elixir/

• It seems like there is a single logger process for all process in the application.

  • It seem like the Logger is a GenServer

Change the the global logging level from an application

• https://christopherjmcclellan.wordpress.com/2018/06/02/how-to-change-elixir-log-levels-for-a-running-application/

require Logger
Logger.configure(level: :debug)

GenServer

Introduction of GenServer

GenServer References

• https://hexdocs.pm/elixir/GenServer.html
• https://elixir-lang.org/getting-started/mix-otp/genserver.html
• https://blog.appsignal.com/2018/06/12/elixir-alchemy-deconstructing-genservers.html

GenServer vocabulary

• Client - The application part that calls the GenServer implementation.
• Server - The server that is derived from the GenServer

TODO - does a call to 'handle_call/3' instantiate a process that dies when it has replied?

GenServer overview

• A GenServer is implemented in two parts:
  • the client API
  • the server callbacks

You can either combine both parts into a single module or you can separate them into a client module and a server module.

• The client is any process that invokes the client function.

• The server is always the process identifier or process name that we will explicitly pass as argument to the client API.

• The whole GenServer instantiation is a single process.

  • The process is kept running(which is how the 'state' is preserved between calls.

• The process handles a single message at a time, and there is no context switch.

  • Each message is process completely before the next message will be handled.

• by default handle_call/3 are only allowed to run for max 5 sec. (the max time can be changed)

• Both the client part and the server part resides in the same module and process.

• The 'State' that is being handed around is due to elixir not having 'global' variables.

• when the processing would be long running, we can use 'handle_continue/2' to do the further processing.

• the 'state' can be any type of variable 'map, list, etc'

I wonder if this is the base concept for a genserver

defmodule PingPong do
  def start_link do
    spawn_link(fn -> loop() end)
  end

  defp loop do
    receive do
      {:ping, sender} ->
        send sender, {:pong, self}
    end
    loop
  end
end

GenServer Callback functions

For both cast and call

• Requests are often specified as tuples, in order to provide more than one "argument" in that first argument slot.
• It is common to specify the action being requested as the first element of a tuple, and arguments for that action in the remaining elements.

See: https://hexdocs.pm/elixir/GenServer.html

• init/1
  • Called from GenServer.start_link/3 (start_link/3 blocks, until init/1 is done)
  • Parameters
    • init_arg - second argument given to start_link/3
  • Return:
    • {:ok, state}
      • Where 'state' is the data you want to transfer between the server functions.
      • 'state' can by any? type.
    • if the first tuple isn't ':ok' then the process will not be started.
• handle_cast/2 - asynchronous
  • Called from the Genserver.cast/?
  • The OTP framework takes care of the internal main loop for each process, dispatching messages to the function[Bal15,p171].
  • Parameters
    • request -Typically, an atom specifying the message type or action the server should perform[Bal15,p171].
    • state - current state(the internal data.
  • Returns: {:noreply, new_state}.
    • :noreply - indicates that the server should not send a reply back to the client(since this is an asynchronous call).
      • {:stop, reason, new_state} stops the loop and terminate/2 is called with the reason reason and state new_state.
        • The process exits with reason reason.
    • new_state - is the new server state. (TODO is this held by the core GenServer process?)
    • The return date is consumed by the OTP framework[Bal15,p171].
• handle_call/3 - synchronous; Caller will block until a reply has been delivered.
  • The OTP framework takes care of the internal main loop for each process, dispatching messages to the function[Bal15,p171].
  • Parameters
    • request -
    • _from - the process from which the request is received
    • state - current state
  • Returns: {:reply, reply, new_state}.
    • :reply - indicates that the server should send a reply back to the client.
      • Returning {:noreply, new_state} does not send a response to the caller and continues the loop with new state new_state. The response must be sent with reply/2.
      • {:stop, reason, reply, new_state} stops the loop and terminate/2 is called with reason reason and state new_state.
        • Then the reply is sent as the response to call and the process exits with reason reason.
    • reply - is what will be sent to the client
    • new_state - is the new server state. (TODO is this held by the core GenServer process?)
    • [timeout] - optional; TODO
    • [{:continue continue}] - handle_continue/2 is invoked immediately after with 'continue' as the first parm value.
• handle_continue/2 - It is useful for performing work after initialization or for splitting the work in a callback in multiple steps, updating the process state along the way.
  • Parameters
    • continue -
    • state
  • Returns - Same as handle_cast/2.
• handle_info/2 - handle all other messages
• terminate/2 - Invoked when the server is about to exit. It should do any cleanup required.

GenServer client operations

• start_link/1 - starts a new GenServer instance, passing a list of options.
  • 'routes' to 'init/1' in the GenServer callback implementation.
  • Parameters
    • The module where the server callbacks are implemented.
    • The initialization arguments.
    • A list of options which can be used to specify things like the name of the server.
  • returns
    • {:ok, pid}[Bal15,p172]
• call/2 -
  • 'routes' to handle_call/3 in the GenServer implementation.

PID vs name reference

See [Bal15,p174]

Mix Bag

Regular expression

Regular Expressions works same as Perl Compatible Regular Expressions

From Tho18, p28

iex> Regex.run ~r{[aeiou]}, "caterpillar"
["a"]
iex> Regex.scan ~r{[aeiou]}, "caterpillar"
[["a"], ["e"], ["i"], ["a"]]
iex> Regex.split ~r{[aeiou]}, "caterpillar"
["c", "t", "rp", "ll", "r"]
iex> Regex.replace ~r{[aeiou]}, "caterpillar", "*"
"c*t*rp*ll*r"

Plug

Purpose

Overview

• Plug seems to be something to handle http traffic.
• It seems to me that you connect several plugs together to handle different situations.
• Plug is part of Cowboy.
• From: Phoenix description of plug
  • Plug is a specification for composable modules in between web applications.
  • It is also an abstraction layer for connection adapters of different web servers.
  • The basic idea of Plug is to unify the concept of a “connection” that we operate on.
• In order to use Plug, you need a webserver and its bindings for Plug.
  • The Cowboy webserver is the most common one.

References

• Introduction to plug
• Plug source code
• A deeper dive
• A deeper dive in Elixir's Plug
• Plug documentation
• Plug.conn documentation

Module Plug

A module plug has to export two functions: init and call(Cai18,p337).

• The init function is called at compile time, receives one argument, and its purpose is to prepare the options that will be passed to call.
• The call function is called at runtime and receives two arguments: the connection and the options returned by init.
  • This function is where we'll transform the connection according to our needs.

Function Plug

Networking

TCP

• https://www.openmymind.net/Elegant-TCP-with-Elixir-Part-2-Message-Boundaries/ *

Protobuf

1. make install-proto
2. mix deps.update --all
3. mix deps.get
4. mix deps.compile

Create struct with default values

• struct = Google.Protobuf.Duration.new(seconds: 22)

Remove pretty print

1. Go to: deps/YOUR_protos/lib/inspectable.ex
2. remove the whole defimpl block
3. remove the _build/dev/lib/YOUR_protos directory

re-run the test(s)

run time configuration

Providing run-time configuration to the application

Use destileri.release and config tuples: https://hexdocs.pm/config_tuples/readme.html

1. The config/config.ex is compiled at compile time, into a config code
2. the config code is read by the application when the application is loaded.
3. The config tupple will be run before the application is executed

• and the config tuple will scan the config/config.ex, when config tupple is started
• config tupple will replace all {:system, code with the ENV var value

4. config tupple will write every value into the config code(which now contains the ENV defined value)
5. the application will read the config code, and now have the ENV provided values.

Databases

mnesia

• (Mnesia - Elixir school)[https://elixirschool.com/en/lessons/specifics/mnesia/]
• (mnesia manual page)[http://erlang.org/doc/man/mnesia.html#system_info-1]
• (Using Mnesia in an Elixir Application)[https://blog.appsignal.com/2020/05/19/using-mnesia-in-an-elixir-application.html]
• (Mnesia And The Art of Remembering)[https://learnyousomeerlang.com/mnesia]

Metaprogramming

• https://blog.appsignal.com/2021/09/07/an-introduction-to-metaprogramming-in-elixir.html
• Mcc15 - https://pragprog.com/titles/cmelixir/metaprogramming-elixir/

Used for:

1. Move computations from run-time to compile-time
2. Generate code using compile-time computations
3. Enable self-modifying code

• Compile-time is the stage at which source code converts to binary code or intermediate binary code for a machine or virtual machine to execute.

• Run-time refers to when code executes.

• In Elixir, metaprogramming allows developers to leverage existing features to build new features that suit their individual business requirements.

• Elixir exposes the AST in a form that can be represented by Elixir’s own data structures and gave us a natural syntax to interact with it(Mcc15, p3).

• When a macro is first declared, the arguments of that macro are automatically converted into AST so that you don't need to parse the arguments manually. The arguments will not be evaluated beforehand.

AST

{
  :+,                                 # operation name,
  [context: Elixir, import: Kernel],  # metadata,
  [1, 2]                              # operation arguments
}

• unquote -Toggles the unquoting behavior in quote.
  • By disabling it, any unquote call is converted to an AST of the macro call (as with any other macro/function call).
  • This defers the evaluation of unquote to a later point.
  • Does 'unquote' inside a 'quote' sort of mean "do not quote this part(because it is already quoted"?
• bind_quoted Disables unquoting behavior in the quote and binds given variables in the body of quote.
  • Binding moves the variable initialization into the body of quote.
  • bind_quoted adds a "copy" of X(the var) into the body of quote by assigning it in the body of quote.
    • In a macro, this is equivalent to binding the variable to the caller context, as the variable is initialized during the evaluation of the callsite.
• location - This option controls whether run-time errors from a macro are reported from the caller or inside the quote.
  • By setting this option to :keep, error messages report specific lines in the macro that cause the error, rather than the line of the callsite.

You can access the AST representation of any Elixir expression by using the quote macro(Mcc15,p3).

Compilation

• https://blog.appsignal.com/2021/10/05/under-the-hood-of-macros-in-elixir.html

1. Parsing - The Elixir source code (program) is parsed into an AST, which we will call the initial AST.
2. Expansion - The initial AST is scanned and macro calls are identified. Macros are executed. Their output (AST) is injected and expanded into the callsite. Expansion occurs recursively, and a final AST is generated.
3. Bytecode generation phase - After the final AST is generated, the compiler performs an additional set of operations that eventually generate and execute BEAM VM bytecode.

Expansion

• Macro expansion occurs recursively, meaning that Elixir will continue expanding a macro until it reaches its most fundamental AST form.
• As macros expand right before bytecode is generated, they can modify a program's behavior during compile-time.

Macro

• A macro is Elixir code that runs at compile time.

  • macros are expanded before compile time.
  • evaluating the macro code and then replacing the call to the macro by the outcome of evaluating the macro[p222]

• A macro takes an internal representation of your source code as input and can create alternative output.

• The tree form of the quoted expressions is created by nesting three-element tuples[Cai18,p219]

• Elixir macros are inspired by LISP and should not be confused with C-style macros.

  • Unlike C/C++ macros, which work with pure text, Elixir macros work on abstract syntax tree (AST) structure, which makes it easier to perform nontrivial manipulations of the input code to obtain alternative output.
  • Of course, Elixir provides helper constructs to simplify this transformation.

• Elixir macros are something of a black art, but they make it possible to flush out nontrivial boilerplate at compile time and extend the language with your own DSL-like constructs.

• Macros contain two contexts — a macro context and a caller context:

1 defmacro foo do
2  # This is the macro's context, this is executed when the macro is called
3
4  # This is the return value of the macro (AST)
5  quote do
6    # This is the caller's context, this is executed when the callsite is called
7  end
8end

• The caller's context is the behavior declared in the quote.
  • The quote generated AST is the macro's output and is injected into and expanded at the callsite.
  • The behavior defined under the caller's context 'belongs' to the caller, not the module where the macro is defined.

Abstract Syntax Trees(AST) - Quoted Expressions

[Cai18,p218]

• literals - has the same value before and after after quoting them.

Trouble shooting

mix.exs

mix.exs an exception was raised

Fix:

1. mix deps.update --all
2. mix deps.compile

code execution

config

The ENV var is not provided, inspect shows the config entry instead

This is because the config tupple code isn't run prior to the appliaction loading, so the raw data you see is the content of the config code from the compile time

e.g. {:system, "TCP_PERF_SERVER_ENABLED", [type: :boolean, default: false]}

See 'Providing run-time configuration to the application' above.

Troubleshooting run-time issues

Switching from Supervisor.start_link(children, strategy: :one_for_one) to Supervisor.init(children, strategy: :one_for_one) seems to fix the problem

** (Mix) Could not start application my_service: MyService.Application.start(:normal, []) returned an error: shutdown: failed to start child: MyService.Supervisor
    ** (EXIT) MyService.Supervisor.init/1 returned a bad value: {:ok, #PID<0.826.0>}