How to learn ZIO? (and functional programming)

2023-01-13•scalaziofp

This post originally appeared on Scalac's blog

Suppose you are a Scala developer who decided to learn ZIO. Maybe you've heard that it's the future of Scala and that every Scala developer will be using it in a couple of years. Maybe you have listened to an interesting talk about how it can solve hard problems in a very straightforward and easy to understand way. Or you had a friend convert you over a series of meetups to Functional Programming, but Haskell and Idris jobs are few and far between, so you'd like to take FP up while earning a living. How would you go about achieving this?

Setting up

I will be showing some simple code examples, for that I'll set up an sbt project based on this guide, but actualized to ZIO 2 and Scala 3.

mkdir learn_zio
cd learn_zio
$EDITOR build.sbt
# if EDITOR is not set on your system, substitute your preferred one manually :)

Then add the following:

scalaVersion := "3.2.0"
name := "learn-zio"
version := "0.0.1"

libraryDependencies ++= Seq(
  "dev.zio" %% "zio" % "2.0.3"
)

Wrapping your head around functional effects

If you are already a pro with Cats Effect, Monix, or Scalaz; or you know Haskell, you can skip this step. Otherwise, you will have to go through the painful mental exercise of grokking functional effects.

So, what is a functional effect?

When you first hear that "pure functional programming means there are no side effects!" you might be thinking: "okay, that must not be very useful for anything then", which is a reasonable, but incorrect assumption. While Scala is a multi-paradigm non-pure language where this doesn't hold, I was first introduced to FP by the way of Haskell, which actually doesn't let you do any of that pesky procedural programming (at least, not on the face of it).

So, how do we get anything interesting done?

Let's start with the most basic one: console I/O.

import zio.*
import zio.Console.*

object Main extends ZIOAppDefault:

  def run =
    for
      _    <- printLine("─" * 25)
      name <- readLine("What is your name? ")
      _    <- printLine(s"Hello, $name!")
    yield ()


─────────────────────────
What is your name? Lancelot
Hello, Lancelot!

The sleight of hand with the 'no side effects whatsoever' statement is that even though your code can not do anything effectful, that doesn't mean the runtime can't. So the basic idea of pure functional programming is: we describe what we want to happen, and pass it to the runtime for execution. In this sense, functional effects are blueprints of the program we want to run. "We're just building data structures", as John De Goes frequently says.

This is somewhat analogous to the idea of garbage collection: we have a tedious responsibility, handling mutable variables (managing memory), which requires great attention to detail, which humans are bad at, but machines can be utilized efficiently for. Of course, this comes with some penalty vis-a-vis performance, but makes it much easier to reason about our programs, especially if concurrency is involved.

In the example above, we have printLine calls, which is a functional version of println. Essentially, it boils down to:

def printLine(value: => Any): UIO[Unit] =
  ZIO.succeed(println(value))

(This is not the actual implementation, because actual ZIO has to consider additional details, like execution traces.)

Note the by-name semantics, that's what allows us to not actually run anything until we desire to. When doing not-completely-pure FP, we call this point 'the end of the world'. If your application is all functional, this would just be the entry point, but we are not always so lucky as to start there, and might need to refactor a legacy application bit-by-bit, or have performance requirements which means we need to have boundaries between the pure and impure parts of our code (think Functional core, imperative shell).

And readLine could be (again, not the actual code):

val readLine: IO[Throwable, String] =
  ZIO.attempt(scala.io.StdIn.readLine())

IO[Throwable, String] represents an effect which can fail with a Throwable or succeed with a String. The UIO in the first example represents an effect which cannot fail, and succeeds with a Unit (a Unit return type typically indicates a side effect: since we are not returning anything, something outside the return value was likely the point of running this method). IO and UIO are more specific type aliases of the ZIO[R, E, A] type, see the documentation for more details.

In this example, the 'end of the world' is kind of obscured by ZIOAppDefault. If we Main to extend the standard App trait, and run the application, we'll see that nothing actually happens:

object Main extends scala.App:

  def run =
    for
      _    <- printLine("─" * 25)
      name <- readLine("What is your name? ")
      _    <- printLine(s"Hello, $name!")
    yield ()

  run

[info] running io.scalac.learn_zio.Main
[success] Total time: 1 s, completed

The run method returns a ZIO of type ZIO[Any, java.io.IOException, Unit] (an effect which requires no environment, can fail with an IOException or succeed with a Unit.

To actually run (or rather, evaluate) our ZIO, we need to pass it to a runtime, which makes the 'end of the world' explicit and visible:

object Main extends scala.App:
  def program =
    scala.io.StdIn.readLine()
    for
      _    <- printLine("─" * 25)
      name <- readLine("What is your name? ")
      _    <- printLine(s"Hello, $name!")
    yield ()

  Unsafe.unsafely (
    Runtime.default.unsafe.run(program).getOrThrowFiberFailure()
  )

Here you can see exactly the point where we leave the world of pure functions.

Implementing ZIO from scratch

I mention this because I saw it from multiple places where I go for my ZIO and Scala knowledge regularly. However, in practice I found it's not that helpful for actual ZIO usage as it is for gaining a deep understanding of the underlying concepts of functional effect systems and the ideas behind ZIO.

If you feel like it's interesting, Ziverge (the people behind ZIO) has a series of videos going through this exercise from 2021 (see Resources section), and at the time of writing this, there's actually a repeat going on, in light of the changes brought by ZIO 2, but those are unfortunately not published yet, I will have to come back and link to the videos

The ZIO model

In addition to being a functional effect system, ZIO aims to be a highly performant concurrency library with great developer experience. Let's explore how these are achieved!

To achieve this, ZIO takes advantage of virtual or 'green' threads (a JVM original, this used to be the only multithreading implementation), known here as fibers. This offers some advantages. For starters, fibers are not mapped to operating system threads, which means we can use a lot more of them than the number of processing cores would permit. They are also much cheaper to start and stop, making use of thread pools unnecessary. This effectively translates to the possibility to have hundreds of thousands of them, without running out of system resources.

The main ZIO datatype looks like this: ZIO[-R, +E, +A]. Not to reiterate the documentation too much, this can be thought of as R => Either[E, A], meaning a function from a parameter R (the environment) returning either a success value A or an error value E.

Aside: Variance

Seasoned users of Scala and effect systems might notice immediately why this is unusual: The parameters are not invariant. Variance is a property of data types that arises from subtyping, which is something most functional langs — most notably, Haskell — lack. Scala, being an originally JVM language, and having the ambitious goal of uniting the functional and object oriented paradigms, inherited (pun intended) this feature. When polymorphism interacts with higher-kinded types (of which generics is a limited, but more widely known subset), or even supplying function arguments or evaluating the return type, the question of substitutability needs to be addressed.

Say you have three types:

class A
class B extends A
class C extends B

If you have a function that takes a parameter of type B, or returns a B, when is it appropriate to use an A or a C in it's place?

def f(in: B): B = ???

Complicating matters even further, what do we (or rather, the compiler) do when we have a container of As, Bs, or Cs, like an Array of them? Is an Array[A] and an Array[C] a subtype or a supertype of Array[B]? This is what's addressed by variance.

Variance comes in three forms:

a) Invariance

This is when you have no variance, e.g. the default behavior. Most effect systems in Scala, hailing from Haskell (no subtyping), follow this convention. In fact, even ZIO was this way at it's beginning, but adopted variant type parameters later as a result of iterative improvement efforts.

b) Covariance

Covariance is the more straightforward case: if B is a subtype of A, then List[B] is a subtype of List[A]. (Arrays are actually invariant, for some fascinating reasons I won't delve into here) Functions are covariant in their return type, meaning returning a value of type C would typecheck for our example function f(). Again, this is especially interesting with inheritance, where an overriding method can legally return a subtype of the original method.

c) Contravariance

In contrast, inputs (parameters) are contravariant. On the type level, this reads:

Function1[-T1, +R]

Functions are represented by FunctionN traits where N is the number of input parameters, in our case, 1.

The contravariance relation means that B being a subtype of A, Function1[A, R] is the subtype of Function1[B, R].

Type hierarchy

Variance also explains why 'none' is represented by different types in the case of the type parameters. An effect that requires no environment, and can't fail is a value of type UIO[A], or ZIO[Any, Nothing, A].

Type hierarchy

Any and Nothing are at opposite ends of the Scala typesystem: everything is descended from Any, and Nothing is a subtype of every other possible type. (this is called top and bottom type in type theory) In the contravariant case, "nothing is required" is understood as "it works with Any input", while the covariant "no errors" case is expressed by a type that has no members, "Nothing comes out".

Working with program state

Let's go up a level, and advance from Console I/O to actual mutation of data. Recall: our code does not actually mutate in place, only describes the workflow, which is then executed by the runtime.

Take the Counter example from the docs:

import zio.*

final case class Counter(
  value: Ref[Int]
):
  def inc: UIO[Unit] = value.update(_ + 1)
  def dec: UIO[Unit] = value.update(_ - 1)
  def get: UIO[Int] = value.get

object Counter:
  def make: UIO[Counter] =
    Ref.make(0).map(Counter(_))

And try it out (only relevant changes included from Main.scala, see the full code here):

def program: ZIO[Any, java.io.IOException, Unit] =
  for
    _    <- printLine("─" * 25)
    cntr <- Counter.make
    v    <- cntr.get
    _    <- ZIO.debug(s"count start: $v")
    _    <- cntr.inc
    _    <- cntr.inc
    v    <- cntr.get
    _    <- ZIO.debug(s"count (inc twice): $v")
    _    <- cntr.dec
    v    <- cntr.get
    _    <- ZIO.debug(s"count (dec once): $v")
  yield ()

─────────────────────────
count start: 0
count (inc twice): 2
count (dec once): 1
[success] Total time: 2 s, completed

That looks good, and what's more, it also works parallelized:

def program: ZIO[Any, java.io.IOException, Unit] =
  for
    _    <- printLine("─" * 25)
    cntr <- Counter.make
    v    <- cntr.get
    _    <- ZIO.debug(s"count start: $v")
    _    <- cntr.inc <&> cntr.inc <&> cntr.dec <&> cntr.inc
    v    <- cntr.get
    _    <- ZIO.debug(s"count end: $v.")
  yield ()

(<&>, also known as zipPar is a parallel combinator)

─────────────────────────
count start: 0
count end: 2.
[success] Total time: 3 s, completed

Do keep in mind, this only works if the operations are independent, and their execution order is indifferent. If you need to do something based on checking the current value first (the canonical example being withdrawing from a bank balance if there are sufficient funds), you'll need to use something that's composable.

Takeaway: Get yourself a small project

I have mentioned a couple of features, barely scratching the surface, but this post is already getting too long. My ultimate advice is to start implementing a weekend project. At Scalac, we have a workshop for this purpose, which touches on STM , ZLayers, ZIO HTTP, persistence solutions, configuration, logging, and doing tests The ZIO Way™, if you find yourself without that, you will need to take initiative and come up with one. Preferably something that really interests you, and keeps you motivated throughout, the best software usually comes from developers wanting to scratch an itch they have.

If you get stuck, the documentation is well written, and the community is friendly and helpful, check out the Discord, and happy hacking!

Resources:

Created with Elmstatic
glider
Git
Artix Git
GitHub