Reactive database access with Slick3

Reactive database
access with Slick3
Naoki Takezoe
@takezoen
BizReach, Inc

Who am I?
● Naoki Takezoe: @takezoen
● Scala programmer at BizReach, Inc
● Long time OSS programmer and technical
writer

Slick3 (a.k.a. Reactive Slick)
http://slick.typesafe.com/

What's Reactive Slick
● Data access framework for Scala
● Powerful type-safe query API
● Parallel execution and streaming

History of Slick
● 2010 ScalaQuery 0.9
● 2012 Slick 0.11 (Move to Typesafe)
● 2013 Slick 1.0
● 2014 Slick 2.0 (Statically Session)
● 2015 Slick 3.0 (Reactive)

History of Slick
● 2010 ScalaQuery 0.9
● 2012 Slick 0.11 (Move to Typesafe)
● 2013 Slick 1.0
● 2014 Slick 2.0 (Statically Session)
● 2015 Slick 3.0 (Reactive)
Large API changes.
Migration was very hard...

Background
● C10K problem
● Non-blocking I/O based web framework
● Microservice architecture
● Big data

Example 1: Simple query
// Define query action
val select: DBIO[Seq[String]] =
Coffee // Table
.filter(_.price <= 100.bind) // Where clause
.map(_.name) // Select clause
.result
// Execute query
db.run(select) // => Future[Seq[String]]

Example 2: Parallel execution
// Define insert actions
val inserts: Seq[DBIO[Int]] = Seq(
Coffee("Colombian", 101, 7.99, 0, 0).insert,
Coffee("French_Roast", 49, 8.99, 0, 0).insert,
Coffee("Espresso", 150, 9.99, 0, 0).insert,
Coffee("Colombian_Decaf", 101, 8.99, 0, 0).insert,
Coffee("French_Roast_Decaf", 49, 9.99, 0, 0).insert
)
// Combine above actions
val combined: DBIO[Seq[Int]] = DBIO.sequence(inserts)
// Run action transactionally
db.run(combined.transactionally) // => Future[Seq[Int]]

Example 3: Work with other aync tasks
def index = Action.async {
for {
res <- ws.url("http://xxx").get()
name <- db.run(
Coffee
.filter(_.id === res.body.bind)
.map(_.name).result.head
)
} yield Ok(name)
}
Future[WSResponse]
Future[String]
Future[Result]

But JDBC is blocking.
Why Slick3?

Separate ExecutionContext
Controller Thread
(Play)
I/O Thread
(Slick)
JDBC

Separate ExecutionContext
Controller Thread
(Play)
I/O Thread
(Slick)
JDBC
for CPU bound
tasks
for I/O bound
tasks

by separating ExecutionContext
● Use server resources effectively
● Avoid exhausting threads by I/O tasks

Future: Real non-blocking database access?
● postgresql-async
● mysql-async
https://github.com/mauricio/postgresql-async

Streaming results
(Reactive Streams)

Streaming results is
● for large data processing
● based on Reactive Streams

Reactive Streams
public interface Publisher<T> {
public void subscribe(Subscriber<? super T> s);
}
public interface Subscriber<T> {
public void onSubscribe(Subscription s);
public void onNext(T t);
public void onError(Throwable t);
public void onComplete();
}
public interface Subscription {
public void request(long n);
public void cancel();
}
public interface Processor<T, R> extends Subscriber<T>, Publisher<R> {
}

Example: Slick API
val publisher: DatabasePublisher[String]
= db.stream(Coffee.map(_.message).result)
// Iterate stream using Slick API
publisher.foreach { name =>
println(name)
}

DataPublisher implementation
def foreach[U](f: T => U)(implicit ec: ExecutionContext): Future[Unit] = {
val p = Promise[Unit]()
@volatile var lastMsg: Future[Any] = null
@volatile var subscr: Subscription = null
subscribe(new Subscriber[T] {
def onSubscribe(s: Subscription): Unit = {
subscr = s
s.request(1L)
}
def onComplete(): Unit = { ... }
def onError(t: Throwable): Unit = { ... }
def onNext(t: T): Unit = {
lastMsg = Future(f(t))
lastMsg.onComplete {
case Success(v) => subscr.request(1L)
case Failure(t) => ...
}
}
})
p.future
}
Implemented based on
Reactive Streams API

Missing piece
Need both of the publicher and the subscriber
Publisher Subscriber / Publisher Subscriber

One idea: Slick3 + Akka Streams?
http://www.slideshare.net/kazukinegoro5/akka-streams-100-scalamatsuri

Problem 2: Difficulty
● Programming
○ Monadic API is hard for non functional programmers
● Configuration
○ Connection pool and ExecutionContext

Summary
● Slick3 provides 2 reactives:
○ Parallel database processing
○ Streaming large data based on Reactive Sreams
● Suitabe usecase for Slick are:
○ Composite database tasks and async tasks
○ Large data processing using Reactive Streams

Reactive database access with Slick3

More Related Content

What's hot

Viewers also liked

Similar to Reactive database access with Slick3

More from takezoe

Recently uploaded

Reactive database access with Slick3