KEMBAR78
Golang Channels | PDF
Golang
Channels
Joris Bonnefoy
DevOps @ OVH
Introduction
“Channels are a typed conduit through
which you can send and receive values”
package main
func main() {
ch := make(chan bool)
go func() {
ch <- true
}()
<-ch
close(ch)
}
g(main) g()
true
“Share memory by communicating”
Some specificities...
Channels are typed
package main
func main() {
ch := make(chan bool)
}
Channels can have directions
package main
func main() {
ch := make(chan bool)
callMe(ch, ch)
}
func callMe(recv <-chan bool, send chan<- bool) {
...
}
“Two” types of channel
Channels can be synchronous (unbuffered)
package main
func main() {
ch := make(chan bool)
ch <- true // It blocks
}
Channels can be asynchronous (buffered)
package main
func main() {
ch := make(chan bool, 1)
ch <- true // It doesn’t block
ch <- true // It blocks
}
The third type :)
Channels can be asynchronous with zero-sized elements
package main
func main() {
done := make(chan struct{}, 1)
go func() {
// Doing some stuff
done <- struct{}{}
}()
<-done // Wait for goroutine to finish its work
}
Channel structure
hchan structure
qcount
uint
number of elements in the buffer
dataqsiz
uint
buffer queue size
buf
*buffer
(unsafe) pointer to chan buffer
closed
uint32
flag that shows whether the channel closed or not
recvq
*linked list
list of goroutines waiting to receive data from the chan
sendq
*linked list
list of goroutines waiting to send data from the chan
lock
mutex
mutex for concurrent accesses to the chan
we need uint to
make atomic
operations on these
fields
New hchan structure fields
● elemtype and elemsize
● waitq structure
● sendx and recvx indexes
Synchronous channel
package main
func main() {
ch := make(chan bool)
go func() {
ch <- true
}()
<-ch
}
qcount 0
dataqsiz 0
buf nil
recvq nil
sendq nil
qcount 0
dataqsiz 0
buf nil
recvq
sendq nil
g(main)
g()
write directly to
main goroutine stack
block main
goroutine
This is the only place in the go runtime
where a goroutine writes directly to the
stack of another
Asynchronous channel
package main
func main() {
ch := make(chan bool, 1)
ch <- true
go func() {
<-ch
}()
ch <- true
}
qcount 0
dataqsiz 1
buf
recvq nil
sendq
true
nil g(main)
block main
goroutine
g()
read from
buffer
read next data
from main
goroutine
put it into
the buffer
unlock main
goroutine
Select statement
Select statement
select {
case data := <-ch:
// Do something
default:
// Do something else
}
A little bit more about select
● inspired from C select function
○ allow to wait I/O from a set of file descriptors
● non-deterministic
● optimistic
● based on 2 structures
○ hselect, representing the select statement
○ scase, representing a select case
● a select with only a default case, or one case + the default case, is
automatically rewritten by the compiler to a simpler structure
Select statement run (selectgo)
1. evaluate all the involved channels and values
2. permuting the cases slice elements (randomize)
3. ordering elements according to cases hchan address
4. locking all the channels
5. starting to loop…
a. checking if a channel is already waiting, so we can use it immediately (and it’s done)
b. otherwise, enqueuing on all channels (and sleep)
c. waiting for another goroutine to wake us up
d. another goroutine wake us up (done = 1)
e. dequeuing of all the other channels
i. if we were waken up by a channel close operation, return to a.
ii. else, we’re done :)
Close statement
Close statement
package main
func main() {
ch := make(chan bool)
close(ch)
}
Close statement
● Iterate through all the senders and receivers in queues and unlock them
● Receivers receive the default value for the chan type
○ When the buffer is empty
● Senders panic
Downsides
Downsides
● close operation can panic
○ on nil channels
○ on closed channels
● and make senders panic too
○ if they send on a closed channel
“Only the sender should close its
channel”
1 producer - N consumers
producer
consumer
consumer
consumer
channel
close
consumers receive
default value
corresponding to
the channel type
M producers - N consumers
producer
consumer
consumer
consumer
channel
producer
producer
close
send
panic
producer
M producers - N consumers - 1st proposition
producer
consumer
consumer
consumer
channel
producer
producer
channel
channel
recv: 0recv: 3recv: 2 send: 3
M producers - N consumers - 2nd proposition
producer
consumer
consumer
consumer
channel
producer
producer
send: 0ok := val -> ch
if !ok {
panic(“Channel closed”)
}
Thank you
@devatoria
References
● http://stackoverflow.com/questions/19621149/how-are-go-channels-implemented
● https://golang.org/src/runtime/chan.go
● https://docs.google.com/document/d/1yIAYmbvL3JxOKOjuCyon7JhW4cSv1wy5hC0ApeGMV9s/pub
● http://dmitryvorobev.blogspot.fr/2016/08/golang-channels-implementation.html
● https://golang.org/doc/effective_go.html#channels
● http://www.jtolds.com/writing/2016/03/go-channels-are-bad-and-you-should-feel-bad/
● https://github.com/golang/go/issues/14601
● https://golang.org/pkg/unsafe/#Pointer
● https://golang.org/src/runtime/malloc.go?h=newarray#L817
● http://stackoverflow.com/questions/37021194/how-are-golang-select-statements-implemented
● https://github.com/golang/go/blob/master/src/runtime/select.go
● http://www.tapirgames.com/blog/golang-concurrent-select-implementation

Golang Channels

  • 1.
  • 2.
  • 3.
  • 4.
    “Channels are atyped conduit through which you can send and receive values”
  • 5.
    package main func main(){ ch := make(chan bool) go func() { ch <- true }() <-ch close(ch) } g(main) g() true
  • 6.
    “Share memory bycommunicating”
  • 7.
  • 8.
    Channels are typed packagemain func main() { ch := make(chan bool) }
  • 9.
    Channels can havedirections package main func main() { ch := make(chan bool) callMe(ch, ch) } func callMe(recv <-chan bool, send chan<- bool) { ... }
  • 10.
  • 11.
    Channels can besynchronous (unbuffered) package main func main() { ch := make(chan bool) ch <- true // It blocks }
  • 12.
    Channels can beasynchronous (buffered) package main func main() { ch := make(chan bool, 1) ch <- true // It doesn’t block ch <- true // It blocks }
  • 13.
  • 14.
    Channels can beasynchronous with zero-sized elements package main func main() { done := make(chan struct{}, 1) go func() { // Doing some stuff done <- struct{}{} }() <-done // Wait for goroutine to finish its work }
  • 15.
  • 16.
    hchan structure qcount uint number ofelements in the buffer dataqsiz uint buffer queue size buf *buffer (unsafe) pointer to chan buffer closed uint32 flag that shows whether the channel closed or not recvq *linked list list of goroutines waiting to receive data from the chan sendq *linked list list of goroutines waiting to send data from the chan lock mutex mutex for concurrent accesses to the chan we need uint to make atomic operations on these fields
  • 17.
    New hchan structurefields ● elemtype and elemsize ● waitq structure ● sendx and recvx indexes
  • 18.
  • 19.
    package main func main(){ ch := make(chan bool) go func() { ch <- true }() <-ch } qcount 0 dataqsiz 0 buf nil recvq nil sendq nil qcount 0 dataqsiz 0 buf nil recvq sendq nil g(main) g() write directly to main goroutine stack block main goroutine This is the only place in the go runtime where a goroutine writes directly to the stack of another
  • 20.
  • 21.
    package main func main(){ ch := make(chan bool, 1) ch <- true go func() { <-ch }() ch <- true } qcount 0 dataqsiz 1 buf recvq nil sendq true nil g(main) block main goroutine g() read from buffer read next data from main goroutine put it into the buffer unlock main goroutine
  • 22.
  • 23.
    Select statement select { casedata := <-ch: // Do something default: // Do something else }
  • 24.
    A little bitmore about select ● inspired from C select function ○ allow to wait I/O from a set of file descriptors ● non-deterministic ● optimistic ● based on 2 structures ○ hselect, representing the select statement ○ scase, representing a select case ● a select with only a default case, or one case + the default case, is automatically rewritten by the compiler to a simpler structure
  • 25.
    Select statement run(selectgo) 1. evaluate all the involved channels and values 2. permuting the cases slice elements (randomize) 3. ordering elements according to cases hchan address 4. locking all the channels 5. starting to loop… a. checking if a channel is already waiting, so we can use it immediately (and it’s done) b. otherwise, enqueuing on all channels (and sleep) c. waiting for another goroutine to wake us up d. another goroutine wake us up (done = 1) e. dequeuing of all the other channels i. if we were waken up by a channel close operation, return to a. ii. else, we’re done :)
  • 26.
  • 27.
    Close statement package main funcmain() { ch := make(chan bool) close(ch) }
  • 28.
    Close statement ● Iteratethrough all the senders and receivers in queues and unlock them ● Receivers receive the default value for the chan type ○ When the buffer is empty ● Senders panic
  • 29.
  • 30.
    Downsides ● close operationcan panic ○ on nil channels ○ on closed channels ● and make senders panic too ○ if they send on a closed channel
  • 31.
    “Only the sendershould close its channel”
  • 32.
    1 producer -N consumers producer consumer consumer consumer channel close consumers receive default value corresponding to the channel type
  • 33.
    M producers -N consumers producer consumer consumer consumer channel producer producer close send panic producer
  • 34.
    M producers -N consumers - 1st proposition producer consumer consumer consumer channel producer producer channel channel
  • 35.
    recv: 0recv: 3recv:2 send: 3 M producers - N consumers - 2nd proposition producer consumer consumer consumer channel producer producer send: 0ok := val -> ch if !ok { panic(“Channel closed”) }
  • 36.
  • 37.
    References ● http://stackoverflow.com/questions/19621149/how-are-go-channels-implemented ● https://golang.org/src/runtime/chan.go ●https://docs.google.com/document/d/1yIAYmbvL3JxOKOjuCyon7JhW4cSv1wy5hC0ApeGMV9s/pub ● http://dmitryvorobev.blogspot.fr/2016/08/golang-channels-implementation.html ● https://golang.org/doc/effective_go.html#channels ● http://www.jtolds.com/writing/2016/03/go-channels-are-bad-and-you-should-feel-bad/ ● https://github.com/golang/go/issues/14601 ● https://golang.org/pkg/unsafe/#Pointer ● https://golang.org/src/runtime/malloc.go?h=newarray#L817 ● http://stackoverflow.com/questions/37021194/how-are-golang-select-statements-implemented ● https://github.com/golang/go/blob/master/src/runtime/select.go ● http://www.tapirgames.com/blog/golang-concurrent-select-implementation