Go - goroutines, channels, and sync 02-09-2017, 05:54 PM
#1
I've decided to keep the Go train, well, Go-ing, so I'll do a quick tutorial on asynchronous functions (called goroutines, you'll see why), channels, and the standard library sync package.
In some languages (PHP, for example), asynchronicity is a bitch to implement. In others (C++/#, Python, Java, etc...) it's easier, but there are some prominent hoops to jump through. In still others (Perl (see here), CLisp (using cl-async), and JavaScript (though sometimes a bit hackish)) it's as easy as a few keyworks or a builtin function. The last group is what async should always be: accessible, easy to use, and practical - and that's exactly how it is in Go.
Goroutines are super simple to use. You can call any predefined function to run in a goroutine, or create one anonymously and call it (see IIFE). As a bonus, deferrals have the same syntax and execute when the function they're defined in returns at any point (which is especially useful when working with databases or any kind of external connection that needs to be closed).
Example 1: goroutines
Example 2: deferring
The catch with goroutines is you can't capture returned values. Channels fix this by introducing a queue-like FIFO (first in, first out) pipeline which allows for thread-safe, synchronous communication between goroutines and/or other functions. Channels are created with the make() builtin, and must be given a type.
You can send to a channel by using the "channel<-value" syntax, and receive with "<-channel".
By default, channels are unbuffered, meaning you can only send to a channel when there's a concurrent receive waiting (in the above example, the goroutine would wait until the arbitrary code completes and the main function reaches the "n:=<-ch" line). To solve this, we can create a buffered channel that can be sent values without a receive waiting by modifying the call to make() with a size parameter:
So that channel can hold one big integer before it starts making other sending operations wait.
Lastly, we can use channels in a range, but they need to be closed after sending is finished or the range will go forever and potentially throw a deadlock panic (which aren't fun to debug).
The examples we've used so far work fine, but what about cases where you don't want to or can't wait for user input, or you have multiple goroutines communicating through a channel with no interaction by the main function? If you're running a program that doesn't quit until forced (e.g. an HTTP server), you could do something like sleep inside an infinite loop, but for programs that exit on their own, that doesn't fly. That's where WaitGroups come in.
WaitGroups act like counters, and calling the WaitGroup.Wait() function will pause the function's execution until the counter equals zero.
Pools are another part of the sync package, and they provide an unordered, untyped (more or less, I'll explain this later) set of temporary objects that can be saved and retrieved. Pools are used when you have lots of values to work with and you want multiple "worker" goroutines to handle them (unlike plain channels, which act exclusively as a pipe). Values can be added to a pool with the Pool.Put() function, and retrieved with Pool.Put()
Note the "bInt:=prod.(big.Int)" line in the sqrt() function. As I said earlier, pools are essentially typeless, in the sense that they use an interface as the type for getting/putting values. Interfaces are all well and good for passing values around, but if you want to call the underlying object's functions, you need to assert the type (hence ".(big.Int)").
Goroutines and deferral
In some languages (PHP, for example), asynchronicity is a bitch to implement. In others (C++/#, Python, Java, etc...) it's easier, but there are some prominent hoops to jump through. In still others (Perl (see here), CLisp (using cl-async), and JavaScript (though sometimes a bit hackish)) it's as easy as a few keyworks or a builtin function. The last group is what async should always be: accessible, easy to use, and practical - and that's exactly how it is in Go.
Goroutines are super simple to use. You can call any predefined function to run in a goroutine, or create one anonymously and call it (see IIFE). As a bonus, deferrals have the same syntax and execute when the function they're defined in returns at any point (which is especially useful when working with databases or any kind of external connection that needs to be closed).
Example 1: goroutines
Code:
package main; // package declaration
import (
"time" // need the Sleep() function
"fmt" // input/printing functions
);
// run some arbitrary stuff
func f(method string){
for i:=0;i<10;i++{
fmt.Printf("%s method: %d\n",method,i)
}
}
func main(){
// call function directly
f("direct")
// create goroutine with IIFE
go func(){
time.Sleep(1) // wait 1 second
fmt.Println("IIFE goroutine") // print a message
}()
// call function as goroutine
go f("goroutine")
// wait for user input, then exit
var input string
fmt.Scanln(&input)
}Example 2: deferring
Code:
package main; // package declaration
import "fmt";
func ex1(n int) (sum int){
// print a message when the function returns
defer fmt.Println("ex1 is returning")
for i:=1;i<100;i++{
sum+=i
if sum%n==0{
return
}
}
return
}
func main(){
sum:=ex1(24)
fmt.Println(sum)
}Channels
The catch with goroutines is you can't capture returned values. Channels fix this by introducing a queue-like FIFO (first in, first out) pipeline which allows for thread-safe, synchronous communication between goroutines and/or other functions. Channels are created with the make() builtin, and must be given a type.
You can send to a channel by using the "channel<-value" syntax, and receive with "<-channel".
Code:
package main; // package declaration
import (
"math/big"
"fmt"
);
func main(){
// create a channel with type big integer
ch:=make(chan big.Int)
// calculate a large base 2 exponent and
// send to the channel
go func(n uint){
// use bitshifting to calculate exponent
prod:=*new(big.Int).Lsh(big.NewInt(1),n)
// send to channel
ch<-prod
}(128)
/* arbitrary other shit */
// recieve channel value
n:=<-ch
// print the value
fmt.Println(n.String())
}By default, channels are unbuffered, meaning you can only send to a channel when there's a concurrent receive waiting (in the above example, the goroutine would wait until the arbitrary code completes and the main function reaches the "n:=<-ch" line). To solve this, we can create a buffered channel that can be sent values without a receive waiting by modifying the call to make() with a size parameter:
Code:
func main(){
ch:=make(chan big.Int,1)
// ...
}So that channel can hold one big integer before it starts making other sending operations wait.
Lastly, we can use channels in a range, but they need to be closed after sending is finished or the range will go forever and potentially throw a deadlock panic (which aren't fun to debug).
Code:
package main; // package declaration
import (
"math/big"
"fmt"
);
func main(){
// create a channel with type big integer
ch:=make(chan big.Int)
// calculate all base 2 exponents up to n and
// send to the channel
go func(n int){
for i:=1;i<=n;i++{
// use bitshifting to calculate exponent
prod:=*new(big.Int).Lsh(big.NewInt(1),uint(i))
// send to channel
ch<-prod
}
// close the channel when sending is finished
close(ch)
}(128)
/* arbitrary other shit */
// loop all channel values
for n:=range ch{
// print the value
fmt.Println(n.String())
}
}WaitGroups
The examples we've used so far work fine, but what about cases where you don't want to or can't wait for user input, or you have multiple goroutines communicating through a channel with no interaction by the main function? If you're running a program that doesn't quit until forced (e.g. an HTTP server), you could do something like sleep inside an infinite loop, but for programs that exit on their own, that doesn't fly. That's where WaitGroups come in.
WaitGroups act like counters, and calling the WaitGroup.Wait() function will pause the function's execution until the counter equals zero.
Code:
package main; // package declaration
import (
"math/big"
"sync"
"fmt"
);
var (
// create a channel with type big integer (globally accessible)
ch chan big.Int=make(chan big.Int)
// create a global WaitGroup
wg sync.WaitGroup
);
// calculate all base 2 exponents up to n and
// send to the channel
func exp(n int){
// decrement the WaitGroup counter when the function returns
defer wg.Done()
for i:=1;i<=n;i++{
// use bitshifting to calculate exponent
prod:=*new(big.Int).Lsh(big.NewInt(1),uint(i))
// send to channel
ch<-prod
}
// close the channel when sending is finished
close(ch)
}
func proc(){
// decrement the WaitGroup counter when the function returns
defer wg.Done()
for n:=range ch{
// define rem as n%17 (modulus)
rem:=new(big.Int).Rem(&n,big.NewInt(17))
fmt.Println(rem)
}
}
func main(){
wg.Add(2) // add 2 to the WaitGroup counter - 1 for each goroutine
go exp(128) // create a goroutine with the exp() function
go proc() // create a goroutine with the proc() function
wg.Wait() // wait for goroutines to decrement their counters
}Pools
Pools are another part of the sync package, and they provide an unordered, untyped (more or less, I'll explain this later) set of temporary objects that can be saved and retrieved. Pools are used when you have lots of values to work with and you want multiple "worker" goroutines to handle them (unlike plain channels, which act exclusively as a pipe). Values can be added to a pool with the Pool.Put() function, and retrieved with Pool.Put()
Code:
package main; // package declaration
import (
"math/big"
"sync"
"fmt"
);
var (
// create a global WaitGroup
wg sync.WaitGroup
// create a global Pool
pl sync.Pool
);
// calculate all base 2 exponents up to n and
// send to the channel
func exp(n int){
// decrement the WaitGroup counter when the function returns
defer wg.Done()
for i:=1;i<=n;i++{
// use bitshifting to calculate exponent
prod:=*new(big.Int).Lsh(big.NewInt(1),uint(i))
// add to Pool
pl.Put(prod)
}
}
// calculate the exponent of a base 2 product with logarithms
func sqrt(){
// decrement the WaitGroup counter when the function returns
defer wg.Done()
// get values until nil is given
prod:=pl.Get()
for ;prod!=nil;prod=pl.Get(){
// assert prod type back to big.Int
bInt:=prod.(big.Int)
// print the result
fmt.Println(bInt.String())
}
}
// generate n worker goroutines
func rFactory(n int){
// add n to the WaitGroup counter
wg.Add(n)
// generate goroutines in a loop
for i:=0;i<n;i++{
go sqrt()
}
}
func main(){
wg.Add(1) // add to the WaitGroup counter for exp()
go exp(1024) // create a goroutine with the exp() function
rFactory(8) // create 8 worker goroutines
wg.Wait() // wait for goroutines to decrement their counters
}Note the "bInt:=prod.(big.Int)" line in the sqrt() function. As I said earlier, pools are essentially typeless, in the sense that they use an interface as the type for getting/putting values. Interfaces are all well and good for passing values around, but if you want to call the underlying object's functions, you need to assert the type (hence ".(big.Int)").
It's often the outcasts, the iconoclasts ... those who have the least to lose because they
don't have much in the first place, who feel the new currents and ride them the farthest.
don't have much in the first place, who feel the new currents and ride them the farthest.
![[+]](https://sinister.li/images/modern/collapse_collapsed.png)
