#+LATEX_HEADER: \newenvironment{lequation}{\begin{equation}\Large}{\end{equation}}
#+OPTIONS: ^:nil
#+ATTR_LATEX: :width 5cm :options angle=90
#+TITLE: Go-Functional
#+AUTHOR: yangruipis
#+EMAIL: yangruipis@163.com
#+KEYWORDS:
#+OPTIONS: H:4 toc:t
#+OPTIONS: tex:t :exports both
#+SETUPFILE: https://gitee.com/yangruigit/my_blogs/raw/master/lib/theme-readtheorg.setup
#+HTML_HEAD:
#+html:
#+html:
** Features
Go-functional is inspired by [[https://github.com/EntilZha/PyFunctional][PyFunctional]] and [[https://github.com/thoas/go-funk][go-funk]]. It makes golang struct data processing easier, especially when we have a data pipeline. Go-functionl is:
- FAST :: there is no reflect at all
- “ CHAIN :: chain functional operations
- LAZY :: most trainsformation operations are lazy evaluated
- ¨ SIMPLE :: use generic, all you need is a ~Map~ , instead of ~MapInt~ ~MapInt32~ ~MapInt***~ ...
- ‘ USER-FRIENDLY :: ~Spark~ style APIs is provided, maybe ~LinQ~ someday.
** Installation ( go>=1.18 )
#+BEGIN_SRC bash
go get github.com/Yangruipis/go-functional
#+END_SRC
** Usage
*** basic import
#+begin_src go
package main
import (
"fmt"
"github.com/Yangruipis/go-functional/pkg/fn"
)
func main() {
fn.RangeSeq(0, 10, 1).ForEach(func(i, v int) {
fmt.Printf("%d\n", v)
})
}
#+end_src
*** Range -> Map -> Filter -> ForEach
#+begin_src go :imports '("fmt" "github.com/Yangruipis/go-functional/pkg/fn") :exports both :results output
fn.RangeSeq(0, 10, 1).Map(
func(k, v int) (int, int) {return k, v + 1 },
).Filter(
func(k, v int) bool { return v >= 3 },
).ForEach(
func(i, v int) { fmt.Printf("%d\n", v) },
)
#+end_src
#+RESULTS:
: 3
: 4
: 5
: 6
: 7
: 8
: 9
: 10
*** Range -> Map -> GroupByKey -> Aggregate -> ForEach
#+begin_src go :imports '("fmt" "github.com/Yangruipis/go-functional/pkg/fn") :exports both :results output
fn.RangeSeq(0, 10, 1).Map(
func(k, v int) (int, int) {return k%2, v }, // split to 2 groups
).GroupByKey().Aggregate(func(vv []int) int {
s := 0
for _, v := range vv {
s += v
}
return s
}).ForEach(
func(k int, v int) { fmt.Printf("sum of group %d is: %v\n", k, v) },
)
#+end_src
#+RESULTS:
: mean of group 0 is: 20
: mean of group 1 is: 25
*** multi path with cache: Range -> Map -> Cache ( --> Filter ) & ( --> Shuffle )
#+begin_src go :imports '("fmt" "github.com/Yangruipis/go-functional/pkg/fn") :exports both :results output
c := fn.RangeSeq(0, 10, 1).Map(
func(k, v int) (int, int) { return k, v * 2 },
).Cache()
c1 := c.Filter(func(k, v int) bool { return v >= 10 } )
fmt.Printf("paths: %v, results: %v \n", c1.Paths, c1.ToSlice())
c2 := c.Shuffle()
fmt.Printf("paths: %v, results: %v \n", c2.Paths, c2.ToSlice())
#+end_src
#+RESULTS:
: paths: [Map Cache Filter], results: [10 12 14 16 18]
: paths: [Map Cache Shuffle], results: [2 14 8 0 18 4 6 10 16 12]
** API list
There are two types of API(consistent with Spark):
1. Transformation: Iterator in, iterator out. Will not be executed until action operation. Iterator is supposed to be consumed only once.
2. Action: Iterator in, results out. All transformation operations are executed here (lazy exec).
| func call[51/51] | chain call[33/51] | name | signature | type |
|------------------+-------------------+-------------+-------------------------------+----------------------|
| … | … | Map | [K, V] -> [K, V] | transformation |
| … | … | Filter | [K, V] -> [K, V] | transformation |
| … | … | Flatten | [K, []V] -> [K, V] | transformation |
| … | … | GroupBy | [K, V] -> [K, []V] | transformation |
| … | … | GroupByKey | [K, V] -> [K, []V] | transformation |
| … | | GroupByVal | [K, V] -> [V, []K] | transformation |
| … | … | FlatMap | [K, []V] -> [K, []V] | transformation |
| … | … | ReduceByKey | [K, []V] -> [K, V] | transformation |
| … | … | CountByKey | [K, V] -> [K, int] | transformation |
| … | | CountByVal | [K, V] -> [V, int] | transformation |
| … | | Union | [K, V] [K, V] -> [K, V] | transformation |
| … | | Intersect | [K, V] [K, V] -> [K, V] | transformation |
| … | | Subtract | [K, V] [K, V] -> [K, V] | transformation |
| … | | Distinct | [K, V] -> [K, V] | transformation |
| … | | UnionBy | [K, V] [K, V] -> [K, V] | transformation |
| … | | IntersectBy | [K, V] [K, V] -> [K, V] | transformation |
| … | | SubtractBy | [K, V] [K, V] -> [K, V] | transformation |
| … | | DistinctBy | [K, V] -> [K, V] | transformation |
| … | | Cartesian | [K, V] [K, V] -> [K, V] | transformation |
| … | … | Chunk | [K, V] -> [K, []V] | transformation |
| … | … | Sort | [K, V] -> [K, V] | transformation |
| … | … | Aggregate | [K, V] -> [K, V1] | transformation |
| … | | Zip | [K, V] [K, V1]-> [K, [V, V1]] | transformation |
| … | | Invert | [K, V] -> [V, K] | transformation |
| … | … | Reverse | [K, V] -> [K, V] | transformation |
| … | … | Shuffle | [K, V] -> [K, V] | transformation |
| … | … | Sample | [K, V] -> [K, V] | transformation |
| … | … | Choices | [K, V] -> [K, V] | transformation |
| … | … | Head | [K, V] -> [K, V] | transformation |
| … | … | Tail | [K, V] -> [K, V] | transformation |
| … | … | Cache | [K, V] -> [K, V] | transformation |
| … | … | Repeat | V -> [int, V] | transformation(init) |
| … | … | Range | int -> [int, int] | transformation(init) |
| … | … | Reduce | [K, V] -> V | action |
| … | … | Size | [K, V] -> int | action |
| … | … | Exists | [K, V] -> bool | action |
| … | … | ExistsBy | [K, V] -> bool | action |
| … | … | All | [K, V] -> bool | action |
| … | … | Any | [K, V] -> bool | action |
| … | … | Count | [K, V] -> int | action |
| … | … | CountBy | [K, V] -> int | action |
| … | … | ToSlice | [K, V] -> []V | action |
| … | … | ToMap | [K, V] -> map[K]V | action |
| … | | ToSet | [K, V] -> map[K]struct{} | action |
| … | | Sum | [K, V] -> int | action |
| … | | Avg | [K, V] -> int | action |
| … | … | ForEach | [K, V] -> void | action |
| … | … | Entries | [K, V] -> [][K, V] | action |
| … | | IndexOf | [K, V] -> []int | action |
| … | | NIndexOf | [K, V] -> int | action |
| … | … | Values | [K, V] -> []V | action |
| … | … | Keys | [K, V] -> []K | action |
** Development
*** run tests
#+BEGIN_SRC
go mod download
go test ./...
#+END_SRC
*** contributing
- code format: ~gofmt~
- commit format: see [[https://github.com/angular/angular/blob/main/CONTRIBUTING.md#-commit-message-format][Angular Commit Message Conventions]]