# Golang杂项 ## 内存不返还问题 https://utcc.utoronto.ca/~cks/space/blog/programming/GoNoMemoryFreeing ## 二进制包大小 一些比较关注包大小的场景(小工具等) 1. 移除debug信息. 编译加`-ldflags="-s -w"`,其中`-s`移除符号表, `-w`移除DWARF信息,不能用gdb调试了. ```bash > GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" pkkr/cmd/tool/mcprobe ``` 2. 使用upx裁切 ```bash > git:(master) ✗ upx --brute mcprobe Ultimate Packer for eXecutables Copyright (C) 1996 - 2018 UPX 3.95 Markus Oberhumer, Laszlo Molnar & John Reiser Aug 26th 2018 File size Ratio Format Name -------------------- ------ ----------- ----------- 16465472 -> 4044368 24.56% linux/amd64 mcprobe ``` 效果为 ```bash # 原始 -rwxr-xr-x 1 xx staff 21M 1 9 18:02 mcprobe # 去调试信息 -rwxr-xr-x 1 xx staff 16M 1 9 18:02 mcprobe # 裁切 -rwxr-xr-x 1 xx staff 3.9M 1 9 18:02 mcprobe ``` ## 逃逸分析 决定对象分配到堆还是栈 ```go package main func f0() *int { var x int = 3 return &x } func f1() int { var x = new(int) *x = 3 return *x } func main() { } ``` 分析 ```bash ➜ go go run -gcflags '-m -l' escapeanalysis.go # command-line-arguments ./escapeanalysis.go:4:6: moved to heap: x ./escapeanalysis.go:9:13: f1 new(int) does not escape ``` ## Bounds Checking Elimination(BCE) https://docs.google.com/document/d/1vdAEAjYdzjnPA9WDOQ1e4e05cYVMpqSxJYZT33Cqw2g/edit# 为了安全进行边界检查, 为了提高性能, 进行BCE. BCE: - Duplicate checks - Constant slice size with masked index - Constant index - Constant index and constant size - Trivial ### 工具 #### 观察bce 加入`gcflags="-d=ssa/check_bce/debug=1"`. `go build -gcflags="-d=ssa/check_bce/debug=1" boundscheck.go` #### 关掉bound-checking 编译时加`gcflags=-B`. ## 锁争用开销分析 一般锁争用的成本评估 指标受机器影响, 只有数量级有参考意义, 简单估计就是高度争用情况下, mutex的获取成本在us量级. 用例,并发度1000, 预先装入20w数据 ```go var ( mu sync.Mutex cache = make(map[string]interface{}) ) func init() { for i := 0; i < 200000; i++ { cache[fmt.Sprintf("key-%d", i)] = i } } func set(k string, v interface{}) { mu.Lock() defer mu.Unlock() cache[k] = v } func BenchmarkCacheContention(b *testing.B) { b.ReportAllocs() b.SetParallelism(1000) b.RunParallel(func(pb *testing.PB) { i := 0 for pb.Next() { set(fmt.Sprintf("key-%d", i), i) i++ } }) } ``` 结果 ``` ➜ cache git:(master) ✗ go test -bench=BenchmarkCacheContention -benchmem -cpuprofile profile.out goos: darwin goarch: amd64 pkg: pkkr/util/cache BenchmarkCacheContention-4 1000000 1462 ns/op 25 B/op 3 allocs/op PASS ok pkkr/util/cache 12.126s ➜ cache git:(master) ✗ go tool pprof profile.out Type: cpu Time: Jan 15, 2020 at 2:54pm (CST) Duration: 11.80s, Total samples = 2.28s (19.32%) Entering interactive mode (type "help" for commands, "o" for options) (pprof) top Showing nodes accounting for 2.10s, 92.11% of 2.28s total Dropped 40 nodes (cum <= 0.01s) Showing top 10 nodes out of 55 flat flat% sum% cum cum% 0.74s 32.46% 32.46% 0.74s 32.46% runtime.pthread_cond_signal 0.48s 21.05% 53.51% 0.48s 21.05% runtime.usleep 0.34s 14.91% 68.42% 0.34s 14.91% runtime.pthread_cond_wait 0.33s 14.47% 82.89% 0.33s 14.47% runtime.pthread_mutex_lock 0.08s 3.51% 86.40% 0.08s 3.51% runtime.memclrNoHeapPointers 0.04s 1.75% 88.16% 0.16s 7.02% runtime.mallocgc 0.03s 1.32% 89.47% 0.03s 1.32% runtime.pthread_mutex_unlock 0.02s 0.88% 90.35% 0.02s 0.88% runtime.(*mspan).refillAllocCache 0.02s 0.88% 91.23% 0.02s 0.88% runtime.findObject 0.02s 0.88% 92.11% 0.06s 2.63% runtime.scanobject (pprof) ``` 刨除掉pprof的一般结果为 ``` ➜ cache git:(master) ✗ go test -bench=BenchmarkCacheContention time 0s,str map[test:Bingo!],exist [] time 1s,str map[test:Bingo!],exist [] time 2s,str map[test:Bingo!],exist [] time 3s,str map[test:Bingo!],exist [] time 4s,str map[test:Bingo!],exist [] time 5s,str map[],exist [test] time 6s,str map[],exist [test] time 7s,str map[],exist [test] time 8s,str map[],exist [test] time 9s,str map[],exist [test] goos: darwin goarch: amd64 pkg: pkkr/util/cache BenchmarkCacheContention-4 1000000 1276 ns/op 25 B/op ``` ## Overlapping Interfaces https://github.com/golang/proposal/blob/master/design/6977-overlapping-interfaces.md 1.14 加入的特性,解决钻石型组合可能导致的问题. 接口embedded时, 允许多个函数签名重复. 但是在接口里显式定义的函数, 还是必须保持唯一. 示例 ```go package user type Database interface { GetAccount(accountID uint64) (model.Account, error) } package device type Database interface { user.Database SaveDevice(accountID uint64, device model.Device) error } package wallet type Database interface { user.Database ReadWallet(accountID uint64) (model.Wallet, error) } package shopping type Database interface { device.Database wallet.Database Buy(accountID uint64, deviceID uint64) error } ``` ## mysql invalid connection - `SetConnMaxLifeTime`比mysql的`wait_timeout`大时. - 客户端读超时设的太小时 ## go mod ### 生成依赖图 modv工具 > go get github.com/poloxue/modv 生成png/svg > git:(refactor_usersvc) ✗ go mod graph | modv | dot -Tpng | open -f -a /Applications/Preview.app > git:(refactor_usersvc) ✗ go mod graph | modv | dot -Tsvg > deps.svg ## json ### `RawMessage` 希望部分字段序列化时(例如广播时负载只序列化一次,放在每个用户的消息上). ### 精度丢失问题 json的数值类型默认是float64, 一个int64从一个object反序列化到一个`map[string]interface{}`时, 会变成float64, 丢掉精度. 解析时需要`Decoder`调用`UseNumber`解析成Number类型. ## 单测缓存问题 重复测试一个rpc, server端加了日志, 一直收不到. 但是单测能跑通, 能拿到结果, 结果一直不改. 换了一个测试输入, 有日志了. 发现是缓存的结果. go.1.11以上, 可以使用`--count=1`禁掉缓存. ```shell zeus@test01:~/cy/pkkr$ go test -v -timeout 30s -run ^TestLogin pkkr/cmd/authsvc/pokekara 2022/05/12 02:49:31 PASS init SnowFlakeIDNode === RUN TestLogin 2022/05/12 02:49:31 account: user: extra: --- PASS: TestLogin (0.02s) PASS ok pkkr/cmd/authsvc/pokekara 0.051s zeus@test01:~/cy/pkkr$ go test --count=1 -v -timeout 30s -run ^TestLogin pkkr/cmd/authsvc/pokekara 2022/05/12 02:50:45 PASS init SnowFlakeIDNode === RUN TestLogin 2022/05/12 02:50:45 account: user: extra:<> --- PASS: TestLogin (0.02s) PASS ok pkkr/cmd/authsvc/pokekara 0.050s ```