Golang杂项¶

内存不返还问题¶

https://utcc.utoronto.ca/~cks/space/blog/programming/GoNoMemoryFreeing

二进制包大小¶

一些比较关注包大小的场景(小工具等)

移除debug信息.

编译加-ldflags="-s -w",其中-s移除符号表, -w移除DWARF信息,不能用gdb调试了.

> GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" pkkr/cmd/tool/mcprobe

使用upx裁切

> git:(master) ✗ upx --brute mcprobe
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2018
UPX 3.95        Markus Oberhumer, Laszlo Molnar & John Reiser   Aug 26th 2018

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
  16465472 ->   4044368   24.56%   linux/amd64   mcprobe

效果为

# 原始
-rwxr-xr-x   1 xx  staff    21M  1  9 18:02 mcprobe
# 去调试信息
-rwxr-xr-x   1 xx  staff    16M  1  9 18:02 mcprobe
# 裁切
-rwxr-xr-x   1 xx  staff    3.9M  1  9 18:02 mcprobe

逃逸分析¶

决定对象分配到堆还是栈

package main

func f0() *int {
	var x int = 3
	return &x
}

func f1() int {
	var x = new(int)
	*x = 3
	return *x
}

func main() {
}

分析

➜  go go run -gcflags '-m -l' escapeanalysis.go
# command-line-arguments
./escapeanalysis.go:4:6: moved to heap: x
./escapeanalysis.go:9:13: f1 new(int) does not escape

Bounds Checking Elimination(BCE)¶

https://docs.google.com/document/d/1vdAEAjYdzjnPA9WDOQ1e4e05cYVMpqSxJYZT33Cqw2g/edit#

为了安全进行边界检查, 为了提高性能, 进行BCE.

BCE:

Duplicate checks
Constant slice size with masked index
Constant index
Constant index and constant size
Trivial

工具¶

观察bce¶

加入gcflags="-d=ssa/check_bce/debug=1".

go build -gcflags="-d=ssa/check_bce/debug=1" boundscheck.go

关掉bound-checking¶

编译时加gcflags=-B.

锁争用开销分析¶

一般锁争用的成本评估

指标受机器影响, 只有数量级有参考意义, 简单估计就是高度争用情况下, mutex的获取成本在us量级.

用例,并发度1000, 预先装入20w数据

var (
 mu    sync.Mutex
 cache = make(map[string]interface{})
)

func init() {
 for i := 0; i < 200000; i++ {
  cache[fmt.Sprintf("key-%d", i)] = i
 }
}

func set(k string, v interface{}) {
 mu.Lock()
 defer mu.Unlock()
 cache[k] = v
}

func BenchmarkCacheContention(b *testing.B) {
 b.ReportAllocs()
 b.SetParallelism(1000)
 b.RunParallel(func(pb *testing.PB) {
  i := 0
  for pb.Next() {
   set(fmt.Sprintf("key-%d", i), i)
   i++
  }
 })
}

结果

➜  cache git:(master) ✗ go test -bench=BenchmarkCacheContention -benchmem -cpuprofile profile.out
goos: darwin
goarch: amd64
pkg: pkkr/util/cache
BenchmarkCacheContention-4       1000000          1462 ns/op          25 B/op          3 allocs/op
PASS
ok      pkkr/util/cache 12.126s
➜  cache git:(master) ✗ go tool pprof profile.out
Type: cpu
Time: Jan 15, 2020 at 2:54pm (CST)
Duration: 11.80s, Total samples = 2.28s (19.32%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 2.10s, 92.11% of 2.28s total
Dropped 40 nodes (cum <= 0.01s)
Showing top 10 nodes out of 55
      flat  flat%   sum%        cum   cum%
     0.74s 32.46% 32.46%      0.74s 32.46%  runtime.pthread_cond_signal
     0.48s 21.05% 53.51%      0.48s 21.05%  runtime.usleep
     0.34s 14.91% 68.42%      0.34s 14.91%  runtime.pthread_cond_wait
     0.33s 14.47% 82.89%      0.33s 14.47%  runtime.pthread_mutex_lock
     0.08s  3.51% 86.40%      0.08s  3.51%  runtime.memclrNoHeapPointers
     0.04s  1.75% 88.16%      0.16s  7.02%  runtime.mallocgc
     0.03s  1.32% 89.47%      0.03s  1.32%  runtime.pthread_mutex_unlock
     0.02s  0.88% 90.35%      0.02s  0.88%  runtime.(*mspan).refillAllocCache
     0.02s  0.88% 91.23%      0.02s  0.88%  runtime.findObject
     0.02s  0.88% 92.11%      0.06s  2.63%  runtime.scanobject
(pprof)

刨除掉pprof的一般结果为

➜  cache git:(master) ✗ go test -bench=BenchmarkCacheContention
time 0s,str map[test:Bingo!],exist []
time 1s,str map[test:Bingo!],exist []
time 2s,str map[test:Bingo!],exist []
time 3s,str map[test:Bingo!],exist []
time 4s,str map[test:Bingo!],exist []
time 5s,str map[],exist [test]
time 6s,str map[],exist [test]
time 7s,str map[],exist [test]
time 8s,str map[],exist [test]
time 9s,str map[],exist [test]
goos: darwin
goarch: amd64
pkg: pkkr/util/cache
BenchmarkCacheContention-4       1000000          1276 ns/op          25 B/op  

Overlapping Interfaces¶

https://github.com/golang/proposal/blob/master/design/6977-overlapping-interfaces.md

1.14 加入的特性,解决钻石型组合可能导致的问题.

接口embedded时, 允许多个函数签名重复. 但是在接口里显式定义的函数, 还是必须保持唯一.

示例

package user

type Database interface {
    GetAccount(accountID uint64) (model.Account, error)
}

package device

type Database interface {
    user.Database
    SaveDevice(accountID uint64, device model.Device) error
}

package wallet

type Database interface {
    user.Database
    ReadWallet(accountID uint64) (model.Wallet, error)
}

package shopping

type Database interface {
    device.Database
    wallet.Database
    Buy(accountID uint64, deviceID uint64) error
}

mysql invalid connection¶

SetConnMaxLifeTime比mysql的wait_timeout大时.
客户端读超时设的太小时

go mod¶

生成依赖图¶

modv工具

go get github.com/poloxue/modv

生成png/svg

git:(refactor_usersvc) ✗ go mod graph | modv | dot -Tpng | open -f -a /Applications/Preview.app git:(refactor_usersvc) ✗ go mod graph | modv | dot -Tsvg > deps.svg

json¶

`RawMessage`¶

希望部分字段序列化时(例如广播时负载只序列化一次,放在每个用户的消息上).

精度丢失问题¶

json的数值类型默认是float64, 一个int64从一个object反序列化到一个map[string]interface{}时, 会变成float64, 丢掉精度.

解析时需要Decoder调用UseNumber解析成Number类型.

单测缓存问题¶

重复测试一个rpc, server端加了日志, 一直收不到. 但是单测能跑通, 能拿到结果, 结果一直不改.

换了一个测试输入, 有日志了. 发现是缓存的结果.

go.1.11以上, 可以使用--count=1禁掉缓存.

zeus@test01:~/cy/pkkr$ go test -v -timeout 30s -run ^TestLogin pkkr/cmd/authsvc/pokekara
2022/05/12 02:49:31 PASS init SnowFlakeIDNode
=== RUN   TestLogin
2022/05/12 02:49:31 account:<uid:"u1524446621015552000" platform:10 platform_uid:"aws-test-id02" token:"{\"aws_id\":\"aws-test-id02\",\"nickname\":\"aws-man01\",\"pk\":\"\"}" secret:"8bee60ee0ff0b3d98a9b5ef4dbe8a6d8f8d2d8618c496cce8de602b46a79c3ebcc75d1b0c04237324b358d7c6fd7898f4c3e2891d9721c26e177e87ef324bc8e" avatar_url:"pokekara/image/16182107407fb3a8c8df554f6a5598e6d02bfdacc7.jpg" ctime:1652291371 mtime:1652291371 > user:<uid:"u1524446621015552000" large_image_uri:"pokekara/image/16182107407fb3a8c8df554f6a5598e6d02bfdacc7.jpg" avatar_uri:"pokekara/image/16182107407fb3a8c8df554f6a5598e6d02bfdacc7.jpg" last_login_time:1652291371 display_id:2080716487 unique_device_id:"test-firetv-login" platform:10 ctime:1652291371 platform_uid:"aws-test-id02" avatar_album_uri:"pokekara/image/16182107407fb3a8c8df554f6a5598e6d02bfdacc7.jpg" > extra:<is_new_user:true >
--- PASS: TestLogin (0.02s)
PASS
ok  	pkkr/cmd/authsvc/pokekara	0.051s

zeus@test01:~/cy/pkkr$ go test --count=1 -v -timeout 30s -run ^TestLogin pkkr/cmd/authsvc/pokekara
2022/05/12 02:50:45 PASS init SnowFlakeIDNode
=== RUN   TestLogin
2022/05/12 02:50:45 account:<id:57237 uid:"u1524446621015552000" platform:10 platform_uid:"aws-test-id02" token:"{\"aws_id\":\"aws-test-id02\",\"nickname\":\"aws-man01\",\"pk\":\"\"}" secret:"8bee60ee0ff0b3d98a9b5ef4dbe8a6d8f8d2d8618c496cce8de602b46a79c3ebcc75d1b0c04237324b358d7c6fd7898f4c3e2891d9721c26e177e87ef324bc8e" avatar_url:"pokekara/image/16182107407fb3a8c8df554f6a5598e6d02bfdacc7.jpg" ctime:1652291372 mtime:1652291372 > user:<uid:"u1524446621015552000" large_image_uri:"pokekara/image/16182107407fb3a8c8df554f6a5598e6d02bfdacc7.jpg" avatar_uri:"pokekara/image/16182107407fb3a8c8df554f6a5598e6d02bfdacc7.jpg" last_login_time:1652291371 display_id:2080716487 unique_device_id:"test-firetv-login" platform:10 ctime:1652291371 platform_uid:"aws-test-id02" avatar_album_uri:"pokekara/image/16182107407fb3a8c8df554f6a5598e6d02bfdacc7.jpg" > extra:<>
--- PASS: TestLogin (0.02s)
PASS
ok  	pkkr/cmd/authsvc/pokekara	0.050s

Contents

Previous topic

Next topic

This Page

Golang杂项¶

内存不返还问题¶

二进制包大小¶

逃逸分析¶

Bounds Checking Elimination(BCE)¶

工具¶

观察bce¶

关掉bound-checking¶

锁争用开销分析¶

Overlapping Interfaces¶

mysql invalid connection¶

go mod¶

生成依赖图¶

json¶

`RawMessage`¶

精度丢失问题¶

单测缓存问题¶

Contents

Previous topic

Next topic

This Page

Golang杂项¶

内存不返还问题¶

二进制包大小¶

逃逸分析¶

Bounds Checking Elimination(BCE)¶

工具¶

观察bce¶

关掉bound-checking¶

锁争用开销分析¶

Overlapping Interfaces¶

mysql invalid connection¶

go mod¶

生成依赖图¶

json¶

RawMessage¶

精度丢失问题¶

单测缓存问题¶

`RawMessage`¶