Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Sha3 as alternative #2480

Merged
merged 10 commits into from
Aug 13, 2020
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/sources/clients/promtail/stages/replace.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,25 +128,25 @@ The log line would become

### With `replace` value in `template` format with hashing for obfuscating data

To obfuscate sensitive data, you can combine the `replace` stage with the `Sha256` template method
To obfuscate sensitive data, you can combine the `replace` stage with the `Hash` or `Sha2Hash` template method.
wardbekker marked this conversation as resolved.
Show resolved Hide resolved

```yaml
- replace:
# SSN
expression: '([0-9]{3}-[0-9]{2}-[0-9]{4})'
replace: '*SSN*{{ .Value | Sha256 "salt" }}*'
replace: '*SSN*{{ .Value | Hash "salt" }}*'
- replace:
# IP4
expression: '(\d{1,3}[.]\d{1,3}[.]\d{1,3}[.]\d{1,3})'
replace: '*IP4*{{ .Value | Sha256 "salt" }}*'
replace: '*IP4*{{ .Value | Hash "salt" }}*'
- replace:
# email
expression: '([\w\.=-]+@[\w\.-]+\.[\w]{2,64})'
replace: '*email*{{ .Value | Sha256 "salt" }}*'
replace: '*email*{{ .Value | Hash "salt" }}*'
- replace:
# creditcard
expression: '((?:\d[ -]*?){13,16})'
replace: '*creditcard*{{ .Value | Sha256 "salt" }}*'
replace: '*creditcard*{{ .Value | Hash "salt" }}*'
```

### `replace` with named captured group
Expand Down
14 changes: 11 additions & 3 deletions docs/sources/clients/promtail/stages/template.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,12 +179,20 @@ and trailing white space removed, as defined by Unicode.
template: '{{ regexReplaceAllLiteral "(ts=)" .Value "timestamp=" }}'
```

### Sha256
### Hash and Sha2Hash

`Sha256` returns a Sha256 hash of the string, represented as a hexadecimal number of 64 digits. It can be used to obfuscate sensitive data / PII in the logs. It requires a (fixed) salt value, to add complexity to low input domains (e.g. all possible Social Security Numbers)
`Hash` returns a Sha3_256 hash of the string, represented as a hexadecimal number of 64 digits. You can use it to obfuscate sensitive data / PII in the logs. It requires a (fixed) salt value, to add complexity to low input domains (e.g. all possible Social Security Numbers).

```yaml
- template:
source: output
template: '{{ Sha256 .Value "salt" }}'
template: '{{ Hash .Value "salt" }}'
```

Alternatively, you can use `Sha2Hash` for calculating the Sha2_256 of the string.
wardbekker marked this conversation as resolved.
Show resolved Hide resolved

```yaml
- template:
source: output
template: '{{ Sha2Hash .Value "salt" }}'
```
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ require (
github.com/weaveworks/common v0.0.0-20200625145055-4b1847531bc9
go.etcd.io/bbolt v1.3.5-0.20200615073812-232d8fc87f50
go.uber.org/atomic v1.6.0
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
golang.org/x/net v0.0.0-20200707034311-ab3426394381
google.golang.org/grpc v1.30.0
gopkg.in/alecthomas/kingpin.v2 v2.2.6
Expand Down
10 changes: 8 additions & 2 deletions pkg/logentry/stages/template.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ import (
"github.com/go-kit/kit/log/level"
"github.com/mitchellh/mapstructure"
"github.com/prometheus/common/model"

"golang.org/x/crypto/sha3"
)

// Config Errors
Expand All @@ -34,8 +36,12 @@ var (
"TrimPrefix": strings.TrimPrefix,
"TrimSuffix": strings.TrimSuffix,
"TrimSpace": strings.TrimSpace,
"Sha256": func(salt string, s string) string {
hash := sha256.Sum256([]byte(salt + s))
"Hash": func(salt string, input string) string {
hash := sha3.Sum256([]byte(salt + input))
return hex.EncodeToString(hash[:])
},
"Sha2Hash": func(salt string, input string) string {
hash := sha256.Sum256([]byte(salt + input))
return hex.EncodeToString(hash[:])
},
"regexReplaceAll": func(regex string, s string, repl string) string {
Expand Down
16 changes: 14 additions & 2 deletions pkg/logentry/stages/template_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -342,10 +342,10 @@ func TestTemplateStage_Process(t *testing.T) {
map[string]interface{}{},
map[string]interface{}{},
},
"Sha256": {
"Sha2Hash": {
TemplateConfig{
Source: "testval",
Template: "{{ Sha256 .Value \"salt\" }}",
Template: "{{ Sha2Hash .Value \"salt\" }}",
},
map[string]interface{}{
"testval": "this is PII data",
Expand All @@ -354,6 +354,18 @@ func TestTemplateStage_Process(t *testing.T) {
"testval": "5526fd6f8ad457279cf8ff06453c6cb61bf479fa826e3b099caa6c846f9376f2",
},
},
"Hash": {
TemplateConfig{
Source: "testval",
Template: "{{ Hash .Value \"salt\" }}",
},
map[string]interface{}{
"testval": "this is PII data",
},
map[string]interface{}{
"testval": "0807ea24e992127128b38e4930f7155013786a4999c73a25910318a793847658",
},
},
}
for name, test := range tests {
test := test
Expand Down
66 changes: 66 additions & 0 deletions vendor/golang.org/x/crypto/sha3/doc.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

97 changes: 97 additions & 0 deletions vendor/golang.org/x/crypto/sha3/hashes.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 27 additions & 0 deletions vendor/golang.org/x/crypto/sha3/hashes_generic.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading