Full Report
Golang's parsing for JSON, XML, and YAML has some peculiar properties that the author of this post decided to investigate. When unmarshalling JSON, the fields in Golang can be explicitly set with the settings for decoding. For instance: type User struct { Username string `json:"username_json_key,omitempty"` Password string `json:"password"` IsAdmin bool `json:"is_admin"` } If the json: string is not included, then Golang will still unmarshal it to the exact name of the field - in this case Username. A less-senior developer may not know this and assume that a field without the `json:"is_admin"` cannot be set at all. To actually tell the parser to skip something - text can be used. There's a funny quirk about this though! If - is used with any other data then the parser will assume that - is the literal field name! For instance, the definition `json:"-,omitempty"`. The author found two occurrences of this that they reported as vulnerabilities, and several thousand are currently on GitHub. Another misuse is setting omitempty in the JSON as the keyname instead of a setting. Both of these can be trivially found with semgrep rules. The next class of issues revolves around parser differentials. They label several common issues of misuse: duplicate fields, and case-insensitive key matching. This mostly applies when parsing data in one language and then having it be processed by another. The final bug class is data format confusion. In some cases, parsers are too lenient and try to get valid data out of whatever you need. The example they use is parsing a JSON file with an XML parser. In the case of Hashicorp, Felix from P0 found that they could smuggle in XML to an endpoint intended for JSON. By doing this, the controlled XML was processed instead of the legitimate one. Eventually this led to an auth bypass. The XML parser will accept leading or trailing garbage data. All of the parsers in Golang will accept unknown keys that don't match the struct. Although this doesn't have an impact by itself, it helps construct malicious payloads when exchanged between parsers. Overall, a good post into the weirdness of parsing libraries in Golang.
Analysis Summary